2020-11-20T11:52:04.394 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-20T11:52:04.396 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --name pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi --archive /home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640321 --lock --description fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} --block -- /tmp/teuthology-worker.os789h44.tmp 2020-11-20T11:52:04.437 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T11:52:04.504 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640321 branch: wip-pdonnell-testing-20201120.053228 description: fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} email: pdonnell@redhat.com first_in_suite: false job_id: '5640321' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: '8.1' overrides: admin_socket: branch: wip-pdonnell-testing-20201120.053228 ceph: cephfs: session_timeout: 300 cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(MON_DOWN\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20201120.053228 sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb owner: scheduled_pdonnell@teuthology priority: 95 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb sleep_before_teardown: 0 suite: fs suite_branch: wip-pdonnell-testing-20201120.053228 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - fs/snaps teuthology_branch: master timestamp: 2020-11-20_08:39:42 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.6017 2020-11-20T11:52:04.505 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa; will attempt to use it 2020-11-20T11:52:04.506 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa/tasks 2020-11-20T11:52:04.506 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-20T11:52:04.508 INFO:teuthology.task.internal:Checking packages... 2020-11-20T11:52:04.527 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'b86e691bf191566922a8f08e2a5abf64fde8cfcb' 2020-11-20T11:52:04.528 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-20T11:52:04.528 INFO:teuthology.packaging:ref: None 2020-11-20T11:52:04.529 INFO:teuthology.packaging:tag: None 2020-11-20T11:52:04.529 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20201120.053228 2020-11-20T11:52:04.529 INFO:teuthology.packaging:sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T11:52:04.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20201120.053228 2020-11-20T11:52:04.697 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-7397.gb86e691b 2020-11-20T11:52:04.697 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-20T11:52:04.707 INFO:teuthology.task.internal:no buildpackages task found 2020-11-20T11:52:04.707 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-20T11:52:04.747 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-20T11:52:04.748 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T11:52:04.804 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:52:14.832 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:52:24.857 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:52:34.892 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pdonnell@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640321'} 2020-11-20T11:52:35.131 DEBUG:teuthology.lock.ops:locked ubuntu@smithi044.front.sepia.ceph.com, ubuntu@smithi079.front.sepia.ceph.com 2020-11-20T11:52:35.198 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f02f8a21bf8>, signals=[15]) 2020-11-20T11:52:35.410 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of centos 8.1 2020-11-20T11:52:35.430 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 8.1 2020-11-20T11:52:35.833 INFO:teuthology.orchestra.console:Power off smithi079 2020-11-20T11:52:35.834 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T11:52:35.861 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:35.896 INFO:teuthology.orchestra.console:Power off smithi044 2020-11-20T11:52:35.898 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T11:52:35.966 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:39.890 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:39.991 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:44.005 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:44.102 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:44.219 INFO:teuthology.orchestra.console:Power off for smithi079 completed 2020-11-20T11:52:44.314 INFO:teuthology.orchestra.console:Power off for smithi044 completed 2020-11-20T11:52:44.320 INFO:teuthology.orchestra.console:Power on smithi079 2020-11-20T11:52:44.320 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T11:52:44.341 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:44.416 INFO:teuthology.orchestra.console:Power on smithi044 2020-11-20T11:52:44.417 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T11:52:44.438 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:48.366 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:48.462 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:52.545 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:52.574 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T11:52:52.789 INFO:teuthology.orchestra.console:Power on for smithi044 completed 2020-11-20T11:52:52.807 INFO:teuthology.orchestra.console:Power on for smithi079 completed 2020-11-20T11:52:52.893 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish 2020-11-20T11:52:52.908 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish 2020-11-20T11:55:09.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:19.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:25.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:28.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:34.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:37.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:43.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:46.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:52.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:55:55.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:56:01.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:56:10.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:56:19.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:56:27.261 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T11:56:27.263 INFO:teuthology.orchestra.run.smithi044:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T11:56:28.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:56:43.010 INFO:teuthology.provision.fog.smithi044:Node is ready 2020-11-20T11:56:43.011 INFO:teuthology.orchestra.run.smithi044:> hostname 2020-11-20T11:56:43.033 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com 2020-11-20T11:56:43.034 INFO:teuthology.orchestra.run.smithi044:> grep smithi044.front.sepia.ceph.com /etc/hosts 2020-11-20T11:56:43.090 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044 2020-11-20T11:56:43.091 INFO:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2020-11-20T11:56:43.227 INFO:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hosts 2020-11-20T11:56:43.264 INFO:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/172.21.15.44/172.21.15.44/g' /etc/hosts 2020-11-20T11:56:43.350 INFO:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hostname 2020-11-20T11:56:43.434 INFO:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2020-11-20T11:56:43.510 INFO:teuthology.provision.fog.smithi044:Deploy complete! 2020-11-20T11:56:43.511 DEBUG:teuthology.parallel:result is None 2020-11-20T11:57:34.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:57:36.506 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T11:57:36.506 INFO:teuthology.orchestra.run.smithi079:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T11:57:36.822 INFO:teuthology.provision.fog.smithi079:Node is ready 2020-11-20T11:57:36.822 INFO:teuthology.orchestra.run.smithi079:> hostname 2020-11-20T11:57:36.841 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com 2020-11-20T11:57:36.842 INFO:teuthology.orchestra.run.smithi079:> grep smithi079.front.sepia.ceph.com /etc/hosts 2020-11-20T11:57:36.897 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079 2020-11-20T11:57:36.898 INFO:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2020-11-20T11:57:37.009 INFO:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hosts 2020-11-20T11:57:37.038 INFO:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/172.21.15.79/172.21.15.79/g' /etc/hosts 2020-11-20T11:57:37.107 INFO:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hostname 2020-11-20T11:57:37.175 INFO:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2020-11-20T11:57:37.243 INFO:teuthology.provision.fog.smithi079:Deploy complete! 2020-11-20T11:57:37.246 DEBUG:teuthology.parallel:result is None 2020-11-20T11:57:37.391 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com 2020-11-20T11:57:37.392 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T11:57:37.411 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com 2020-11-20T11:57:37.412 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T11:57:37.426 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server 2020-11-20T11:57:37.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T11:57:37.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:57:37.499 INFO:teuthology.orchestra.run.smithi044:> true 2020-11-20T11:57:37.679 INFO:teuthology.orchestra.run.smithi044:> uname -m 2020-11-20T11:57:37.734 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2020-11-20T11:57:37.734 INFO:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2020-11-20T11:57:37.789 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Linux" 2020-11-20T11:57:37.789 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8 (Core)" 2020-11-20T11:57:37.790 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2020-11-20T11:57:37.790 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2020-11-20T11:57:37.790 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8" 2020-11-20T11:57:37.790 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2020-11-20T11:57:37.790 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T11:57:37.791 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2020-11-20T11:57:37.791 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T11:57:37.791 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T11:57:37.791 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T11:57:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T11:57:37.792 INFO:teuthology.orchestra.run.smithi044.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T11:57:37.792 INFO:teuthology.orchestra.run.smithi044.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T11:57:37.792 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T11:57:37.792 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T11:57:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T11:57:37.794 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2020-11-20T11:57:37.811 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server 2020-11-20T11:57:37.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T11:57:37.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:57:37.877 INFO:teuthology.orchestra.run.smithi079:> true 2020-11-20T11:57:38.034 INFO:teuthology.orchestra.run.smithi079:> uname -m 2020-11-20T11:57:38.088 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2020-11-20T11:57:38.089 INFO:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2020-11-20T11:57:38.143 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Linux" 2020-11-20T11:57:38.143 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8 (Core)" 2020-11-20T11:57:38.143 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2020-11-20T11:57:38.143 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2020-11-20T11:57:38.144 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8" 2020-11-20T11:57:38.144 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2020-11-20T11:57:38.144 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T11:57:38.144 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2020-11-20T11:57:38.144 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T11:57:38.145 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T11:57:38.145 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T11:57:38.145 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T11:57:38.145 INFO:teuthology.orchestra.run.smithi079.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T11:57:38.145 INFO:teuthology.orchestra.run.smithi079.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T11:57:38.146 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T11:57:38.146 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T11:57:38.146 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T11:57:38.161 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2020-11-20T11:57:38.181 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-11-20T11:57:38.223 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi044.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCl10bOspedT7YS1oXeglqpT7ktwelK/Q86t0MBR9xm+UURVA6meBVFcIoDhear3I1rgltd1bdrxS9r9Hfsa1bQKI3WlFhgTKTiP8NF79KRFkLSiipP/SvWhvB6CaHD8N2nsJRAEUOwCoN+6RveN8CwAlai2yd2peVAn52cTAWTdd95pEBQOTcf78KnLJeImNAnNW10ohBwCETu7hOtSk4jXtCZQ83DrMmNB5qTTXjIB51nveB1JlDBR+YRjP0EMOH6fAmaYXTghScevPj9qBJKfUR39OHhur6G0iFOWE7mvIlqzzGpxQgnmC4EqDIJDWzdZSYZ1IzXVAC/K7DtvdLI1cQ0upAxk+PS+vd0K28v3MfKVP5Kf768yi3BkHgJjJ4xLjT3c5ksIR9B90CTp5kiKubasFDtjLkqdvkVHpECHsW0LfiouJFbi0C6eFNRIK/g54sxGy41bn9UnWkKeYFpFtsI8SWquEWHmSeqwQv2nXbirdng5mgQ7IDj3luMCoc= smithi079.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDqNQM/Nvn6E1TgaiyIBJo1T3KqoP8jOjDbIL5PQZ5jSXzxK+uhcb7SJn8FlZzY809zObUtoCCy25mGYaJOhe+LHjXt+tiYRYH7jmRXbRiqOhH5cuqLY1MV0c8qgsyTloDv89aRlcmF2VkORG7A+OCAVaMdU4q8FJskpgdcCE8pQtrInXJiqPtD8DcRsIc/JCpYAx/gzQxaax14KfpTqfkYIfQsSBhO02hh+HbleSUxuyNJ/v0okBwkeQcsJn63lit7vzabqMUwYntl37jGXnelZvcyxZbXg9xCGwAvpp1RT/Wi3DjlUO4Hfm9z518KHeXPomVzFK1GuUXmWbDGwmv/N7vTlLOQVOASjeTkORatueGvn0KR4oxeDPC5ukZU5a/tyVoIHC76BRaUmTSkFHVDjlF9Zj64wvAs112rx/bY0FYbzFeFhRzXWUOqSg3xW2CH6LbZXMid41VxpcDSqIIV37+FpeSacnbHemSY4wCa4AGazHRX1LK34C83+xcr0GM= 2020-11-20T11:57:38.223 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T11:57:38.281 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-20T11:57:38.290 INFO:teuthology.task.internal:Saving configuration 2020-11-20T11:57:38.308 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-20T11:57:38.321 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-20T11:57:38.339 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 11:52:35.115764', 'description': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640321', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFjkvMLTIy651AXAvVTrPWfTflsWJGalh//fU7HYCfgt6kZ//rz/khQ0N+O/UqBhmVYfjWp8/v+xriQ1YKmkais=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi044.front.sepia.ceph.com'} 2020-11-20T11:57:38.365 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 11:52:35.115818', 'description': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640321', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRqw3PViJCCjqgy0hQgvTOM0lkjsoGvnHxEU5J2JXy13G7iQ3x4sea1VaL59Rz6oAsOJmJ89ERU/w31E24HWjE=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi079.front.sepia.ceph.com'} 2020-11-20T11:57:38.366 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-20T11:57:38.381 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2020-11-20T11:57:38.382 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2020-11-20T11:57:38.383 INFO:teuthology.run_tasks:Running task console_log... 2020-11-20T11:57:38.503 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f02fc526048>, signals=[15]) 2020-11-20T11:57:38.504 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-20T11:57:38.522 INFO:teuthology.task.internal:Opening connections... 2020-11-20T11:57:38.523 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2020-11-20T11:57:38.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:57:38.664 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2020-11-20T11:57:38.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T11:57:38.794 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-20T11:57:38.808 INFO:teuthology.orchestra.run.smithi044:> uname -m 2020-11-20T11:57:38.829 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2020-11-20T11:57:38.830 INFO:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2020-11-20T11:57:38.886 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Linux" 2020-11-20T11:57:38.886 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8 (Core)" 2020-11-20T11:57:38.886 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2020-11-20T11:57:38.886 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2020-11-20T11:57:38.887 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8" 2020-11-20T11:57:38.887 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2020-11-20T11:57:38.887 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T11:57:38.887 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2020-11-20T11:57:38.887 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T11:57:38.888 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T11:57:38.888 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T11:57:38.888 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T11:57:38.888 INFO:teuthology.orchestra.run.smithi044.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T11:57:38.888 INFO:teuthology.orchestra.run.smithi044.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T11:57:38.890 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T11:57:38.890 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T11:57:38.890 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T11:57:38.891 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2020-11-20T11:57:38.915 INFO:teuthology.orchestra.run.smithi079:> uname -m 2020-11-20T11:57:38.932 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2020-11-20T11:57:38.933 INFO:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2020-11-20T11:57:38.987 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Linux" 2020-11-20T11:57:38.994 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8 (Core)" 2020-11-20T11:57:38.994 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2020-11-20T11:57:38.994 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2020-11-20T11:57:38.995 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8" 2020-11-20T11:57:38.995 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2020-11-20T11:57:38.995 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T11:57:38.995 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2020-11-20T11:57:38.995 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T11:57:38.996 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T11:57:38.996 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T11:57:38.996 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T11:57:38.997 INFO:teuthology.orchestra.run.smithi079.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T11:57:38.997 INFO:teuthology.orchestra.run.smithi079.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T11:57:38.997 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T11:57:38.997 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T11:57:38.997 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T11:57:38.998 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2020-11-20T11:57:39.019 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-20T11:57:39.050 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-20T11:57:39.085 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-20T11:57:39.086 INFO:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2020-11-20T11:57:39.092 INFO:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2020-11-20T11:57:39.107 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-20T11:57:39.119 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-20T11:57:39.120 INFO:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2020-11-20T11:57:39.146 INFO:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2020-11-20T11:57:39.174 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-20T11:57:39.241 INFO:teuthology.run_tasks:Running task kernel... 2020-11-20T11:57:39.264 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}} 2020-11-20T11:57:39.265 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-11-20T11:57:39.265 INFO:teuthology.orchestra.run.smithi044:> uname -r 2020-11-20T11:57:39.282 INFO:teuthology.orchestra.run.smithi044.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-20T11:57:39.283 INFO:teuthology.task.kernel:Running kernel on smithi044: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T11:57:39.283 INFO:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2020-11-20T11:57:41.939 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-8 - AppStream 37 MB/s | 7.0 MB 00:00 2020-11-20T11:57:43.188 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-20T11:57:43.637 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-8 - Extras 83 kB/s | 5.5 kB 00:00 2020-11-20T11:57:44.039 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 24 MB/s | 8.3 MB 00:00 2020-11-20T11:57:45.853 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 5.3 MB/s | 23 kB 00:00 2020-11-20T11:57:46.830 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-20T11:57:46.831 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-20T11:57:46.993 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2020-11-20T11:57:46.994 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2020-11-20T11:57:46.995 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2020-11-20T11:57:47.120 INFO:teuthology.orchestra.run.smithi044:> echo no | sudo yum reinstall kernel || true 2020-11-20T11:57:47.555 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 11:57:45 AM UTC. 2020-11-20T11:57:47.840 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T11:57:47.841 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T11:57:47.887 INFO:teuthology.orchestra.run.smithi044.stderr:Error: No packages marked for reinstall. 2020-11-20T11:57:47.928 INFO:teuthology.orchestra.run.smithi044:> sudo yum reinstall -y kernel || true 2020-11-20T11:57:48.359 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Nov 2020 11:57:45 AM UTC. 2020-11-20T11:57:48.620 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T11:57:48.620 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T11:57:48.681 INFO:teuthology.orchestra.run.smithi044.stderr:Error: No packages marked for reinstall. 2020-11-20T11:57:48.736 INFO:teuthology.orchestra.run.smithi044:> rpm -q kernel --last | head -n 1 2020-11-20T11:57:48.801 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-20T11:57:48.802 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T11:57:48.803 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-20T11:57:48.803 INFO:teuthology.orchestra.run.smithi079:> uname -r 2020-11-20T11:57:48.820 INFO:teuthology.orchestra.run.smithi079.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-20T11:57:48.820 INFO:teuthology.task.kernel:Running kernel on smithi079: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T11:57:48.820 INFO:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2020-11-20T11:57:51.463 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-8 - AppStream 33 MB/s | 7.0 MB 00:00 2020-11-20T11:57:52.673 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-20T11:57:53.105 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-8 - Extras 81 kB/s | 5.5 kB 00:00 2020-11-20T11:57:53.480 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 25 MB/s | 8.3 MB 00:00 2020-11-20T11:57:55.221 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 5.3 MB/s | 23 kB 00:00 2020-11-20T11:57:56.187 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-20T11:57:56.534 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-20T11:57:56.535 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2020-11-20T11:57:56.535 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2020-11-20T11:57:56.535 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2020-11-20T11:57:56.536 INFO:teuthology.orchestra.run.smithi079:> echo no | sudo yum reinstall kernel || true 2020-11-20T11:57:56.944 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Nov 2020 11:57:55 AM UTC. 2020-11-20T11:57:57.193 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T11:57:57.194 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T11:57:57.269 INFO:teuthology.orchestra.run.smithi079.stderr:Error: No packages marked for reinstall. 2020-11-20T11:57:57.349 INFO:teuthology.orchestra.run.smithi079:> sudo yum reinstall -y kernel || true 2020-11-20T11:57:57.752 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 11:57:55 AM UTC. 2020-11-20T11:57:58.018 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T11:57:58.018 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T11:57:58.066 INFO:teuthology.orchestra.run.smithi079.stderr:Error: No packages marked for reinstall. 2020-11-20T11:57:58.115 INFO:teuthology.orchestra.run.smithi079:> rpm -q kernel --last | head -n 1 2020-11-20T11:57:58.185 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-20T11:57:58.186 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T11:57:58.187 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-20T11:57:58.187 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-11-20T11:57:58.188 INFO:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-20T11:57:58.240 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2020-11-20T11:57:58.256 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-11-20T11:57:58.257 INFO:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-20T11:57:58.295 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2020-11-20T11:57:58.308 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-20T11:57:58.325 INFO:teuthology.task.internal:Creating test directory... 2020-11-20T11:57:58.325 INFO:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T11:57:58.328 INFO:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T11:57:58.362 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-20T11:57:58.382 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-20T11:57:58.397 INFO:teuthology.task.internal:Creating archive directory... 2020-11-20T11:57:58.398 INFO:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T11:57:58.401 INFO:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T11:57:58.438 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-20T11:57:58.447 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-20T11:57:58.448 INFO:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T11:57:58.481 INFO:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T11:57:58.510 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T11:57:58.523 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T11:57:58.541 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T11:57:58.559 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T11:57:58.560 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-20T11:57:58.568 INFO:teuthology.task.internal:Configuring sudo... 2020-11-20T11:57:58.569 INFO:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T11:57:58.571 INFO:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T11:57:58.602 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-20T11:57:58.618 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-20T11:57:58.619 INFO:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T11:57:58.644 INFO:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T11:57:58.659 INFO:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T11:57:58.731 INFO:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T11:57:58.793 INFO:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T11:57:58.868 INFO:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T11:57:58.922 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T11:57:58.923 INFO:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T11:57:58.980 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T11:57:58.996 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T11:57:58.996 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T11:57:58.997 INFO:teuthology.orchestra.run.smithi044.stderr:152 bytes copied, 3.1619e-05 s, 4.8 MB/s 2020-11-20T11:57:58.998 INFO:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T11:57:59.036 INFO:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T11:57:59.112 INFO:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T11:57:59.146 INFO:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T11:57:59.195 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T11:57:59.195 INFO:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T11:57:59.247 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T11:57:59.260 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T11:57:59.261 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T11:57:59.261 INFO:teuthology.orchestra.run.smithi079.stderr:152 bytes copied, 2.6913e-05 s, 5.6 MB/s 2020-11-20T11:57:59.262 INFO:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2020-11-20T11:57:59.267 INFO:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2020-11-20T11:57:59.330 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-20T11:57:59.369 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-20T11:57:59.774 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-20T11:57:59.789 INFO:teuthology.task.internal:Starting timer... 2020-11-20T11:57:59.790 INFO:teuthology.run_tasks:Running task pcp... 2020-11-20T11:57:59.837 INFO:teuthology.run_tasks:Running task selinux... 2020-11-20T11:57:59.887 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-11-20T11:57:59.888 INFO:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2020-11-20T11:57:59.954 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: [ OK ] 2020-11-20T11:57:59.955 INFO:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2020-11-20T11:58:00.013 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: [ OK ] 2020-11-20T11:58:00.014 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-20T11:58:00.015 INFO:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2020-11-20T11:58:00.034 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2020-11-20T11:58:00.035 INFO:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2020-11-20T11:58:00.059 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2020-11-20T11:58:00.060 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi079.front.sepia.ceph.com': 'permissive'} 2020-11-20T11:58:00.060 INFO:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-20T11:58:00.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T11:58:00.108 INFO:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-20T11:58:00.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T11:58:00.140 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-20T11:58:00.140 INFO:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2020-11-20T11:58:00.179 INFO:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2020-11-20T11:58:00.211 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-20T11:58:00.221 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-11-20T11:58:00.222 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-20T11:58:00.234 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-20T11:58:00.235 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi079.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-20T11:58:00.640 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-20T11:58:01.525 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-20T11:58:04.128 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-20T11:58:04.147 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:04 +0000 (0:00:00.071) 0:00:00.071 ******* 2020-11-20T11:58:04.160 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:04 +0000 (0:00:00.013) 0:00:00.084 ******* 2020-11-20T11:58:05.614 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi079.front.sepia.ceph.com] 2020-11-20T11:58:05.656 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] 2020-11-20T11:58:05.720 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:05 +0000 (0:00:01.557) 0:00:01.642 ******* 2020-11-20T11:58:05.761 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:05 +0000 (0:00:00.042) 0:00:01.685 ******* 2020-11-20T11:58:06.041 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-20T11:58:06.044 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T11:58:06.074 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:06 +0000 (0:00:00.312) 0:00:01.998 ******* 2020-11-20T11:58:06.080 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T11:58:06.117 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:06 +0000 (0:00:00.042) 0:00:02.040 ******* 2020-11-20T11:58:06.518 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** ok: [smithi079.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} ok: [smithi044.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-20T11:58:07.211 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:06 +0000 (0:00:00.435) 0:00:02.476 ******* Friday 20 November 2020 11:58:06 +0000 (0:00:00.034) 0:00:02.510 ******* TASK [ansible-managed : Delete the ansible users password.] ******************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:58:06 +0000 (0:00:00.059) 0:00:02.569 ******* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:58:06 +0000 (0:00:00.086) 0:00:02.656 ******* TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi079.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} Friday 20 November 2020 11:58:07 +0000 (0:00:00.288) 0:00:02.945 ******* ok: [smithi044.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} Friday 20 November 2020 11:58:07 +0000 (0:00:00.043) 0:00:02.988 ******* 2020-11-20T11:58:08.095 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605873487.080782-178903669374974/source", "state": "file", "uid": 0} changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605873487.037252-186330406128096/source", "state": "file", "uid": 0} 2020-11-20T11:58:08.129 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:08 +0000 (0:00:01.063) 0:00:04.051 ******* 2020-11-20T11:58:08.160 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:08 +0000 (0:00:00.031) 0:00:04.083 ******* 2020-11-20T11:58:08.534 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-20T11:58:08.536 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T11:58:08.538 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T11:58:08.572 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T11:58:08.574 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T11:58:08.689 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-11-20T11:58:08.698 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-11-20T11:58:08.857 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:08 +0000 (0:00:00.694) 0:00:04.778 ******* 2020-11-20T11:58:08.885 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:08 +0000 (0:00:00.030) 0:00:04.809 ******* 2020-11-20T11:58:09.512 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-11-20T11:58:09.514 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] 2020-11-20T11:58:09.522 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] 2020-11-20T11:58:09.621 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-20T11:58:09.622 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:09 +0000 (0:00:00.736) 0:00:05.545 ******* 2020-11-20T11:58:09.655 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:09 +0000 (0:00:00.034) 0:00:05.579 ******* 2020-11-20T11:58:10.103 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-20T11:58:10.104 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.184373", "end": "2020-11-20 11:58:10.076091", "rc": 0, "start": "2020-11-20 11:58:09.891718", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} changed: [smithi044.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.178089", "end": "2020-11-20 11:58:10.078020", "rc": 0, "start": "2020-11-20 11:58:09.899931", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T11:58:10.137 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.481) 0:00:06.061 ******* 2020-11-20T11:58:10.167 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.030) 0:00:06.091 ******* 2020-11-20T11:58:10.384 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.023698", "end": "2020-11-20 11:58:10.361781", "failed_when_result": false, "rc": 0, "start": "2020-11-20 11:58:10.338083", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T11:58:10.418 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.251) 0:00:06.342 ******* 2020-11-20T11:58:10.457 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-20T11:58:10.492 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.073) 0:00:06.415 ******* 2020-11-20T11:58:10.518 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-20T11:58:10.520 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.573 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.081) 0:00:06.496 ******* 2020-11-20T11:58:10.603 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-20T11:58:10.605 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.658 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.085) 0:00:06.582 ******* 2020-11-20T11:58:10.687 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-20T11:58:10.689 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.720 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.062) 0:00:06.644 ******* 2020-11-20T11:58:10.747 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.782 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.060) 0:00:06.705 ******* 2020-11-20T11:58:10.807 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-20T11:58:10.809 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.841 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:10 +0000 (0:00:00.059) 0:00:06.765 ******* 2020-11-20T11:58:10.890 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-20T11:58:10.892 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.906 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.913 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.915 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.925 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.939 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.941 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.951 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.952 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.964 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.966 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.976 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.988 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:10.998 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.000 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.010 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.023 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.034 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.046 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.047 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.059 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.068 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.070 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.080 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.082 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.092 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.104 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.115 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.119 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.126 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.129 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.137 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.139 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.149 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.152 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.160 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.162 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.172 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.174 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.184 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.198 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.199 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.209 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.210 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.221 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.223 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.233 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.244 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.255 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.257 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.266 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.268 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.278 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.279 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.290 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.301 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.302 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.313 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.324 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.326 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.349 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.360 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.361 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:01.011381", "end": "2020-11-20 11:58:11.326194", "failed_when_result": false, "rc": 0, "start": "2020-11-20 11:58:10.314813", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T11:58:11.361 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.402 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.560) 0:00:07.325 ******* 2020-11-20T11:58:11.411 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f9 2020-11-20T11:58:11.416 INFO:teuthology.task.ansible.out:97d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac634 2020-11-20T11:58:11.418 INFO:teuthology.task.ansible.out:6cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.427 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-20T11:58:11.430 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.463 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.061) 0:00:07.387 ******* 2020-11-20T11:58:11.471 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.473 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.484 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.486 INFO:teuthology.task.ansible.out: skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.486 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.521 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.057) 0:00:07.444 ******* 2020-11-20T11:58:11.527 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-20T11:58:11.529 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.529 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.530 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thi 2020-11-20T11:58:11.531 INFO:teuthology.task.ansible.out:nkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.542 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.543 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ 2020-11-20T11:58:11.544 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.581 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.057) 0:00:07.501 ******* 2020-11-20T11:58:11.585 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.586 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.586 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.588 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.588 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.589 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.600 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.602 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.635 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.057) 0:00:07.559 ******* 2020-11-20T11:58:11.642 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.644 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.644 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.645 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansib 2020-11-20T11:58:11.648 INFO:teuthology.task.ansible.out:le_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.656 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.658 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.673 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-20T11:58:11.735 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:58:11 +0000 (0:00:00.071) 0:00:07.631 ******* 2020-11-20T11:58:11.738 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.031) 0:00:07.662 ******* 2020-11-20T11:58:11.746 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.797 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.057) 0:00:07.719 ******* 2020-11-20T11:58:11.808 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-20T11:58:11.840 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.044) 0:00:07.764 ******* 2020-11-20T11:58:11.853 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.859 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.863 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.873 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.915 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.074) 0:00:07.838 ******* 2020-11-20T11:58:11.917 INFO:teuthology.task.ansible.out: 2020-11-20T11:58:11.922 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.924 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.924 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAA 2020-11-20T11:58:11.926 INFO:teuthology.task.ansible.out:DAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "a 2020-11-20T11:58:11.927 INFO:teuthology.task.ansible.out:emerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.929 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.940 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.955 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d3430 2020-11-20T11:58:11.958 INFO:teuthology.task.ansible.out:0b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:11.994 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:11 +0000 (0:00:00.079) 0:00:07.918 ******* 2020-11-20T11:58:12.001 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.004 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.005 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.007 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d8 2020-11-20T11:58:12.008 INFO:teuthology.task.ansible.out:9bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ov 2020-11-20T11:58:12.010 INFO:teuthology.task.ansible.out:pn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.021 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.024 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.058 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.063) 0:00:07.982 ******* 2020-11-20T11:58:12.069 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJi 2020-11-20T11:58:12.075 INFO:teuthology.task.ansible.out:hFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.081 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.083 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.088 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.105 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.140 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.082) 0:00:08.064 ******* 2020-11-20T11:58:12.156 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'nam 2020-11-20T11:58:12.159 INFO:teuthology.task.ansible.out:e': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.164 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.181 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.218 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.073) 0:00:08.137 ******* 2020-11-20T11:58:12.222 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.228 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name 2020-11-20T11:58:12.233 INFO:teuthology.task.ansible.out:': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.239 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": 2020-11-20T11:58:12.245 INFO:teuthology.task.ansible.out:false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.249 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.250 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.261 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.265 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.277 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.288 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.290 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.301 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.303 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.321 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.323 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.335 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.372 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.386 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.389 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.391 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.393 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.402 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.405 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.406 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeit 2020-11-20T11:58:12.407 INFO:teuthology.task.ansible.out:hle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-11-20T11:58:12.410 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.420 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.425 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "ite 2020-11-20T11:58:12.428 INFO:teuthology.task.ansible.out:m", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 2020-11-20T11:58:12.432 INFO:teuthology.task.ansible.out:'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.438 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.453 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.465 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.467 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.482 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.491 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'r 2020-11-20T11:58:12.494 INFO:teuthology.task.ansible.out:akesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.505 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563 2020-11-20T11:58:12.508 INFO:teuthology.task.ansible.out:a02"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.519 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.522 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.533 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-20T11:58:12.539 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3db 2020-11-20T11:58:12.543 INFO:teuthology.task.ansible.out:e54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.589 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.372) 0:00:08.510 ******* 2020-11-20T11:58:12.602 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name 2020-11-20T11:58:12.605 INFO:teuthology.task.ansible.out:": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.615 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.629 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.642 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.644 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.653 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.655 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.674 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.682 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.694 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.700 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.745 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.158) 0:00:08.669 ******* 2020-11-20T11:58:12.768 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-20T11:58:12.811 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.064) 0:00:08.734 ******* 2020-11-20T11:58:12.839 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.877 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.065) 0:00:08.800 ******* 2020-11-20T11:58:12.905 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-20T11:58:12.907 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:12.938 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.062) 0:00:08.862 ******* 2020-11-20T11:58:12.967 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.000 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:12 +0000 (0:00:00.060) 0:00:08.923 ******* 2020-11-20T11:58:13.027 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-20T11:58:13.029 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.066 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:13 +0000 (0:00:00.066) 0:00:08.990 ******* 2020-11-20T11:58:13.094 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.129 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:13 +0000 (0:00:00.062) 0:00:09.053 ******* 2020-11-20T11:58:13.176 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi044.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.187 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.210 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.213 INFO:teuthology.task.ansible.out: 2020-11-20T11:58:13.224 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.227 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.250 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.251 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.251 INFO:teuthology.task.ansible.out: 2020-11-20T11:58:13.255 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.255 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.278 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.280 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.292 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.294 INFO:teuthology.task.ansible.out: skipping: [smithi044.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.296 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.307 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.318 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.320 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.330 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.332 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.342 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.344 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.354 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.355 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.366 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.378 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.390 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.393 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.403 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.414 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-20T11:58:13.416 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** skipping: [smithi044.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.458 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:13 +0000 (0:00:00.329) 0:00:09.382 ******* 2020-11-20T11:58:13.490 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:13 +0000 (0:00:00.031) 0:00:09.414 ******* 2020-11-20T11:58:13.518 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi079.front.sepia.ceph.com] => { "msg": "Host smithi079.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-20T11:58:13.552 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:13 +0000 (0:00:00.061) 0:00:09.475 ******* 2020-11-20T11:58:13.593 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-20T11:58:13.595 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/{{ container_mirror }}", "nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-20T11:58:13.650 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:13 +0000 (0:00:00.097) 0:00:09.573 ******* 2020-11-20T11:58:13.676 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:13.710 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:13 +0000 (0:00:00.060) 0:00:09.634 ******* 2020-11-20T11:58:13.961 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-20T11:58:14.008 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.297) 0:00:09.932 ******* 2020-11-20T11:58:14.039 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi044.front.sepia.ceph.com] => { "msg": "Host smithi044.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-20T11:58:14.085 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.076) 0:00:10.009 ******* 2020-11-20T11:58:14.124 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-20T11:58:14.126 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/{{ container_mirror }}", "nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-20T11:58:14.158 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.073) 0:00:10.082 ******* 2020-11-20T11:58:14.186 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.220 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.061) 0:00:10.143 ******* 2020-11-20T11:58:14.226 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-20T11:58:14.228 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.318144", "end": "2020-11-20 11:58:14.195030", "rc": 0, "start": "2020-11-20 11:58:13.876886", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T11:58:14.263 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.042) 0:00:10.186 ******* 2020-11-20T11:58:14.301 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.336 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.073) 0:00:10.260 ******* 2020-11-20T11:58:14.373 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-20T11:58:14.376 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.411 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.074) 0:00:10.335 ******* 2020-11-20T11:58:14.437 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-20T11:58:14.439 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.472 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.059) 0:00:10.394 ******* 2020-11-20T11:58:14.497 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-20T11:58:14.500 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.531 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.060) 0:00:10.455 ******* 2020-11-20T11:58:14.557 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.590 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.059) 0:00:10.514 ******* 2020-11-20T11:58:14.617 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-20T11:58:14.619 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.651 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.060) 0:00:10.575 ******* 2020-11-20T11:58:14.689 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi079.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:14.691 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:14.721 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.070) 0:00:10.645 ******* 2020-11-20T11:58:14.748 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-20T11:58:14.750 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.790 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.068) 0:00:10.714 ******* 2020-11-20T11:58:14.797 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-20T11:58:14.798 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.343400", "end": "2020-11-20 11:58:14.741549", "rc": 0, "start": "2020-11-20 11:58:14.398149", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T11:58:14.835 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.044) 0:00:10.758 ******* 2020-11-20T11:58:14.841 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-20T11:58:14.843 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.880 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.043) 0:00:10.802 ******* 2020-11-20T11:58:14.884 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-20T11:58:14.886 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:14.923 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.044) 0:00:10.847 ******* 2020-11-20T11:58:14.932 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi079.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:14.967 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:14 +0000 (0:00:00.043) 0:00:10.890 ******* 2020-11-20T11:58:14.976 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.009 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.043) 0:00:10.933 ******* 2020-11-20T11:58:15.017 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.060 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.048) 0:00:10.982 ******* 2020-11-20T11:58:15.066 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.103 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.044) 0:00:11.026 ******* 2020-11-20T11:58:15.112 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi079.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:15.146 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.043) 0:00:11.069 ******* 2020-11-20T11:58:15.152 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-20T11:58:15.154 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.188 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.042) 0:00:11.112 ******* 2020-11-20T11:58:15.197 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.230 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.042) 0:00:11.154 ******* 2020-11-20T11:58:15.237 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-20T11:58:15.238 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.279 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.049) 0:00:11.203 ******* 2020-11-20T11:58:15.292 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.324 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.044) 0:00:11.248 ******* 2020-11-20T11:58:15.333 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.365 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.041) 0:00:11.289 ******* 2020-11-20T11:58:15.375 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.044) 0:00:11.333 ******* 2020-11-20T11:58:15.417 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi044.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi044.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:15.452 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.041) 0:00:11.375 ******* 2020-11-20T11:58:15.458 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-20T11:58:15.461 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.497 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.045) 0:00:11.420 ******* 2020-11-20T11:58:15.508 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-20T11:58:15.510 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.548 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.047) 0:00:11.468 ******* 2020-11-20T11:58:15.553 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi079.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:15.613 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.066) 0:00:11.535 ******* 2020-11-20T11:58:15.617 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-20T11:58:15.620 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.655 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.043) 0:00:11.578 ******* 2020-11-20T11:58:15.662 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.698 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.043) 0:00:11.621 ******* 2020-11-20T11:58:15.706 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-20T11:58:15.707 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:15.743 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.045) 0:00:11.667 ******* 2020-11-20T11:58:15.750 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.785 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.041) 0:00:11.709 ******* 2020-11-20T11:58:15.792 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.835 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.043) 0:00:11.752 ******* TASK [common : Store list of enabled repos] ************************************ 2020-11-20T11:58:15.841 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.885 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.054) 0:00:11.807 ******* 2020-11-20T11:58:15.950 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi044.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Friday 20 November 2020 11:58:15 +0000 (0:00:00.042) 0:00:11.849 ******* TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:15.981 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:15 +0000 (0:00:00.055) 0:00:11.905 ******* 2020-11-20T11:58:15.989 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-20T11:58:15.991 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.031 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.046) 0:00:11.952 ******* 2020-11-20T11:58:16.037 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.078 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.045) 0:00:11.998 ******* 2020-11-20T11:58:16.081 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.116 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.041) 0:00:12.039 ******* 2020-11-20T11:58:16.125 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.164 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.047) 0:00:12.086 ******* 2020-11-20T11:58:16.168 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-20T11:58:16.171 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.215 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.053) 0:00:12.139 ******* 2020-11-20T11:58:16.222 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-20T11:58:16.225 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:16.259 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.043) 0:00:12.183 ******* 2020-11-20T11:58:16.277 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.316 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.054) 0:00:12.237 ******* 2020-11-20T11:58:16.322 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.355 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.041) 0:00:12.279 ******* 2020-11-20T11:58:16.368 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi044.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:16.402 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.046) 0:00:12.325 ******* 2020-11-20T11:58:16.408 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-20T11:58:16.410 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.445 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.043) 0:00:12.369 ******* 2020-11-20T11:58:16.451 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T11:58:16.453 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.486 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.040) 0:00:12.410 ******* 2020-11-20T11:58:16.525 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.557 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.071) 0:00:12.481 ******* 2020-11-20T11:58:16.587 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.620 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.063) 0:00:12.544 ******* 2020-11-20T11:58:16.652 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-20T11:58:16.657 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.688 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.065) 0:00:12.610 ******* 2020-11-20T11:58:16.693 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-20T11:58:16.694 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T11:58:16.732 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.045) 0:00:12.655 ******* 2020-11-20T11:58:16.743 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.785 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.041) 0:00:12.697 ******* 2020-11-20T11:58:16.819 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.855 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.079) 0:00:12.776 ******* 2020-11-20T11:58:16.892 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi044.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:58:16.931 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:16 +0000 (0:00:00.078) 0:00:12.855 ******* 2020-11-20T11:58:16.972 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:16.974 INFO:teuthology.task.ansible.out: 2020-11-20T11:58:17.012 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:17 +0000 (0:00:00.080) 0:00:12.936 ******* 2020-11-20T11:58:17.065 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:17.101 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:17 +0000 (0:00:00.087) 0:00:13.023 ******* 2020-11-20T11:58:17.341 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi079.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-20T11:58:17.373 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi044.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T11:58:17.407 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:17 +0000 (0:00:00.307) 0:00:13.331 ******* 2020-11-20T11:58:17.414 INFO:teuthology.task.ansible.out: 2020-11-20T11:58:17.837 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi079.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-20T11:58:17.892 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:17 +0000 (0:00:00.480) 0:00:13.812 ******* 2020-11-20T11:58:18.025 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-20T11:58:18.490 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi079.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-20T11:58:18.621 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi044.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-20T11:58:18.679 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:18 +0000 (0:00:00.790) 0:00:14.602 ******* 2020-11-20T11:58:18.971 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-20T11:58:18.973 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-20T11:58:19.011 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.330) 0:00:14.933 ******* 2020-11-20T11:58:19.047 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T11:58:19.057 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.084 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.074) 0:00:15.007 ******* 2020-11-20T11:58:19.114 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.146 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.060) 0:00:15.068 ******* 2020-11-20T11:58:19.172 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-20T11:58:19.175 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.207 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.062) 0:00:15.131 ******* 2020-11-20T11:58:19.212 INFO:teuthology.task.ansible.out: 2020-11-20T11:58:19.241 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.293 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.080) 0:00:15.211 ******* 2020-11-20T11:58:19.297 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi044.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-20T11:58:19.333 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.385 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.091) 0:00:15.303 ******* 2020-11-20T11:58:19.420 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.457 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.077) 0:00:15.380 ******* 2020-11-20T11:58:19.483 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-20T11:58:19.484 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.516 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.059) 0:00:15.440 ******* 2020-11-20T11:58:19.545 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.588 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.068) 0:00:15.509 ******* 2020-11-20T11:58:19.613 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.647 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.061) 0:00:15.570 ******* 2020-11-20T11:58:19.776 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi044.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-20T11:58:19.811 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.164) 0:00:15.735 ******* 2020-11-20T11:58:19.861 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.901 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.086) 0:00:15.821 ******* 2020-11-20T11:58:19.937 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:19.972 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:19 +0000 (0:00:00.072) 0:00:15.894 ******* 2020-11-20T11:58:19.997 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:20.033 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.061) 0:00:15.955 ******* 2020-11-20T11:58:20.059 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:20.096 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.065) 0:00:16.020 ******* 2020-11-20T11:58:20.123 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T11:58:20.125 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:20.159 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.060) 0:00:16.081 ******* 2020-11-20T11:58:20.188 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:20.221 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.061) 0:00:16.142 ******* 2020-11-20T11:58:20.245 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-20T11:58:20.247 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:20.282 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.061) 0:00:16.203 ******* 2020-11-20T11:58:20.307 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:20.346 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.061) 0:00:16.265 ******* 2020-11-20T11:58:20.374 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:20.407 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.065) 0:00:16.330 ******* 2020-11-20T11:58:20.523 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-20T11:58:20.525 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.686599", "end": "2020-11-20 11:58:20.500331", "rc": 0, "start": "2020-11-20 11:58:19.813732", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:58:20.571 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:20 +0000 (0:00:00.164) 0:00:16.495 ******* 2020-11-20T11:58:21.527 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.909158", "end": "2020-11-20 11:58:21.500366", "rc": 0, "start": "2020-11-20 11:58:20.591208", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:58:21.569 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:21 +0000 (0:00:00.993) 0:00:17.488 ******* 2020-11-20T11:58:21.825 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-20T11:58:21.876 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:21 +0000 (0:00:00.304) 0:00:17.792 ******* 2020-11-20T11:58:21.912 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:21.956 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:21 +0000 (0:00:00.087) 0:00:17.880 ******* 2020-11-20T11:58:21.989 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:22.039 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.082) 0:00:17.963 ******* 2020-11-20T11:58:22.066 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:22.101 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.062) 0:00:18.025 ******* 2020-11-20T11:58:22.128 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-20T11:58:22.130 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:22.163 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.062) 0:00:18.087 ******* 2020-11-20T11:58:22.605 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-20T11:58:22.607 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-20T11:58:22.640 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.476) 0:00:18.564 ******* 2020-11-20T11:58:22.646 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-20T11:58:22.648 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-20T11:58:22.686 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.042) 0:00:18.606 ******* 2020-11-20T11:58:22.722 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:22.761 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.078) 0:00:18.685 ******* 2020-11-20T11:58:22.803 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:22.836 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.074) 0:00:18.760 ******* 2020-11-20T11:58:22.863 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-20T11:58:22.865 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:22.916 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.080) 0:00:18.840 ******* 2020-11-20T11:58:22.957 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-20T11:58:22.959 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:23.000 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:22 +0000 (0:00:00.083) 0:00:18.924 ******* 2020-11-20T11:58:23.543 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-20T11:58:23.577 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:23 +0000 (0:00:00.576) 0:00:19.501 ******* 2020-11-20T11:58:26.196 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-20T11:58:26.198 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-20T11:58:26.230 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:26 +0000 (0:00:02.653) 0:00:22.154 ******* 2020-11-20T11:58:26.752 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T11:58:26.753 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-20T11:58:27.219 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T11:58:27.221 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-20T11:58:27.258 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:27 +0000 (0:00:01.027) 0:00:23.182 ******* 2020-11-20T11:58:27.691 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T11:58:27.972 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} Friday 20 November 2020 11:58:27 +0000 (0:00:00.471) 0:00:23.653 ******* ok: [smithi044.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} Friday 20 November 2020 11:58:27 +0000 (0:00:00.233) 0:00:23.887 ******* 2020-11-20T11:58:28.341 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T11:58:28.343 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T11:58:28.418 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi079.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-20T11:58:28.420 INFO:teuthology.task.ansible.out: 2020-11-20T11:58:28.836 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi044.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-20T11:58:28.895 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:28 +0000 (0:00:00.930) 0:00:24.817 ******* 2020-11-20T11:58:28.910 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi079.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-20T11:58:29.114 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T11:58:29.151 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:29 +0000 (0:00:00.257) 0:00:25.075 ******* 2020-11-20T11:58:29.415 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-20T11:58:29.416 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-20T11:58:29.453 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:29 +0000 (0:00:00.300) 0:00:25.376 ******* 2020-11-20T11:58:29.497 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:29.530 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:29 +0000 (0:00:00.077) 0:00:25.453 ******* 2020-11-20T11:58:29.580 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-20T11:58:29.616 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:29 +0000 (0:00:00.086) 0:00:25.539 ******* 2020-11-20T11:58:29.724 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi044.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-20T11:58:29.866 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi079.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-20T11:58:29.933 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:29 +0000 (0:00:00.313) 0:00:25.853 ******* 2020-11-20T11:58:29.990 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-20T11:58:29.994 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:30.002 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:30.014 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:30.029 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:30.060 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:30 +0000 (0:00:00.130) 0:00:25.983 ******* 2020-11-20T11:58:30.238 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-20T11:58:30.240 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-20T11:58:30.733 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-20T11:58:30.772 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:30 +0000 (0:00:00.712) 0:00:26.696 ******* 2020-11-20T11:58:30.803 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-20T11:58:30.805 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:30.851 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:30 +0000 (0:00:00.078) 0:00:26.774 ******* 2020-11-20T11:58:30.899 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-20T11:58:30.901 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-20T11:58:30.935 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:30 +0000 (0:00:00.084) 0:00:26.859 ******* 2020-11-20T11:58:31.015 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-20T11:58:31.017 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-20T11:58:31.052 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:31 +0000 (0:00:00.116) 0:00:26.976 ******* 2020-11-20T11:58:31.090 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-20T11:58:31.091 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:31.128 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:31 +0000 (0:00:00.075) 0:00:27.051 ******* 2020-11-20T11:58:31.144 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-20T11:58:31.146 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-20T11:58:31.179 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:31 +0000 (0:00:00.052) 0:00:27.103 ******* 2020-11-20T11:58:31.215 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi044.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:31.236 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:31.238 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:31.248 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:31.281 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:31 +0000 (0:00:00.101) 0:00:27.205 ******* 2020-11-20T11:58:31.580 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-20T11:58:31.582 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-20T11:58:31.614 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:31 +0000 (0:00:00.332) 0:00:27.538 ******* 2020-11-20T11:58:32.063 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-20T11:58:32.065 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-20T11:58:32.097 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.483) 0:00:28.021 ******* 2020-11-20T11:58:32.136 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi079.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:32.169 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.071) 0:00:28.093 ******* 2020-11-20T11:58:32.290 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-20T11:58:32.292 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-20T11:58:32.326 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.157) 0:00:28.250 ******* 2020-11-20T11:58:32.353 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:32.386 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.059) 0:00:28.310 ******* 2020-11-20T11:58:32.393 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-20T11:58:32.395 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T11:58:32.434 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.047) 0:00:28.357 ******* 2020-11-20T11:58:32.661 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-20T11:58:32.663 INFO:teuthology.task.ansible.out:fatal: [smithi079.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-20T11:58:32.694 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.260) 0:00:28.618 ******* 2020-11-20T11:58:32.722 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:32.779 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.084) 0:00:28.702 ******* 2020-11-20T11:58:32.867 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-20T11:58:32.903 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:32 +0000 (0:00:00.124) 0:00:28.827 ******* 2020-11-20T11:58:33.251 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-20T11:58:33.253 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-20T11:58:33.289 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:33 +0000 (0:00:00.385) 0:00:29.212 ******* 2020-11-20T11:58:33.384 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-20T11:58:33.421 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:33 +0000 (0:00:00.131) 0:00:29.344 ******* 2020-11-20T11:58:33.451 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi044.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:33.463 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T11:58:33.496 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:33 +0000 (0:00:00.075) 0:00:29.420 ******* 2020-11-20T11:58:33.728 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-20T11:58:33.730 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T11:58:33.786 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:33 +0000 (0:00:00.286) 0:00:29.707 ******* 2020-11-20T11:58:34.052 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi044.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-20T11:58:34.106 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:34 +0000 (0:00:00.317) 0:00:30.024 ******* 2020-11-20T11:58:34.113 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-20T11:58:34.117 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 11:56:57 UTC", "ActiveEnterTimestampMonotonic": "15426721", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target system.slice tmp.mount local-fs.target var-run.mount sysinit.target time-sync.target network.target systemd-tmpfiles-setup.service basic.target systemd-journald.socket -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 11:56:57 UTC", "AssertTimestampMonotonic": "15425966", "Before": "getty@tty1.service xdm.service shutdown.target multi-user.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 11:56:57 UTC", "ConditionTimestampMonotonic": "15425966", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-20T11:58:34.118 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2494", "ExecMainStartTimestamp": "Fri 2020-11-20 11:56:57 UTC", "ExecMainStartTimestampMonotonic": "15426696", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 11:56:57 UTC", "InactiveExitTimestampMonotonic": "15426721", "InvocationID": "d77b811aa9974e9c83b9ec7e1fce0813", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-20T11:58:34.120 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2494", "MemoryAccounting": "yes", "MemoryCurrent": "1208320", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/run/nrpe /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-20T11:58:34.122 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 11:56:57 UTC", "StateChangeTimestampMonotonic": "15426721", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 11:56:57 UTC", "WatchdogTimestampMonotonic": "15426720", "WatchdogUSec": "0"}} 2020-11-20T11:58:34.160 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:34 +0000 (0:00:00.058) 0:00:30.082 ******* 2020-11-20T11:58:34.166 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-20T11:58:34.170 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:58:34.211 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:34 +0000 (0:00:00.052) 0:00:30.134 ******* 2020-11-20T11:58:34.359 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001622", "end": "2020-11-20 11:58:34.333921", "rc": 0, "start": "2020-11-20 11:58:34.332299", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-20T11:58:34.394 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:34 +0000 (0:00:00.183) 0:00:30.318 ******* 2020-11-20T11:58:34.705 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-20T11:58:34.740 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:34 +0000 (0:00:00.345) 0:00:30.664 ******* 2020-11-20T11:58:35.071 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-20T11:58:35.072 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 11:56:24 UTC", "ActiveEnterTimestampMonotonic": "16186588", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "tmp.mount time-sync.target -.mount basic.target nss-lookup.target systemd-journald.socket local-fs.target var-run.mount systemd-tmpfiles-setup.service network.target sysinit.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 11:56:24 UTC", "AssertTimestampMonotonic": "16185573", "Before": "xdm.service getty@tty1.service shutdown.target multi-user.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 11:56:24 UTC", "ConditionTimestampMonotonic": "16185572", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-20T11:58:35.073 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2465", "ExecMainStartTimestamp": "Fri 2020-11-20 11:56:24 UTC", "ExecMainStartTimestampMonotonic": "16186548", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 11:56:24 UTC", "InactiveExitTimestampMonotonic": "16186588", "InvocationID": "fb7ca872733341068f448fdbf5f3d6e9", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-20T11:58:35.075 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2465", "MemoryAccounting": "yes", "MemoryCurrent": "1429504", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/run/nrpe /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-20T11:58:35.076 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 11:56:24 UTC", "StateChangeTimestampMonotonic": "16186588", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 11:56:24 UTC", "WatchdogTimestampMonotonic": "16186587", "WatchdogUSec": "0"}} 2020-11-20T11:58:35.107 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:35 +0000 (0:00:00.366) 0:00:31.030 ******* 2020-11-20T11:58:35.299 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-20T11:58:35.300 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-20T11:58:35.333 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:35 +0000 (0:00:00.226) 0:00:31.257 ******* 2020-11-20T11:58:35.340 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-20T11:58:35.341 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001781", "end": "2020-11-20 11:58:35.298493", "rc": 0, "start": "2020-11-20 11:58:35.296712", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-20T11:58:35.376 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:35 +0000 (0:00:00.042) 0:00:31.300 ******* 2020-11-20T11:58:36.259 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-20T11:58:36.262 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-20T11:58:36.294 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:36 +0000 (0:00:00.917) 0:00:32.217 ******* 2020-11-20T11:58:36.387 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-20T11:58:36.389 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-20T11:58:36.423 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:36 +0000 (0:00:00.129) 0:00:32.347 ******* 2020-11-20T11:58:37.461 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-20T11:58:37.463 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-20T11:58:37.496 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:37 +0000 (0:00:01.072) 0:00:33.419 ******* 2020-11-20T11:58:37.732 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-20T11:58:37.734 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-20T11:58:37.768 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:37 +0000 (0:00:00.271) 0:00:33.691 ******* 2020-11-20T11:58:38.588 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-20T11:58:38.624 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:38 +0000 (0:00:00.856) 0:00:34.548 ******* 2020-11-20T11:58:54.228 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-20T11:58:54.230 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.259973", "end": "2020-11-20 11:58:54.198031", "failed_when_result": false, "rc": 0, "start": "2020-11-20 11:58:37.938058", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-20T11:58:54.265 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:54 +0000 (0:00:15.640) 0:00:50.189 ******* 2020-11-20T11:58:54.725 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-20T11:58:54.726 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-20T11:58:54.761 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:54 +0000 (0:00:00.495) 0:00:50.685 ******* 2020-11-20T11:58:54.968 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.019581", "end": "2020-11-20 11:58:54.945631", "rc": 0, "start": "2020-11-20 11:58:54.926050", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:58:55.004 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:54 +0000 (0:00:00.243) 0:00:50.928 ******* 2020-11-20T11:58:55.209 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-20T11:58:55.211 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.008750", "end": "2020-11-20 11:58:55.187118", "rc": 0, "start": "2020-11-20 11:58:55.178368", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:58:55.242 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:55 +0000 (0:00:00.238) 0:00:51.166 ******* 2020-11-20T11:58:55.781 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-20T11:58:55.783 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.950326", "end": "2020-11-20 11:58:55.756722", "failed_when_result": false, "rc": 0, "start": "2020-11-20 11:58:38.806396", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-20T11:58:55.816 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:55 +0000 (0:00:00.573) 0:00:51.740 ******* 2020-11-20T11:58:56.291 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-20T11:58:56.292 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-20T11:58:56.326 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:56 +0000 (0:00:00.509) 0:00:52.250 ******* 2020-11-20T11:58:56.554 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-20T11:58:56.556 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.017875", "end": "2020-11-20 11:58:56.523781", "rc": 0, "start": "2020-11-20 11:58:56.505906", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:58:56.601 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:56 +0000 (0:00:00.275) 0:00:52.525 ******* 2020-11-20T11:58:56.844 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-20T11:58:56.847 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012701", "end": "2020-11-20 11:58:56.810898", "rc": 0, "start": "2020-11-20 11:58:56.798197", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:58:56.888 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:58:56 +0000 (0:00:00.284) 0:00:52.810 ******* 2020-11-20T11:59:11.731 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.281103", "end": "2020-11-20 11:59:11.694636", "rc": 0, "start": "2020-11-20 11:58:55.413533", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:11.768 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:11 +0000 (0:00:14.881) 0:01:07.691 ******* 2020-11-20T11:59:11.992 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-20T11:59:12.032 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:12 +0000 (0:00:00.264) 0:01:07.955 ******* 2020-11-20T11:59:12.059 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:12.107 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:12 +0000 (0:00:00.075) 0:01:08.031 ******* 2020-11-20T11:59:12.145 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T11:59:12.147 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T11:59:12.180 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:12 +0000 (0:00:00.072) 0:01:08.104 ******* 2020-11-20T11:59:12.207 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-20T11:59:12.209 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:12.241 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:12 +0000 (0:00:00.060) 0:01:08.164 ******* 2020-11-20T11:59:12.278 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:12.314 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:12 +0000 (0:00:00.072) 0:01:08.236 ******* 2020-11-20T11:59:12.371 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T11:59:12.373 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi079.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:59:12.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:12 +0000 (0:00:00.097) 0:01:08.333 ******* 2020-11-20T11:59:12.629 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-11-20T11:59:12.631 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T11:59:12.668 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:12 +0000 (0:00:00.257) 0:01:08.591 ******* 2020-11-20T11:59:13.241 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-20T11:59:13.244 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605873552.695693-75700244732694/source", "state": "file", "uid": 0} 2020-11-20T11:59:13.280 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:13 +0000 (0:00:00.612) 0:01:09.204 ******* 2020-11-20T11:59:13.311 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-20T11:59:13.313 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-11-20T11:59:13.347 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:13 +0000 (0:00:00.067) 0:01:09.271 ******* 2020-11-20T11:59:13.385 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi079.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T11:59:13.418 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:13 +0000 (0:00:00.071) 0:01:09.342 ******* 2020-11-20T11:59:13.457 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-20T11:59:13.458 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-20T11:59:13.492 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:13 +0000 (0:00:00.073) 0:01:09.416 ******* 2020-11-20T11:59:13.530 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi079.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T11:59:13.564 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:13 +0000 (0:00:00.071) 0:01:09.488 ******* 2020-11-20T11:59:13.749 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-20T11:59:13.751 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T11:59:13.784 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:13 +0000 (0:00:00.219) 0:01:09.707 ******* 2020-11-20T11:59:13.938 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-20T11:59:13.941 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.831219", "end": "2020-11-20 11:59:13.910110", "rc": 0, "start": "2020-11-20 11:58:57.078891", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:13.976 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:13 +0000 (0:00:00.191) 0:01:09.899 ******* 2020-11-20T11:59:13.997 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T11:59:14.058 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.082) 0:01:09.982 ******* 2020-11-20T11:59:14.088 INFO:teuthology.task.ansible.out: 2020-11-20T11:59:14.091 INFO:teuthology.task.ansible.out:TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:14.123 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.064) 0:01:10.047 ******* 2020-11-20T11:59:14.154 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:14.195 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.071) 0:01:10.118 ******* 2020-11-20T11:59:14.254 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-20T11:59:14.256 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-20T11:59:14.300 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.105) 0:01:10.224 ******* 2020-11-20T11:59:14.339 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:14.374 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.073) 0:01:10.298 ******* 2020-11-20T11:59:14.415 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T11:59:14.448 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.074) 0:01:10.372 ******* 2020-11-20T11:59:14.456 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-20T11:59:14.458 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-20T11:59:14.493 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.044) 0:01:10.417 ******* 2020-11-20T11:59:14.500 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-20T11:59:14.502 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:14.536 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.042) 0:01:10.460 ******* 2020-11-20T11:59:14.563 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T11:59:14.565 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:14.599 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.063) 0:01:10.523 ******* 2020-11-20T11:59:14.658 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T11:59:14.660 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi044.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T11:59:14.694 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:14 +0000 (0:00:00.094) 0:01:10.617 ******* 2020-11-20T11:59:15.006 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T11:59:15.050 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:15 +0000 (0:00:00.355) 0:01:10.973 ******* 2020-11-20T11:59:15.057 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-20T11:59:15.059 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-20T11:59:15.096 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:15 +0000 (0:00:00.046) 0:01:11.020 ******* 2020-11-20T11:59:15.137 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-20T11:59:15.946 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi079"}, "changed": false} Friday 20 November 2020 11:59:15 +0000 (0:00:00.079) 0:01:11.099 ******* TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605873555.1062927-210572201690770/source", "state": "file", "uid": 0} Friday 20 November 2020 11:59:15 +0000 (0:00:00.579) 0:01:11.678 ******* TASK [testnode : Include package type specific vars.] ************************** ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} Friday 20 November 2020 11:59:15 +0000 (0:00:00.070) 0:01:11.749 ******* TASK [testnode : Including distro specific variables.] *********************** 2020-11-20T11:59:15.948 INFO:teuthology.task.ansible.out:** ok: [smithi044.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} Friday 20 November 2020 11:59:15 +0000 (0:00:00.087) 0:01:11.837 ******* 2020-11-20T11:59:15.954 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [smithi044.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.y 2020-11-20T11:59:15.957 INFO:teuthology.task.ansible.out:ml"} 2020-11-20T11:59:15.989 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:15 +0000 (0:00:00.075) 0:01:11.912 ******* 2020-11-20T11:59:16.026 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi044.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T11:59:16.060 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.071) 0:01:11.984 ******* 2020-11-20T11:59:16.266 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-20T11:59:16.268 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T11:59:16.306 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.245) 0:01:12.229 ******* 2020-11-20T11:59:16.346 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi079", "ansible_hostname": "smithi079", "ansible_nodename": "smithi079"}, "changed": false, "name": "smithi079"} 2020-11-20T11:59:16.387 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.080) 0:01:12.310 ******* 2020-11-20T11:59:16.520 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-20T11:59:16.522 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T11:59:16.555 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.168) 0:01:12.479 ******* 2020-11-20T11:59:16.583 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-20T11:59:16.585 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:16.616 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.060) 0:01:12.540 ******* 2020-11-20T11:59:16.643 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-20T11:59:16.645 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:16.677 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.060) 0:01:12.600 ******* 2020-11-20T11:59:16.879 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-20T11:59:16.913 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.236) 0:01:12.837 ******* 2020-11-20T11:59:16.933 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi044.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-20T11:59:16.994 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:16 +0000 (0:00:00.080) 0:01:12.917 ******* 2020-11-20T11:59:17.384 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-20T11:59:17.386 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-20T11:59:17.419 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:17 +0000 (0:00:00.425) 0:01:13.343 ******* 2020-11-20T11:59:17.553 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-20T11:59:17.593 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:17 +0000 (0:00:00.173) 0:01:13.516 ******* 2020-11-20T11:59:17.619 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi044"}, "changed": false} 2020-11-20T11:59:17.652 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:17 +0000 (0:00:00.059) 0:01:13.576 ******* 2020-11-20T11:59:18.040 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-20T11:59:18.042 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-20T11:59:18.080 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:18 +0000 (0:00:00.428) 0:01:14.004 ******* 2020-11-20T11:59:18.245 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-20T11:59:18.247 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi044", "ansible_hostname": "smithi044", "ansible_nodename": "smithi044"}, "changed": false, "name": "smithi044"} 2020-11-20T11:59:18.284 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:18 +0000 (0:00:00.203) 0:01:14.208 ******* 2020-11-20T11:59:18.556 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-20T11:59:18.558 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-20T11:59:18.593 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:18 +0000 (0:00:00.308) 0:01:14.516 ******* 2020-11-20T11:59:18.767 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-20T11:59:18.768 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-20T11:59:18.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:18 +0000 (0:00:00.207) 0:01:14.724 ******* 2020-11-20T11:59:18.808 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T11:59:18.810 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T11:59:18.847 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:18 +0000 (0:00:00.045) 0:01:14.770 ******* 2020-11-20T11:59:19.045 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T11:59:19.048 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-20T11:59:19.080 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:19 +0000 (0:00:00.233) 0:01:15.004 ******* 2020-11-20T11:59:19.284 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-20T11:59:19.320 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:19 +0000 (0:00:00.236) 0:01:15.241 ******* 2020-11-20T11:59:19.325 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T11:59:19.326 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T11:59:19.363 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:19 +0000 (0:00:00.046) 0:01:15.287 ******* 2020-11-20T11:59:19.829 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T11:59:19.832 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-20T11:59:19.865 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:19 +0000 (0:00:00.501) 0:01:15.789 ******* 2020-11-20T11:59:19.936 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-20T11:59:19.937 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-20T11:59:19.972 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:19 +0000 (0:00:00.106) 0:01:15.896 ******* 2020-11-20T11:59:20.345 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T11:59:20.369 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-20T11:59:20.404 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:20 +0000 (0:00:00.431) 0:01:16.327 ******* 2020-11-20T11:59:20.514 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-20T11:59:20.554 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:20 +0000 (0:00:00.151) 0:01:16.478 ******* 2020-11-20T11:59:20.792 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T11:59:20.827 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:20 +0000 (0:00:00.273) 0:01:16.751 ******* 2020-11-20T11:59:21.041 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-20T11:59:21.076 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:21 +0000 (0:00:00.248) 0:01:17.000 ******* 2020-11-20T11:59:21.309 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* ok: [smithi044.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T11:59:21.348 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:21 +0000 (0:00:00.271) 0:01:17.271 ******* 2020-11-20T11:59:21.888 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-20T11:59:21.924 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:21 +0000 (0:00:00.574) 0:01:17.846 ******* 2020-11-20T11:59:22.237 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T11:59:22.239 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.610504", "end": "2020-11-20 11:59:22.204453", "rc": 0, "start": "2020-11-20 11:59:20.593949", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:22.279 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:22 +0000 (0:00:00.356) 0:01:18.202 ******* 2020-11-20T11:59:22.440 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T11:59:22.441 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-20T11:59:22.485 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:22 +0000 (0:00:00.205) 0:01:18.408 ******* 2020-11-20T11:59:22.789 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T11:59:22.791 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-20T11:59:23.237 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-20T11:59:23.712 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-20T11:59:23.762 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:23 +0000 (0:00:01.277) 0:01:19.686 ******* 2020-11-20T11:59:24.270 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-20T11:59:24.272 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-20T11:59:24.308 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:24 +0000 (0:00:00.546) 0:01:20.232 ******* 2020-11-20T11:59:24.593 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T11:59:24.595 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.889998", "end": "2020-11-20 11:59:24.557929", "rc": 0, "start": "2020-11-20 11:59:22.667931", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:24.633 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:24 +0000 (0:00:00.322) 0:01:20.555 ******* 2020-11-20T11:59:24.855 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T11:59:24.857 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-20T11:59:25.160 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** ok: [smithi044.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-20T11:59:25.311 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T11:59:25.313 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-20T11:59:25.648 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T11:59:25.653 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-20T11:59:25.783 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T11:59:26.170 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} Friday 20 November 2020 11:59:25 +0000 (0:00:01.198) 0:01:21.754 ******* TASK [testnode : Enable copr repos] ******************************************** Friday 20 November 2020 11:59:25 +0000 (0:00:00.064) 0:01:21.818 ******* TASK [testnode : Install GPG] ************************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:59:25 +0000 (0:00:00.072) 0:01:21.891 ******* TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi079.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} Friday 20 N 2020-11-20T11:59:26.172 INFO:teuthology.task.ansible.out:ovember 2020 11:59:26 +0000 (0:00:00.102) 0:01:21.993 ******* TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:59:26 +0000 (0:00:00.060) 0:01:22.054 ******* TASK [testnode : set_fact] ***************************************************** ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-20T11:59:26.202 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:26 +0000 (0:00:00.071) 0:01:22.126 ******* 2020-11-20T11:59:26.252 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-20T11:59:26.254 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure local mirrorlists] ********************************** ok: [smithi044.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-20T11:59:26.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:26 +0000 (0:00:00.093) 0:01:22.220 ******* 2020-11-20T11:59:26.334 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:26 +0000 (0:00:00.038) 0:01:22.258 ******* 2020-11-20T11:59:26.410 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:26.455 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:26 +0000 (0:00:00.121) 0:01:22.379 ******* 2020-11-20T11:59:26.677 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* fatal: [smithi079.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006723", "end": "2020-11-20 11:59:26.649428", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 11:59:26.642705", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-20T11:59:26.711 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:26 +0000 (0:00:00.255) 0:01:22.635 ******* 2020-11-20T11:59:26.749 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T11:59:26.751 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:26.788 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:26 +0000 (0:00:00.074) 0:01:22.709 ******* 2020-11-20T11:59:26.925 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-20T11:59:26.927 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-20T11:59:26.965 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:26 +0000 (0:00:00.179) 0:01:22.888 ******* 2020-11-20T11:59:27.527 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T11:59:27.529 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-20T11:59:28.009 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-20T11:59:28.477 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-20T11:59:28.513 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:01.547) 0:01:24.436 ******* 2020-11-20T11:59:28.539 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-20T11:59:28.573 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:00.060) 0:01:24.497 ******* 2020-11-20T11:59:28.599 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-20T11:59:28.601 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:28.633 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:00.060) 0:01:24.557 ******* 2020-11-20T11:59:28.671 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-20T11:59:28.673 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:28.683 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:28.716 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:00.083) 0:01:24.640 ******* 2020-11-20T11:59:28.757 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:28.788 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:00.072) 0:01:24.712 ******* 2020-11-20T11:59:28.815 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T11:59:28.817 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-20T11:59:28.849 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:00.060) 0:01:24.772 ******* 2020-11-20T11:59:28.878 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-20T11:59:28.880 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:28.915 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:00.065) 0:01:24.838 ******* 2020-11-20T11:59:28.965 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-20T11:59:28.968 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph packages are not present.] ************************ changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-20T11:59:29.005 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:28 +0000 (0:00:00.088) 0:01:24.927 ******* 2020-11-20T11:59:29.043 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:29 +0000 (0:00:00.039) 0:01:24.967 ******* 2020-11-20T11:59:29.232 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-20T11:59:29.234 INFO:teuthology.task.ansible.out:fatal: [smithi044.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007295", "end": "2020-11-20 11:59:29.202503", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 11:59:29.195208", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-20T11:59:29.269 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:29 +0000 (0:00:00.226) 0:01:25.193 ******* 2020-11-20T11:59:29.308 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T11:59:29.310 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:29.346 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:29 +0000 (0:00:00.076) 0:01:25.270 ******* 2020-11-20T11:59:30.883 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-20T11:59:30.916 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:30 +0000 (0:00:01.570) 0:01:26.840 ******* 2020-11-20T11:59:30.918 INFO:teuthology.task.ansible.out: 2020-11-20T11:59:31.892 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T11:59:31.893 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-20T11:59:31.936 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:31 +0000 (0:00:01.019) 0:01:27.860 ******* 2020-11-20T11:59:33.937 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T11:59:33.939 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-20T11:59:33.973 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:33 +0000 (0:00:02.036) 0:01:29.897 ******* 2020-11-20T11:59:33.979 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T11:59:33.981 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-20T11:59:34.015 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:34 +0000 (0:00:00.042) 0:01:29.939 ******* 2020-11-20T11:59:34.021 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-20T11:59:34.022 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:34.061 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:34 +0000 (0:00:00.046) 0:01:29.985 ******* 2020-11-20T11:59:35.040 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T11:59:35.042 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-20T11:59:35.080 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:35 +0000 (0:00:01.018) 0:01:31.004 ******* 2020-11-20T11:59:35.082 INFO:teuthology.task.ansible.out: 2020-11-20T11:59:35.118 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T11:59:35.120 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:35.152 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:35 +0000 (0:00:00.071) 0:01:31.075 ******* 2020-11-20T11:59:35.478 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-20T11:59:35.484 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service basic.target dbus.service sysinit.target system.slice dbus.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ipset.service ebtables.service ip6tables.service iptables.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-20T11:59:35.485 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", 2020-11-20T11:59:35.487 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice dbus.socket", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-20T11:59:35.488 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-20T11:59:35.524 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:35 +0000 (0:00:00.371) 0:01:31.447 ******* 2020-11-20T11:59:35.823 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-20T11:59:35.825 INFO:teuthology.task.ansible.out:fatal: [smithi079.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-20T11:59:35.858 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:35 +0000 (0:00:00.334) 0:01:31.782 ******* 2020-11-20T11:59:36.594 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-20T11:59:36.597 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-11-20T11:59:36.648 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:36 +0000 (0:00:00.790) 0:01:32.572 ******* 2020-11-20T11:59:36.683 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-20T11:59:36.685 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:36.729 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:36 +0000 (0:00:00.079) 0:01:32.652 ******* 2020-11-20T11:59:36.761 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-20T11:59:36.763 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:36.796 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:36 +0000 (0:00:00.067) 0:01:32.720 ******* 2020-11-20T11:59:36.824 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-20T11:59:36.826 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:36.860 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:36 +0000 (0:00:00.063) 0:01:32.784 ******* 2020-11-20T11:59:36.909 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:36.943 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:36 +0000 (0:00:00.082) 0:01:32.867 ******* 2020-11-20T11:59:36.970 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.004 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:36 +0000 (0:00:00.061) 0:01:32.928 ******* 2020-11-20T11:59:37.034 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.069 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.063) 0:01:32.992 ******* 2020-11-20T11:59:37.108 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-20T11:59:37.161 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.088) 0:01:33.080 ******* 2020-11-20T11:59:37.192 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.226 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.069) 0:01:33.150 ******* 2020-11-20T11:59:37.264 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-20T11:59:37.266 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.299 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.073) 0:01:33.223 ******* 2020-11-20T11:59:37.318 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T11:59:37.319 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-20T11:59:37.365 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.065) 0:01:33.288 ******* 2020-11-20T11:59:37.372 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-20T11:59:37.374 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.045) 0:01:33.334 ******* 2020-11-20T11:59:37.417 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.467 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.051) 0:01:33.385 ******* 2020-11-20T11:59:37.469 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-20T11:59:37.469 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.507 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.045) 0:01:33.431 ******* 2020-11-20T11:59:37.541 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-20T11:59:37.575 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.067) 0:01:33.498 ******* 2020-11-20T11:59:37.602 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.651 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.075) 0:01:33.574 ******* 2020-11-20T11:59:37.680 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.714 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.063) 0:01:33.638 ******* 2020-11-20T11:59:37.741 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.775 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.060) 0:01:33.698 ******* 2020-11-20T11:59:37.802 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T11:59:37.805 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.838 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.063) 0:01:33.762 ******* 2020-11-20T11:59:37.865 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-20T11:59:37.867 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.907 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.068) 0:01:33.830 ******* 2020-11-20T11:59:37.946 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T11:59:37.948 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:37.981 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:37 +0000 (0:00:00.074) 0:01:33.905 ******* 2020-11-20T11:59:38.009 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-20T11:59:38.011 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.052 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.070) 0:01:33.976 ******* 2020-11-20T11:59:38.091 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-20T11:59:38.092 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.124 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.071) 0:01:34.048 ******* 2020-11-20T11:59:38.152 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.185 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.060) 0:01:34.108 ******* 2020-11-20T11:59:38.211 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-20T11:59:38.213 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.256 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.071) 0:01:34.180 ******* 2020-11-20T11:59:38.286 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.319 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.062) 0:01:34.242 ******* 2020-11-20T11:59:38.346 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-20T11:59:38.348 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.379 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.060) 0:01:34.303 ******* 2020-11-20T11:59:38.405 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-20T11:59:38.407 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.439 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.060) 0:01:34.363 ******* 2020-11-20T11:59:38.473 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T11:59:38.475 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} TASK [testnode : Stop apache2] ************************************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.525 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.085) 0:01:34.449 ******* 2020-11-20T11:59:38.559 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.033) 0:01:34.482 ******* 2020-11-20T11:59:38.565 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T11:59:38.567 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.601 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.041) 0:01:34.524 ******* 2020-11-20T11:59:38.608 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T11:59:38.610 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.656 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.055) 0:01:34.580 ******* 2020-11-20T11:59:38.684 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.724 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.066) 0:01:34.646 ******* 2020-11-20T11:59:38.750 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T11:59:38.752 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.792 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.069) 0:01:34.716 ******* 2020-11-20T11:59:38.819 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T11:59:38.820 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.852 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.060) 0:01:34.776 ******* 2020-11-20T11:59:38.879 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T11:59:38.880 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:38.912 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.060) 0:01:34.836 ******* 2020-11-20T11:59:38.929 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-20T11:59:38.932 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service polkit.service sysinit.target basic.target system.slice dbus.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ipset.service ebtables.service iptables.service shutdown.target ip6tables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-20T11:59:38.933 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", 2020-11-20T11:59:38.935 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice dbus.socket", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-20T11:59:38.937 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-20T11:59:38.965 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:38 +0000 (0:00:00.052) 0:01:34.888 ******* 2020-11-20T11:59:38.973 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.021 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.056) 0:01:34.945 ******* 2020-11-20T11:59:39.054 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T11:59:39.056 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.089 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.068) 0:01:35.013 ******* 2020-11-20T11:59:39.116 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T11:59:39.117 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.152 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.062) 0:01:35.076 ******* 2020-11-20T11:59:39.179 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T11:59:39.181 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.212 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.060) 0:01:35.136 ******* 2020-11-20T11:59:39.239 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T11:59:39.240 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.272 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.059) 0:01:35.196 ******* 2020-11-20T11:59:39.277 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-20T11:59:39.279 INFO:teuthology.task.ansible.out:fatal: [smithi044.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-20T11:59:39.312 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.040) 0:01:35.236 ******* 2020-11-20T11:59:39.320 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T11:59:39.321 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.359 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.046) 0:01:35.283 ******* 2020-11-20T11:59:39.387 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T11:59:39.389 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.424 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.064) 0:01:35.347 ******* 2020-11-20T11:59:39.465 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T11:59:39.468 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.498 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.074) 0:01:35.422 ******* 2020-11-20T11:59:39.611 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-11-20T11:59:39.645 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.146) 0:01:35.569 ******* 2020-11-20T11:59:39.685 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.719 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.074) 0:01:35.643 ******* 2020-11-20T11:59:39.728 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-20T11:59:39.762 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.042) 0:01:35.686 ******* 2020-11-20T11:59:39.769 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.806 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.043) 0:01:35.729 ******* 2020-11-20T11:59:39.833 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-20T11:59:39.834 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.868 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.062) 0:01:35.792 ******* 2020-11-20T11:59:39.895 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-20T11:59:39.897 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.929 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.060) 0:01:35.853 ******* 2020-11-20T11:59:39.956 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-20T11:59:39.958 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:39.991 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:39 +0000 (0:00:00.062) 0:01:35.915 ******* 2020-11-20T11:59:39.998 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T11:59:40.000 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T11:59:40.033 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.041) 0:01:35.957 ******* 2020-11-20T11:59:40.039 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-20T11:59:40.041 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.083 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.050) 0:01:36.007 ******* 2020-11-20T11:59:40.112 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-20T11:59:40.146 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.062) 0:01:36.070 ******* 2020-11-20T11:59:40.174 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-20T11:59:40.176 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.207 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.061) 0:01:36.131 ******* 2020-11-20T11:59:40.245 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-20T11:59:40.247 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make fsstress same path as ubuntu.] *************************** [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-20T11:59:40.281 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.073) 0:01:36.205 ******* 2020-11-20T11:59:40.318 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.037) 0:01:36.242 ******* 2020-11-20T11:59:40.336 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-20T11:59:40.338 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.370 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.051) 0:01:36.294 ******* 2020-11-20T11:59:40.376 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T11:59:40.378 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.414 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.044) 0:01:36.338 ******* 2020-11-20T11:59:40.420 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-20T11:59:40.422 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.455 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.040) 0:01:36.379 ******* 2020-11-20T11:59:40.462 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T11:59:40.464 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.502 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.046) 0:01:36.426 ******* 2020-11-20T11:59:40.509 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-20T11:59:40.544 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.041) 0:01:36.467 ******* 2020-11-20T11:59:40.554 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T11:59:40.556 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.592 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.048) 0:01:36.516 ******* 2020-11-20T11:59:40.598 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-20T11:59:40.600 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.641 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.049) 0:01:36.565 ******* 2020-11-20T11:59:40.649 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T11:59:40.650 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.691 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.049) 0:01:36.615 ******* 2020-11-20T11:59:40.697 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-20T11:59:40.699 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.734 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.043) 0:01:36.658 ******* 2020-11-20T11:59:40.741 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T11:59:40.743 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.775 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.040) 0:01:36.699 ******* 2020-11-20T11:59:40.781 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-20T11:59:40.783 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.817 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.041) 0:01:36.740 ******* 2020-11-20T11:59:40.823 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T11:59:40.825 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.860 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.043) 0:01:36.784 ******* 2020-11-20T11:59:40.867 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T11:59:40.869 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.903 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.041) 0:01:36.826 ******* 2020-11-20T11:59:40.908 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T11:59:40.910 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.943 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.040) 0:01:36.866 ******* 2020-11-20T11:59:40.949 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:40.987 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:40 +0000 (0:00:00.043) 0:01:36.910 ******* 2020-11-20T11:59:40.994 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T11:59:40.997 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.030 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.043) 0:01:36.954 ******* 2020-11-20T11:59:41.037 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T11:59:41.039 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.073 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.042) 0:01:36.996 ******* 2020-11-20T11:59:41.079 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T11:59:41.081 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.125 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.051) 0:01:37.048 ******* 2020-11-20T11:59:41.131 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-20T11:59:41.135 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.167 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.042) 0:01:37.091 ******* 2020-11-20T11:59:41.174 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-20T11:59:41.177 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.208 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.041) 0:01:37.132 ******* 2020-11-20T11:59:41.217 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.265 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.056) 0:01:37.189 ******* 2020-11-20T11:59:41.273 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-20T11:59:41.275 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.317 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.051) 0:01:37.241 ******* 2020-11-20T11:59:41.323 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-20T11:59:41.325 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.358 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.040) 0:01:37.282 ******* 2020-11-20T11:59:41.365 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.400 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.041) 0:01:37.323 ******* 2020-11-20T11:59:41.406 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-20T11:59:41.408 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.440 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.040) 0:01:37.364 ******* 2020-11-20T11:59:41.446 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T11:59:41.448 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.481 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.041) 0:01:37.405 ******* 2020-11-20T11:59:41.487 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-20T11:59:41.489 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.528 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.047) 0:01:37.452 ******* 2020-11-20T11:59:41.535 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-20T11:59:41.537 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.572 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.043) 0:01:37.495 ******* 2020-11-20T11:59:41.579 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.618 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.045) 0:01:37.541 ******* 2020-11-20T11:59:41.624 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-20T11:59:41.626 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.662 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.044) 0:01:37.586 ******* 2020-11-20T11:59:41.670 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-20T11:59:41.672 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.706 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.043) 0:01:37.630 ******* 2020-11-20T11:59:41.718 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.749 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.041) 0:01:37.671 ******* 2020-11-20T11:59:41.753 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-20T11:59:41.756 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.804 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.056) 0:01:37.727 ******* 2020-11-20T11:59:41.816 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.874 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.070) 0:01:37.798 ******* 2020-11-20T11:59:41.883 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T11:59:41.885 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:41.950 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.071) 0:01:37.870 ******* 2020-11-20T11:59:41.957 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.001 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:41 +0000 (0:00:00.050) 0:01:37.920 ******* 2020-11-20T11:59:42.003 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.041 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.041) 0:01:37.961 ******* 2020-11-20T11:59:42.100 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:59:42 +0000 (0:00:00.044) 0:01:38.005 ******* TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.145 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.063) 0:01:38.069 ******* 2020-11-20T11:59:42.157 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.191 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.045) 0:01:38.114 ******* 2020-11-20T11:59:42.199 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.237 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.044) 0:01:38.159 ******* 2020-11-20T11:59:42.244 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-20T11:59:42.247 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.285 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.049) 0:01:38.209 ******* 2020-11-20T11:59:42.291 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T11:59:42.293 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.328 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.043) 0:01:38.252 ******* 2020-11-20T11:59:42.335 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-20T11:59:42.337 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.375 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.046) 0:01:38.299 ******* 2020-11-20T11:59:42.385 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.427 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.050) 0:01:38.349 ******* 2020-11-20T11:59:42.432 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T11:59:42.434 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T11:59:42.469 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.043) 0:01:38.393 ******* 2020-11-20T11:59:42.477 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.523 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.050) 0:01:38.443 ******* 2020-11-20T11:59:42.541 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.569 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T11:59:42.583 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.586 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi079.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.628 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.108) 0:01:38.551 ******* 2020-11-20T11:59:42.637 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.679 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.051) 0:01:38.603 ******* 2020-11-20T11:59:42.685 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T11:59:42.687 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.722 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.042) 0:01:38.645 ******* 2020-11-20T11:59:42.750 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.797 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.073) 0:01:38.719 ******* 2020-11-20T11:59:42.828 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:42.868 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.073) 0:01:38.792 ******* 2020-11-20T11:59:42.906 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T11:59:42.908 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi079.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003850", "end": "2020-11-20 11:59:42.880552", "rc": 0, "start": "2020-11-20 11:59:42.876702", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T11:59:42.945 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.076) 0:01:38.868 ******* 2020-11-20T11:59:42.999 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:42 +0000 (0:00:00.054) 0:01:38.923 ******* 2020-11-20T11:59:43.006 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T11:59:43.007 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:43.042 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.042) 0:01:38.966 ******* 2020-11-20T11:59:43.055 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:43.094 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.051) 0:01:39.017 ******* 2020-11-20T11:59:43.131 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-20T11:59:43.133 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:43.171 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.077) 0:01:39.095 ******* 2020-11-20T11:59:43.199 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-20T11:59:43.201 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:43.251 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.079) 0:01:39.175 ******* 2020-11-20T11:59:43.280 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:43.327 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.075) 0:01:39.251 ******* 2020-11-20T11:59:43.335 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T11:59:43.338 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-20T11:59:43.371 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.044) 0:01:39.295 ******* 2020-11-20T11:59:43.602 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T11:59:43.604 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T11:59:43.635 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.263) 0:01:39.559 ******* 2020-11-20T11:59:43.855 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-20T11:59:43.895 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.259) 0:01:39.818 ******* 2020-11-20T11:59:43.929 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:43.965 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:43 +0000 (0:00:00.070) 0:01:39.889 ******* 2020-11-20T11:59:43.994 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T11:59:43.997 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.037 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.071) 0:01:39.961 ******* 2020-11-20T11:59:44.082 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T11:59:44.083 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.116 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.078) 0:01:40.040 ******* 2020-11-20T11:59:44.143 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T11:59:44.145 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.183 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.066) 0:01:40.107 ******* 2020-11-20T11:59:44.211 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.256 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.072) 0:01:40.179 ******* 2020-11-20T11:59:44.275 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-20T11:59:44.277 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-20T11:59:44.325 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.069) 0:01:40.249 ******* 2020-11-20T11:59:44.332 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.369 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.042) 0:01:40.291 ******* 2020-11-20T11:59:44.385 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T11:59:44.387 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529271808, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179998, 'block_used': 12915110, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179998, "block_size": 4096, "block_total": 240095108, "block_used": 12915110, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529271808, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-20T11:59:44.419 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.052) 0:01:40.343 ******* 2020-11-20T11:59:44.426 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T11:59:44.428 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.467 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.047) 0:01:40.391 ******* 2020-11-20T11:59:44.494 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.529 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.062) 0:01:40.453 ******* 2020-11-20T11:59:44.561 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T11:59:44.563 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.602 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.073) 0:01:40.526 ******* 2020-11-20T11:59:44.621 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-20T11:59:44.623 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.004953", "end": "2020-11-20 11:59:44.593686", "rc": 0, "start": "2020-11-20 11:59:44.588733", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:44.660 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.057) 0:01:40.584 ******* 2020-11-20T11:59:44.669 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-20T11:59:44.671 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.714 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.053) 0:01:40.637 ******* 2020-11-20T11:59:44.720 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-20T11:59:44.722 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529271808, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179998, 'block_used': 12915110, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179998, "block_size": 4096, "block_total": 240095108, "block_used": 12915110, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529271808, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.757 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.043) 0:01:40.680 ******* 2020-11-20T11:59:44.763 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-20T11:59:44.765 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.797 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.040) 0:01:40.721 ******* 2020-11-20T11:59:44.803 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T11:59:44.837 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.040) 0:01:40.761 ******* 2020-11-20T11:59:44.844 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T11:59:44.846 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.887 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.049) 0:01:40.811 ******* 2020-11-20T11:59:44.918 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T11:59:44.920 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:44.955 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:44 +0000 (0:00:00.068) 0:01:40.879 ******* 2020-11-20T11:59:44.983 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-20T11:59:44.985 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.032 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.076) 0:01:40.955 ******* 2020-11-20T11:59:45.062 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-20T11:59:45.063 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.097 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.065) 0:01:41.021 ******* 2020-11-20T11:59:45.135 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-20T11:59:45.137 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.174 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.077) 0:01:41.098 ******* 2020-11-20T11:59:45.203 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-20T11:59:45.208 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.240 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.065) 0:01:41.164 ******* 2020-11-20T11:59:45.268 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-20T11:59:45.270 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.303 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.062) 0:01:41.226 ******* 2020-11-20T11:59:45.323 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-20T11:59:45.326 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.377 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.074) 0:01:41.301 ******* 2020-11-20T11:59:45.419 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.458 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.081) 0:01:41.382 ******* 2020-11-20T11:59:45.486 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-20T11:59:45.488 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.522 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.063) 0:01:41.446 ******* 2020-11-20T11:59:45.550 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.583 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.060) 0:01:41.506 ******* 2020-11-20T11:59:45.610 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T11:59:45.612 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T11:59:45.644 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.060) 0:01:41.567 ******* 2020-11-20T11:59:45.681 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-20T11:59:45.683 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.693 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T11:59:45.703 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.715 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.717 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.726 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.738 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-20T11:59:45.776 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:45 +0000 (0:00:00.132) 0:01:41.700 ******* 2020-11-20T11:59:45.982 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-20T11:59:45.984 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005479", "end": "2020-11-20 11:59:45.953520", "rc": 0, "start": "2020-11-20 11:59:45.948041", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T11:59:46.022 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:46 +0000 (0:00:00.245) 0:01:41.946 ******* 2020-11-20T11:59:46.049 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:46.101 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:46 +0000 (0:00:00.079) 0:01:42.025 ******* 2020-11-20T11:59:46.139 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-20T11:59:46.141 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:46.172 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:46 +0000 (0:00:00.071) 0:01:42.096 ******* 2020-11-20T11:59:46.201 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:46.235 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:46 +0000 (0:00:00.062) 0:01:42.158 ******* 2020-11-20T11:59:46.263 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:46.300 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:46 +0000 (0:00:00.065) 0:01:42.224 ******* 2020-11-20T11:59:46.406 INFO:teuthology.task.ansible.out: 2020-11-20T11:59:46.408 INFO:teuthology.task.ansible.out:TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adX2PeTLgaFd0kBVL7fVQzaC6oQrfA5RRD'], 'uuids': ['e4dd38fc-c5a2-470d-b9cb-b39d411484e5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.335022", "end": "2020-11-20 11:59:46.363072", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adX2PeTLgaFd0kBVL7fVQzaC6oQrfA5RRD"], "labels": [], "masters": [], "uuids": ["e4dd38fc-c5a2-470d-b9cb-b39d411484e5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:45.028050", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk usi 2020-11-20T11:59:46.410 INFO:teuthology.task.ansible.out:ng fdisk or", "other utilities."]} 2020-11-20T11:59:47.314 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-20T11:59:47.348 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:47 +0000 (0:00:01.047) 0:01:43.272 ******* 2020-11-20T11:59:47.397 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T11:59:47.399 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529202176, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179981, 'block_used': 12915127, 'inode_total': 61054976, 'inode_available': 60990120, 'inode_used': 64856, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179981, "block_size": 4096, "block_total": 240095108, "block_used": 12915127, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990120, "inode_total": 61054976, "inode_used": 64856, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529202176, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-20T11:59:47.434 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:47 +0000 (0:00:00.086) 0:01:43.358 ******* 2020-11-20T11:59:47.643 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-20T11:59:47.644 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005270", "end": "2020-11-20 11:59:47.616392", "rc": 0, "start": "2020-11-20 11:59:47.611122", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:47.677 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:47 +0000 (0:00:00.242) 0:01:43.601 ******* 2020-11-20T11:59:47.694 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T11:59:47.696 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-4l6nmz-e3wf-I6Ga-iPOV-3XzO-TC2w-fFGEVy', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204013G400BGN', 'nvme-nvme.8086-50484654363230343031334734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.068718", "end": "2020-11-20 11:59:47.661434", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-4l6nmz-e3wf-I6Ga-iPOV-3XzO-TC2w-fFGEVy", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013G400BGN", "nvme-nvme.8086-50484654363230343031334734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:46.592716", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T11:59:47.698 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:47.717 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-20T11:59:47.719 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529202176, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179981, 'block_used': 12915127, 'inode_total': 61054976, 'inode_available': 60990120, 'inode_used': 64856, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179981, "block_size": 4096, "block_total": 240095108, "block_used": 12915127, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990120, "inode_total": 61054976, "inode_used": 64856, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529202176, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:47.758 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:47 +0000 (0:00:00.080) 0:01:43.681 ******* 2020-11-20T11:59:47.785 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T11:59:47.818 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:47 +0000 (0:00:00.060) 0:01:43.742 ******* 2020-11-20T11:59:48.959 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adcfW8BOe1Nm6agLmjaly8brsCLvAR6hQI'], 'uuids': ['67d12878-231a-4c2f-b6b5-67df197423ba'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.064730", "end": "2020-11-20 11:59:48.921979", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adcfW8BOe1Nm6agLmjaly8brsCLvAR6hQI"], "labels": [], "masters": [], "uuids": ["67d12878-231a-4c2f-b6b5-67df197423ba"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:47.857249", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk usi 2020-11-20T11:59:48.963 INFO:teuthology.task.ansible.out:ng fdisk or", "other utilities."]} 2020-11-20T11:59:49.468 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SIRVShfI87f8xDYSMVQOceqlDmian623i'], 'uuids': ['69d52cd4-6a8c-4724-99bb-00483c328e1e'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.398839", "end": "2020-11-20 11:59:49.433814", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SIRVShfI87f8xDYSMVQOceqlDmian623i"], "labels": [], "masters": [], "uuids": ["69d52cd4-6a8c-4724-99bb-00483c328e1e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:48.034975", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:50.208 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adfucBmaDcRcK5RVcM5l5LQdC7Ug7e4JyE'], 'uuids': ['989e3345-3a4a-4a49-874c-5b87f9de5d5b'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.059609", "end": "2020-11-20 11:59:50.179756", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adfucBmaDcRcK5RVcM5l5LQdC7Ug7e4JyE"], "labels": [], "masters": [], "uuids": ["989e3345-3a4a-4a49-874c-5b87f9de5d5b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:49.120147", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:50.758 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-IyKeYN-6M0U-kQUC-Ae5e-PfbA-i6eR-Xu3vt2', 'nvme-INTEL_SSDPEDMD400G4_CVFT5331001K400BGN', 'nvme-nvme.8086-43564654353333313030314b34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.062303", "end": "2020-11-20 11:59:50.713989", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-IyKeYN-6M0U-kQUC-Ae5e-PfbA-i6eR-Xu3vt2", "nvme-INTEL_SSDPEDMD400G4_CVFT5331001K400BGN", "nvme-nvme.8086-43564654353333313030314b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:49.651686", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T11:59:50.760 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:51.463 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adx2Prj33KGIMpE5I5Lybxe9H91V3n5XGk'], 'uuids': ['5dae2a0e-ae24-4c87-b267-430df09c1cb2'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.061601", "end": "2020-11-20 11:59:51.431036", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adx2Prj33KGIMpE5I5Lybxe9H91V3n5XGk"], "labels": [], "masters": [], "uuids": ["5dae2a0e-ae24-4c87-b267-430df09c1cb2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:50.369435", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:51.476 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QDP6', 'wwn-0x5000c50091e0487b'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QDP6', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e0487b', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QDP6-part1', 'wwn-0x5000c50091e0487b-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QDP6", "wwn-0x5000c50091e0487b"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QDP6-part1", "wwn-0x5000c50091e0487b-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QDP6", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e0487b"}}, 2020-11-20T11:59:51.478 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T11:59:51.981 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SdTBo5pPS9es6ZaNbfJErd12Bn7PfCu7k'], 'uuids': ['3e281a08-7f96-4375-8aab-a43afa12fe95'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.038847", "end": "2020-11-20 11:59:51.952793", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SdTBo5pPS9es6ZaNbfJErd12Bn7PfCu7k"], "labels": [], "masters": [], "uuids": ["3e281a08-7f96-4375-8aab-a43afa12fe95"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:50.913946", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:52.718 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adkokZXkr9TjWlNvvwkIwDaWKWkHNRLYcx'], 'uuids': ['9a689d00-6040-4553-9a33-7ba8bd7671f1'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.049096", "end": "2020-11-20 11:59:52.679538", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adkokZXkr9TjWlNvvwkIwDaWKWkHNRLYcx"], "labels": [], "masters": [], "uuids": ["9a689d00-6040-4553-9a33-7ba8bd7671f1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:51.630442", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:52.758 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:52 +0000 (0:00:04.940) 0:01:48.682 ******* 2020-11-20T11:59:53.135 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T11:59:53.136 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adX2PeTLgaFd0kBVL7fVQzaC6oQrfA5RRD'], 'uuids': ['e4dd38fc-c5a2-470d-b9cb-b39d411484e5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.145142", "end": "2020-11-20 11:59:53.096693", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adX2PeTLgaFd0kBVL7fVQzaC6oQrfA5RRD"], "labels": [], "masters": [], "uuids": ["e4dd38fc-c5a2-470d-b9cb-b39d411484e5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:52.951551", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.142575 s, 809 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.142575 s, 809 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:53.255 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SL1uAt09gxnmdYl15TCORiVDCc8vnkeR4'], 'uuids': ['8b3dad34-16c6-453f-badc-c48b74c1bce7'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.063081", "end": "2020-11-20 11:59:53.220097", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SL1uAt09gxnmdYl15TCORiVDCc8vnkeR4"], "labels": [], "masters": [], "uuids": ["8b3dad34-16c6-453f-badc-c48b74c1bce7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:52.157016", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-11-20T11:59:53.257 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-11-20T11:59:53.395 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T11:59:53.397 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-4l6nmz-e3wf-I6Ga-iPOV-3XzO-TC2w-fFGEVy', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204013G400BGN', 'nvme-nvme.8086-50484654363230343031334734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036945", "end": "2020-11-20 11:59:53.369139", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-4l6nmz-e3wf-I6Ga-iPOV-3XzO-TC2w-fFGEVy", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013G400BGN", "nvme-nvme.8086-50484654363230343031334734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:53.332194", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0344301 s, 3.4 GB/s", "std 2020-11-20T11:59:53.399 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0344301 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:53.756 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adcfW8BOe1Nm6agLmjaly8brsCLvAR6hQI'], 'uuids': ['67d12878-231a-4c2f-b6b5-67df197423ba'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.140241", "end": "2020-11-20 11:59:53.719949", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adcfW8BOe1Nm6agLmjaly8brsCLvAR6hQI"], "labels": [], "masters": [], "uuids": ["67d12878-231a-4c2f-b6b5-67df197423ba"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:53.579708", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137642 s, 838 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137642 s, 838 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:54.093 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adfucBmaDcRcK5RVcM5l5LQdC7Ug7e4JyE'], 'uuids': ['989e3345-3a4a-4a49-874c-5b87f9de5d5b'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.149625", "end": "2020-11-20 11:59:54.062188", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adfucBmaDcRcK5RVcM5l5LQdC7Ug7e4JyE"], "labels": [], "masters": [], "uuids": ["989e3345-3a4a-4a49-874c-5b87f9de5d5b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:53.912563", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.147159 s, 784 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.147159 s, 784 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:54.434 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adx2Prj33KGIMpE5I5Lybxe9H91V3n5XGk'], 'uuids': ['5dae2a0e-ae24-4c87-b267-430df09c1cb2'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.142096", "end": "2020-11-20 11:59:54.402060", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adx2Prj33KGIMpE5I5Lybxe9H91V3n5XGk"], "labels": [], "masters": [], "uuids": ["5dae2a0e-ae24-4c87-b267-430df09c1cb2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:54.259964", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139551 s, 827 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139551 s, 827 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:54.436 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QDP6', 'wwn-0x5000c50091e0487b'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QDP6', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e0487b', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QDP6-part1', 'wwn-0x5000c50091e0487b-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QDP6", "wwn-0x5000c50091e0487b"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QDP6-part1", "wwn-0x5000c50091e0487b-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QDP6", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e0487b"}}, 2020-11-20T11:59:54.438 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T11:59:54.541 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T11:59:54.543 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SCbA5MnXSUzbFaZ8bcHL74RnRifFUfbwr'], 'uuids': ['c29d680a-8aa1-4cb0-bb7e-1ac30fe431bd'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.055291", "end": "2020-11-20 11:59:54.503987", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SCbA5MnXSUzbFaZ8bcHL74RnRifFUfbwr"], "labels": [], "masters": [], "uuids": ["c29d680a-8aa1-4cb0-bb7e-1ac30fe431bd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:53.448696", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:54.564 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRJL', 'wwn-0x5000c500871a4e42'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HRJL', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a4e42', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRJL-part1', 'wwn-0x5000c500871a4e42-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRJL", "wwn-0x5000c500871a4e42"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRJL-part1", "wwn-0x5000c500871a4e42-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HRJL", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a4e42"}}, 2020-11-20T11:59:54.566 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T11:59:54.776 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T11:59:54.778 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adkokZXkr9TjWlNvvwkIwDaWKWkHNRLYcx'], 'uuids': ['9a689d00-6040-4553-9a33-7ba8bd7671f1'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.142376", "end": "2020-11-20 11:59:54.746886", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-faHRX1ScjEOHrtw5jkOjbrk6XBeMR1adkokZXkr9TjWlNvvwkIwDaWKWkHNRLYcx"], "labels": [], "masters": [], "uuids": ["9a689d00-6040-4553-9a33-7ba8bd7671f1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:54.604510", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139671 s, 826 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139671 s, 826 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:54.817 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:54 +0000 (0:00:02.058) 0:01:50.740 ******* 2020-11-20T11:59:55.076 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-20T11:59:55.078 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.068614", "end": "2020-11-20 11:59:55.051216", "rc": 0, "start": "2020-11-20 11:59:54.982602", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:55.110 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.293) 0:01:51.034 ******* 2020-11-20T11:59:55.349 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-20T11:59:55.351 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.050824", "end": "2020-11-20 11:59:55.328044", "rc": 0, "start": "2020-11-20 11:59:55.277220", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:55.385 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.274) 0:01:51.308 ******* 2020-11-20T11:59:55.411 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-20T11:59:55.448 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.063) 0:01:51.372 ******* 2020-11-20T11:59:55.475 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:55.511 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.063) 0:01:51.435 ******* 2020-11-20T11:59:55.537 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-20T11:59:55.539 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:55.572 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.060) 0:01:51.495 ******* 2020-11-20T11:59:55.598 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:55.634 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.062) 0:01:51.558 ******* 2020-11-20T11:59:55.661 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:55.695 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.060) 0:01:51.618 ******* 2020-11-20T11:59:55.806 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T11:59:55.808 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8Svk49hha85YK4ffeHV1M0YCcZBFc10r0q'], 'uuids': ['e552c9d6-b8c9-4490-b2eb-9f5c8beaefdc'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.040130", "end": "2020-11-20 11:59:55.779657", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8Svk49hha85YK4ffeHV1M0YCcZBFc10r0q"], "labels": [], "masters": [], "uuids": ["e552c9d6-b8c9-4490-b2eb-9f5c8beaefdc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:54.739527", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T11:59:55.860 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.165) 0:01:51.783 ******* 2020-11-20T11:59:55.887 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi079.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T11:59:55.931 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:55 +0000 (0:00:00.071) 0:01:51.855 ******* 2020-11-20T11:59:55.971 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi079.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529271808, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179998, 'block_used': 12915110, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179998, "block_size": 4096, "block_total": 240095108, "block_used": 12915110, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529271808, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:56.007 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:56 +0000 (0:00:00.075) 0:01:51.931 ******* 2020-11-20T11:59:56.034 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:56.067 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:56 +0000 (0:00:00.060) 0:01:51.991 ******* 2020-11-20T11:59:56.094 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:56.127 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:56 +0000 (0:00:00.059) 0:01:52.051 ******* 2020-11-20T11:59:56.156 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:56.188 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:56 +0000 (0:00:00.060) 0:01:52.112 ******* 2020-11-20T11:59:56.215 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-20T11:59:56.217 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:56.252 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:56 +0000 (0:00:00.064) 0:01:52.176 ******* 2020-11-20T11:59:56.260 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T11:59:56.262 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SIRVShfI87f8xDYSMVQOceqlDmian623i'], 'uuids': ['69d52cd4-6a8c-4724-99bb-00483c328e1e'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.135631", "end": "2020-11-20 11:59:56.204349", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SIRVShfI87f8xDYSMVQOceqlDmian623i"], "labels": [], "masters": [], "uuids": ["69d52cd4-6a8c-4724-99bb-00483c328e1e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:56.068718", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.132847 s, 868 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.132847 s, 868 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:56.284 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:56.319 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:56 +0000 (0:00:00.066) 0:01:52.243 ******* 2020-11-20T11:59:56.347 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-20T11:59:56.349 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:56.382 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:56 +0000 (0:00:00.062) 0:01:52.306 ******* 2020-11-20T11:59:56.475 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-IyKeYN-6M0U-kQUC-Ae5e-PfbA-i6eR-Xu3vt2', 'nvme-INTEL_SSDPEDMD400G4_CVFT5331001K400BGN', 'nvme-nvme.8086-43564654353333313030314b34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036399", "end": "2020-11-20 11:59:56.441514", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-IyKeYN-6M0U-kQUC-Ae5e-PfbA-i6eR-Xu3vt2", "nvme-INTEL_SSDPEDMD400G4_CVFT5331001K400BGN", "nvme-nvme.8086-43564654353333313030314b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:56.405115", "stderr": "110+0 records in\n110+0 2020-11-20T11:59:56.477 INFO:teuthology.task.ansible.out: records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0335785 s, 3.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0335785 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:56.805 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SdTBo5pPS9es6ZaNbfJErd12Bn7PfCu7k'], 'uuids': ['3e281a08-7f96-4375-8aab-a43afa12fe95'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.137327", "end": "2020-11-20 11:59:56.773444", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SdTBo5pPS9es6ZaNbfJErd12Bn7PfCu7k"], "labels": [], "masters": [], "uuids": ["3e281a08-7f96-4375-8aab-a43afa12fe95"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:56.636117", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.134672 s, 856 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.134672 s, 856 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:57.080 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-20T11:59:57.082 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-20T11:59:57.134 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:57 +0000 (0:00:00.752) 0:01:53.058 ******* 2020-11-20T11:59:57.152 INFO:teuthology.task.ansible.out: 2020-11-20T11:59:57.154 INFO:teuthology.task.ansible.out:TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SL1uAt09gxnmdYl15TCORiVDCc8vnkeR4'], 'uuids': ['8b3dad34-16c6-453f-badc-c48b74c1bce7'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.145245", "end": "2020-11-20 11:59:57.109762", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SL1uAt09gxnmdYl15TCORiVDCc8vnkeR4"], "labels": [], "masters": [], "uuids": ["8b3dad34-16c6-453f-badc-c48b74c1bce7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:56.964517", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.142604 s, 809 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.142604 s, 809 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:57.488 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SCbA5MnXSUzbFaZ8bcHL74RnRifFUfbwr'], 'uuids': ['c29d680a-8aa1-4cb0-bb7e-1ac30fe431bd'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.148372", "end": "2020-11-20 11:59:57.459228", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8SCbA5MnXSUzbFaZ8bcHL74RnRifFUfbwr"], "labels": [], "masters": [], "uuids": ["c29d680a-8aa1-4cb0-bb7e-1ac30fe431bd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:57.310856", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.145636 s, 792 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.145636 s, 792 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:57.502 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRJL', 'wwn-0x5000c500871a4e42'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HRJL', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a4e42', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HRJL-part1', 'wwn-0x5000c500871a4e42-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRJL", "wwn-0x5000c500871a4e42"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HRJL-part1", "wwn-0x5000c500871a4e42-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HRJL", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a4e42"}}, 2020-11-20T11:59:57.505 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T11:59:57.750 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T11:59:57.825 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T11:59:57.827 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8Svk49hha85YK4ffeHV1M0YCcZBFc10r0q'], 'uuids': ['e552c9d6-b8c9-4490-b2eb-9f5c8beaefdc'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.135595", "end": "2020-11-20 11:59:57.801757", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-186JUmMaPc7HO1QRjIAhlrXF90rWRh8Svk49hha85YK4ffeHV1M0YCcZBFc10r0q"], "labels": [], "masters": [], "uuids": ["e552c9d6-b8c9-4490-b2eb-9f5c8beaefdc"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 11:59:57.666162", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.132946 s, 868 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.132946 s, 868 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T11:59:57.879 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:57 +0000 (0:00:00.745) 0:01:53.803 ******* 2020-11-20T11:59:58.094 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T11:59:58.965 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi044.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.073394", "end": "2020-11-20 11:59:58.130884", "rc": 0, "start": "2020-11-20 11:59:58.057490", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Friday 20 November 2020 11:59:58 +0000 (0:00:00.324) 0:01:54.128 ******* TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Check for physical volumes] *********************************** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.060328", "end": "2020-11-20 11:59:58.448462", "rc": 0, "start": "2020-11-20 11:59:58.388134", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Friday 20 November 2020 11:59:58 +0000 (0:00:00.315) 0:01:54.443 ******* TASK [testnode : Remove physical volumes] ************************************** Friday 20 November 2020 11:59:58 +0000 (0:00:00.078) 0:01:54.522 ******* TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:59:58 +0000 (0:00:00.077) 0:01:54.600 ******* TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:59:58 +0000 (0:00:00.062) 2020-11-20T11:59:58.967 INFO:teuthology.task.ansible.out: 0:01:54.662 ******* TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:59:58 +0000 (0:00:00.083) 0:01:54.746 ******* TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 11:59:58 +0000 (0:00:00.064) 0:01:54.811 ******* 2020-11-20T11:59:59.103 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi044.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T11:59:59.153 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.263) 0:01:55.074 ******* 2020-11-20T11:59:59.181 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T11:59:59.240 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.089) 0:01:55.164 ******* 2020-11-20T11:59:59.260 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi044.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529202176, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179981, 'block_used': 12915127, 'inode_total': 61054976, 'inode_available': 60990120, 'inode_used': 64856, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179981, "block_size": 4096, "block_total": 240095108, "block_used": 12915127, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990120, "inode_total": 61054976, "inode_used": 64856, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529202176, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:59.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.055) 0:01:55.220 ******* 2020-11-20T11:59:59.337 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:59.381 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.076) 0:01:55.296 ******* 2020-11-20T11:59:59.434 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:59.504 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.129) 0:01:55.425 ******* 2020-11-20T11:59:59.521 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T11:59:59.572 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.070) 0:01:55.496 ******* 2020-11-20T11:59:59.582 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T11:59:59.584 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:59.627 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.054) 0:01:55.550 ******* 2020-11-20T11:59:59.668 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:59.705 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.078) 0:01:55.628 ******* 2020-11-20T11:59:59.732 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-20T11:59:59.734 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:59.766 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.061) 0:01:55.690 ******* 2020-11-20T11:59:59.795 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T11:59:59.799 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T11:59:59.819 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-20T11:59:59.821 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T11:59:59.855 INFO:teuthology.task.ansible.out:Friday 20 November 2020 11:59:59 +0000 (0:00:00.088) 0:01:55.778 ******* 2020-11-20T11:59:59.982 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:00:00.163 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:00:00.335 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-20T12:00:00.376 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:00 +0000 (0:00:00.521) 0:01:56.300 ******* 2020-11-20T12:00:00.398 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T12:00:00.399 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:00:00.403 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:00.404 INFO:teuthology.task.ansible.out: 2020-11-20T12:00:00.455 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:00 +0000 (0:00:00.079) 0:01:56.379 ******* 2020-11-20T12:00:00.664 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:00:00.698 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:00 +0000 (0:00:00.242) 0:01:56.621 ******* 2020-11-20T12:00:00.739 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-20T12:00:00.784 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:00 +0000 (0:00:00.086) 0:01:56.708 ******* 2020-11-20T12:00:00.791 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T12:00:00.798 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:00:01.152 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:00:01.524 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:00:01.861 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:00:02.211 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:00:02.254 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:02 +0000 (0:00:01.469) 0:01:58.177 ******* 2020-11-20T12:00:02.359 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-20T12:00:02.361 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true} 2020-11-20T12:00:02.397 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:02 +0000 (0:00:00.143) 0:01:58.321 ******* 2020-11-20T12:00:02.491 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T12:00:02.541 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:02 +0000 (0:00:00.139) 0:01:58.460 ******* 2020-11-20T12:00:02.853 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T12:00:02.855 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:00:03.072 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:00:03.148 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-20T12:00:03.193 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.655) 0:01:59.116 ******* 2020-11-20T12:00:03.234 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-20T12:00:03.236 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.277 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.085) 0:01:59.201 ******* 2020-11-20T12:00:03.286 INFO:teuthology.task.ansible.out: 2020-11-20T12:00:03.288 INFO:teuthology.task.ansible.out:TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:00:03.311 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-20T12:00:03.315 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.348 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.070) 0:01:59.271 ******* 2020-11-20T12:00:03.375 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-20T12:00:03.380 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.408 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.060) 0:01:59.332 ******* 2020-11-20T12:00:03.435 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-20T12:00:03.437 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.470 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.061) 0:01:59.393 ******* 2020-11-20T12:00:03.478 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T12:00:03.480 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} skipping: [smithi044.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.517 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.047) 0:01:59.441 ******* 2020-11-20T12:00:03.523 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-20T12:00:03.525 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.560 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.043) 0:01:59.484 ******* 2020-11-20T12:00:03.590 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-20T12:00:03.592 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.623 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.062) 0:01:59.547 ******* 2020-11-20T12:00:03.650 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-20T12:00:03.652 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.704 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.081) 0:01:59.628 ******* 2020-11-20T12:00:03.736 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:00:03.738 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.772 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.068) 0:01:59.696 ******* 2020-11-20T12:00:03.808 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.035) 0:01:59.732 ******* 2020-11-20T12:00:03.816 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-20T12:00:03.856 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.048) 0:01:59.780 ******* 2020-11-20T12:00:03.863 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-20T12:00:03.866 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.900 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.044) 0:01:59.824 ******* 2020-11-20T12:00:03.932 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:03.963 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:03 +0000 (0:00:00.063) 0:01:59.887 ******* 2020-11-20T12:00:04.126 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 12:00:04 +0000 (0:00:00.063) 0:01:59.950 ******* 2020-11-20T12:00:04.217 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi044.front.sepia.ceph.com] => {"changed": true} 2020-11-20T12:00:04.257 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:04 +0000 (0:00:00.227) 0:02:00.178 ******* 2020-11-20T12:00:04.555 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-20T12:00:04.590 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:04 +0000 (0:00:00.335) 0:02:00.514 ******* 2020-11-20T12:00:04.610 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-20T12:00:04.618 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-20T12:00:04.667 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:04 +0000 (0:00:00.077) 0:02:00.591 ******* 2020-11-20T12:00:04.724 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:04.768 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:04 +0000 (0:00:00.100) 0:02:00.692 ******* 2020-11-20T12:00:04.799 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-20T12:00:04.802 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:04.839 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:04 +0000 (0:00:00.071) 0:02:00.763 ******* 2020-11-20T12:00:04.869 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:04.910 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:04 +0000 (0:00:00.066) 0:02:00.830 ******* 2020-11-20T12:00:04.934 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:04.971 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:04 +0000 (0:00:00.063) 0:02:00.893 ******* 2020-11-20T12:00:04.976 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-20T12:00:04.978 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 11:56:57 UTC", "ActiveEnterTimestampMonotonic": "15449160", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sntp.service ntpd.service tmp.mount basic.target system.slice ntpdate.service -.mount systemd-journald.socket systemd-tmpfiles-setup.service sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 11:56:56 UTC", "AssertTimestampMonotonic": "14421837", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 11:56:56 UTC", "ConditionTimestampMonotonic": "14421811", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-20T12:00:04.980 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2465", "ExecMainStartTimestamp": "Fri 2020-11-20 11:56:57 UTC", "ExecMainStartTimestampMonotonic": "15408993", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 11:56:56 UTC", "InactiveExitTimestampMonotonic": "14581601", "InvocationID": "b0dd213ee40c44628d9669382ab604f4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-20T12:00:04.981 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2465", "MemoryAccounting": "yes", "MemoryCurrent": "2768896", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-20T12:00:04.983 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 11:56:57 UTC", "StateChangeTimestampMonotonic": "15449160", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 11:56:57 UTC", "WatchdogTimestampMonotonic": "15409010", "WatchdogUSec": "0"}} 2020-11-20T12:00:05.017 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.047) 0:02:00.940 ******* 2020-11-20T12:00:05.025 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.068 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.050) 0:02:00.991 ******* 2020-11-20T12:00:05.076 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T12:00:05.117 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.047) 0:02:01.038 ******* 2020-11-20T12:00:05.122 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.191 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.075) 0:02:01.114 ******* 2020-11-20T12:00:05.200 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:00:05.202 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-20T12:00:05.239 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.049) 0:02:01.163 ******* 2020-11-20T12:00:05.250 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.291 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.051) 0:02:01.215 ******* 2020-11-20T12:00:05.300 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-20T12:00:05.301 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.343 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.049) 0:02:01.265 ******* 2020-11-20T12:00:05.348 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.399 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.057) 0:02:01.322 ******* 2020-11-20T12:00:05.408 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-20T12:00:05.410 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.461 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.062) 0:02:01.385 ******* 2020-11-20T12:00:05.468 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-20T12:00:05.470 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.514 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.052) 0:02:01.437 ******* 2020-11-20T12:00:05.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-20T12:00:05.522 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.572 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.059) 0:02:01.496 ******* 2020-11-20T12:00:05.582 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-20T12:00:05.587 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.689 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.088) 0:02:01.584 ******* 2020-11-20T12:00:05.702 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-20T12:00:05.706 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:05.765 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.101) 0:02:01.686 ******* 2020-11-20T12:00:05.823 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-20T12:00:05.828 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T12:00:05.867 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:05 +0000 (0:00:00.104) 0:02:01.790 ******* 2020-11-20T12:00:06.323 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-20T12:00:06.325 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-20T12:00:06.357 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:06 +0000 (0:00:00.491) 0:02:02.281 ******* 2020-11-20T12:00:06.419 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-20T12:00:06.421 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-20T12:00:06.465 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:06 +0000 (0:00:00.107) 0:02:02.389 ******* 2020-11-20T12:00:06.705 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-20T12:00:06.708 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 11:56:24 UTC", "ActiveEnterTimestampMonotonic": "16132638", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sntp.service ntpd.service tmp.mount basic.target systemd-tmpfiles-setup.service sysinit.target -.mount systemd-journald.socket system.slice ntpdate.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 11:56:23 UTC", "AssertTimestampMonotonic": "14761006", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 11:56:23 UTC", "ConditionTimestampMonotonic": "14760962", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-20T12:00:06.715 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1565", "ExecMainStartTimestamp": "Fri 2020-11-20 11:56:24 UTC", "ExecMainStartTimestampMonotonic": "15970923", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 11:56:23 UTC", "InactiveExitTimestampMonotonic": "14771375", "InvocationID": "a0c7c0f4538d41a19e2e8e3a672ab5a1", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-20T12:00:06.717 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127450", "LimitNPROCSoft": "127450", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127450", "LimitSIGPENDINGSoft": "127450", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1565", "MemoryAccounting": "yes", "MemoryCurrent": "2953216", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-20T12:00:06.719 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 11:56:24 UTC", "StateChangeTimestampMonotonic": "16132638", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 11:56:24 UTC", "WatchdogTimestampMonotonic": "15970942", "WatchdogUSec": "0"}} 2020-11-20T12:00:06.763 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:06 +0000 (0:00:00.298) 0:02:02.687 ******* 2020-11-20T12:00:06.793 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T12:00:06.838 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:06 +0000 (0:00:00.073) 0:02:02.761 ******* 2020-11-20T12:00:06.879 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-20T12:00:06.916 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:06 +0000 (0:00:00.078) 0:02:02.840 ******* 2020-11-20T12:00:06.949 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:06.986 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:06 +0000 (0:00:00.070) 0:02:02.910 ******* 2020-11-20T12:00:07.029 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:07.073 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:07 +0000 (0:00:00.086) 0:02:02.997 ******* 2020-11-20T12:00:07.102 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-20T12:00:07.104 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:07.152 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:07 +0000 (0:00:00.079) 0:02:03.076 ******* 2020-11-20T12:00:07.362 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-20T12:00:07.363 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T12:00:07.399 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:07 +0000 (0:00:00.246) 0:02:03.322 ******* 2020-11-20T12:00:07.824 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T12:00:07.870 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:07 +0000 (0:00:00.471) 0:02:03.793 ******* 2020-11-20T12:00:07.897 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-20T12:00:07.899 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:07.939 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:07 +0000 (0:00:00.069) 0:02:03.863 ******* 2020-11-20T12:00:07.955 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-20T12:00:07.958 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-20T12:00:07.990 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:07 +0000 (0:00:00.050) 0:02:03.914 ******* 2020-11-20T12:00:07.997 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-20T12:00:07.999 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-20T12:00:08.033 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:08 +0000 (0:00:00.042) 0:02:03.957 ******* 2020-11-20T12:00:08.246 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-20T12:00:08.248 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-20T12:00:08.299 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:08 +0000 (0:00:00.266) 0:02:04.223 ******* 2020-11-20T12:00:08.327 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-20T12:00:08.328 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:08.371 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:08 +0000 (0:00:00.071) 0:02:04.295 ******* 2020-11-20T12:00:08.400 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-20T12:00:08.402 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T12:00:08.434 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:08 +0000 (0:00:00.062) 0:02:04.358 ******* 2020-11-20T12:00:08.472 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-20T12:00:08.474 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:08.507 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:08 +0000 (0:00:00.073) 0:02:04.431 ******* 2020-11-20T12:00:08.731 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi079.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T12:00:08.777 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:08 +0000 (0:00:00.269) 0:02:04.700 ******* 2020-11-20T12:00:08.997 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-20T12:00:08.999 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T12:00:09.049 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.272) 0:02:04.973 ******* 2020-11-20T12:00:09.078 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-20T12:00:09.079 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.118 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.068) 0:02:05.041 ******* 2020-11-20T12:00:09.155 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-20T12:00:09.157 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.167 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.169 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.202 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.084) 0:02:05.126 ******* 2020-11-20T12:00:09.242 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi079.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.253 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.254 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T12:00:09.289 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.085) 0:02:05.212 ******* 2020-11-20T12:00:09.296 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-20T12:00:09.298 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi079.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.332 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.044) 0:02:05.256 ******* 2020-11-20T12:00:09.339 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-20T12:00:09.341 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.375 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.042) 0:02:05.299 ******* 2020-11-20T12:00:09.383 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:00:09.385 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.420 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.044) 0:02:05.344 ******* 2020-11-20T12:00:09.427 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-20T12:00:09.430 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-11-20T12:00:09.462 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.041) 0:02:05.386 ******* 2020-11-20T12:00:09.470 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.507 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.044) 0:02:05.431 ******* 2020-11-20T12:00:09.535 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-20T12:00:09.537 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.582 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.075) 0:02:05.506 ******* 2020-11-20T12:00:09.610 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-20T12:00:09.612 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.644 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.061) 0:02:05.568 ******* 2020-11-20T12:00:09.662 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi044.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-20T12:00:09.697 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.052) 0:02:05.620 ******* 2020-11-20T12:00:09.702 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-20T12:00:09.704 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.740 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.043) 0:02:05.664 ******* 2020-11-20T12:00:09.746 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-20T12:00:09.748 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.782 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.041) 0:02:05.706 ******* 2020-11-20T12:00:09.790 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T12:00:09.791 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.828 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.046) 0:02:05.752 ******* 2020-11-20T12:00:09.837 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-20T12:00:09.838 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T12:00:09.872 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.043) 0:02:05.796 ******* 2020-11-20T12:00:09.878 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-20T12:00:09.880 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.919 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.046) 0:02:05.842 ******* 2020-11-20T12:00:09.925 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-20T12:00:09.931 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:09.972 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:09 +0000 (0:00:00.052) 0:02:05.895 ******* 2020-11-20T12:00:09.979 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T12:00:09.981 INFO:teuthology.task.ansible.out:skipping: [smithi079.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.015 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.043) 0:02:05.939 ******* 2020-11-20T12:00:10.219 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-20T12:00:10.220 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T12:00:10.254 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.238) 0:02:06.178 ******* 2020-11-20T12:00:10.488 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-20T12:00:10.491 INFO:teuthology.task.ansible.out:changed: [smithi044.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T12:00:10.523 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.268) 0:02:06.447 ******* 2020-11-20T12:00:10.551 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-20T12:00:10.553 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.592 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.069) 0:02:06.516 ******* 2020-11-20T12:00:10.648 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi044.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.655 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.656 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.666 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.668 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.701 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.109) 0:02:06.625 ******* 2020-11-20T12:00:10.747 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-20T12:00:10.749 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.761 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi044.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.773 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.775 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.813 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.111) 0:02:06.736 ******* 2020-11-20T12:00:10.839 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:00:10.841 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.876 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.063) 0:02:06.800 ******* 2020-11-20T12:00:10.903 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:10.947 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:10 +0000 (0:00:00.070) 0:02:06.870 ******* 2020-11-20T12:00:10.980 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:11.034 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.083) 0:02:06.954 ******* 2020-11-20T12:00:11.065 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:11.112 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.081) 0:02:07.035 ******* 2020-11-20T12:00:11.138 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:11.183 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.072) 0:02:07.107 ******* 2020-11-20T12:00:11.211 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:11.247 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.063) 0:02:07.171 ******* 2020-11-20T12:00:11.264 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T12:00:11.266 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:01.006041", "end": "2020-11-20 12:00:11.222718", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 12:00:10.216677", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:11.279 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-20T12:00:11.281 INFO:teuthology.task.ansible.out:skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:11.317 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.068) 0:02:07.239 ******* 2020-11-20T12:00:11.343 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi044.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:00:11.388 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.071) 0:02:07.311 ******* 2020-11-20T12:00:11.446 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi079.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003071", "end": "2020-11-20 12:00:11.416737", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 12:00:11.413666", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:11.633 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002782", "end": "2020-11-20 12:00:11.595316", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 12:00:11.592534", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:11.828 INFO:teuthology.task.ansible.out:ok: [smithi079.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.023702", "end": "2020-11-20 12:00:11.795494", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 12:00:11.771792", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:11.863 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.475) 0:02:07.787 ******* 2020-11-20T12:00:11.891 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-20T12:00:11.927 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:11 +0000 (0:00:00.064) 0:02:07.851 ******* 2020-11-20T12:00:12.123 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-20T12:00:12.125 INFO:teuthology.task.ansible.out:changed: [smithi079.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T12:00:12.629 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T12:00:12.631 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:01.006509", "end": "2020-11-20 12:00:12.586102", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 12:00:11.579593", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:12.829 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003150", "end": "2020-11-20 12:00:12.797886", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 12:00:12.794736", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:13.030 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002876", "end": "2020-11-20 12:00:12.988701", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 12:00:12.985825", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:13.228 INFO:teuthology.task.ansible.out:ok: [smithi044.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.016714", "end": "2020-11-20 12:00:13.199907", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 12:00:13.183193", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:00:13.263 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:13 +0000 (0:00:01.335) 0:02:09.186 ******* 2020-11-20T12:00:13.290 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-20T12:00:13.323 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:00:13 +0000 (0:00:00.061) 0:02:09.247 ******* 2020-11-20T12:00:13.531 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi044.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T12:00:13.624 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-11-20T12:00:13.628 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-20T12:00:13.631 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-11-20T12:00:13.633 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi044.front.sepia.ceph.com : ok=115 changed=33 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 smithi079.front.sepia.ceph.com : ok=115 changed=33 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 Friday 20 November 2020 12:00:13 +0000 (0:00:00.308) 0:02:09.556 ******* =============================================================================== common : nrpe - Remove SELinux policy package -------------------------- 15.64s common : nrpe - Load SELinux policy package ---------------------------- 14.88s testnode : Zap all non-root disks --------------------------------------- 4.94s common : Make sure smartmontools is installed --------------------------- 2.65s testnode : Ensure ceph dependency packages are not present. ------------- 2.04s testnode : Ensure ceph packages are not present. ------------------------ 1.57s Gathering Facts --------------------------------------------------------- 1.56s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.55s testnode : Rebuild rpmdb ------------------------------------------------ 1.28s testnode : Configure local mirrorlists ---------------------------------- 1.20s common : nrpe - Ensure SELinux policy is up to date --------------------- 1.07s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.06s testnode : Make sure rpm dependencies are installed --------------------- 1.05s common : nrpe - Install SELinux dependencies ---------------------------- 0.92s Gathering Facts --------------------------------------------------------- 0.74s ansible-managed : Add authorized keys for the ansible user. ------------- 0.69s testnode : Write /scratch_devs ------------------------------------------ 0.66s common : Add krb5 config file ------------------------------------------- 0.58s testnode : Configure /etc/sudoers. ----------- 2020-11-20T12:00:13.635 INFO:teuthology.task.ansible.out:--------------------------- 0.57s testnode : Create volume group(s) --------------------------------------- 0.52s 2020-11-20T12:00:13.970 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi079.front.sepia.ceph.com')] 2020-11-20T12:00:13.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:00:13.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:00:14.043 INFO:teuthology.orchestra.run.smithi044:> true 2020-11-20T12:00:14.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:00:14.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:00:14.207 INFO:teuthology.orchestra.run.smithi079:> true 2020-11-20T12:00:14.294 INFO:teuthology.run_tasks:Running task clock... 2020-11-20T12:00:14.305 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-20T12:00:14.305 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T12:00:14.306 INFO:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T12:00:14.351 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-20T12:00:14.373 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-20T12:00:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2020-11-20T12:00:14.439 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2020-11-20T12:00:14.461 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-20T12:00:14.481 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-20T12:00:14.548 INFO:teuthology.orchestra.run.smithi044.stderr:bash: ntpq: command not found 2020-11-20T12:00:14.601 INFO:teuthology.orchestra.run.smithi044.stdout:210 Number of sources = 4 2020-11-20T12:00:14.602 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-20T12:00:14.602 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2020-11-20T12:00:14.602 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.603 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.603 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.603 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.603 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T12:00:14.604 INFO:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T12:00:14.664 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-20T12:00:14.684 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-20T12:00:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2020-11-20T12:00:14.743 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2020-11-20T12:00:14.761 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-20T12:00:14.780 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-20T12:00:14.837 INFO:teuthology.orchestra.run.smithi079.stderr:bash: ntpq: command not found 2020-11-20T12:00:14.838 INFO:teuthology.orchestra.run.smithi079.stdout:210 Number of sources = 4 2020-11-20T12:00:14.838 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-20T12:00:14.839 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2020-11-20T12:00:14.839 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.839 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.839 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.840 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T12:00:14.840 INFO:teuthology.run_tasks:Running task install... 2020-11-20T12:00:14.853 DEBUG:teuthology.task.install:project ceph 2020-11-20T12:00:14.854 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}} 2020-11-20T12:00:14.854 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'} 2020-11-20T12:00:14.855 INFO:teuthology.task.install:Using flavor: basic 2020-11-20T12:00:14.862 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-20T12:00:14.862 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-11-20T12:00:14.863 INFO:teuthology.orchestra.run.smithi044:> sudo lsb_release -is 2020-11-20T12:00:14.908 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS 2020-11-20T12:00:14.910 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T12:00:14.910 INFO:teuthology.orchestra.run.smithi079:> sudo lsb_release -is 2020-11-20T12:00:14.911 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb', 'tag': None, 'wait_for_package': False} 2020-11-20T12:00:14.912 INFO:teuthology.orchestra.run.smithi044:> sudo lsb_release -is 2020-11-20T12:00:14.952 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS 2020-11-20T12:00:14.954 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T12:00:14.954 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb', 'tag': None, 'wait_for_package': False} 2020-11-20T12:00:14.954 INFO:teuthology.orchestra.run.smithi079:> sudo lsb_release -is 2020-11-20T12:00:14.997 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS 2020-11-20T12:00:14.999 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T12:00:14.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:00:15.037 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS 2020-11-20T12:00:15.037 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T12:00:15.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:00:15.176 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/ 2020-11-20T12:00:15.176 INFO:teuthology.task.install.rpm:Package version is 16.0.0-7397.gb86e691b 2020-11-20T12:00:15.200 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/ 2020-11-20T12:00:15.201 INFO:teuthology.task.install.rpm:Package version is 16.0.0-7397.gb86e691b 2020-11-20T12:00:15.307 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-20T12:00:15.308 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:00:15.309 INFO:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T12:00:15.324 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T12:00:15.333 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-20T12:00:15.335 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:00:15.336 INFO:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T12:00:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:1+1 records in 2020-11-20T12:00:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:1+1 records out 2020-11-20T12:00:15.343 INFO:teuthology.orchestra.run.smithi044.stderr:676 bytes copied, 4.2699e-05 s, 15.8 MB/s 2020-11-20T12:00:15.344 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-20T12:00:15.344 INFO:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b86e691bf191566922a8f08e2a5abf64fde8cfcb/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-20T12:00:15.356 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T12:00:15.369 INFO:teuthology.orchestra.run.smithi079.stderr:1+1 records in 2020-11-20T12:00:15.369 INFO:teuthology.orchestra.run.smithi079.stderr:1+1 records out 2020-11-20T12:00:15.370 INFO:teuthology.orchestra.run.smithi079.stderr:676 bytes copied, 3.4921e-05 s, 19.4 MB/s 2020-11-20T12:00:15.370 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-20T12:00:15.371 INFO:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b86e691bf191566922a8f08e2a5abf64fde8cfcb/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-20T12:00:15.417 INFO:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-20T12:00:15.439 INFO:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-20T12:00:15.506 INFO:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-20T12:00:15.521 INFO:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-20T12:00:15.536 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2020-11-20T12:00:15.539 INFO:teuthology.orchestra.run.smithi044:> sudo yum clean all 2020-11-20T12:00:15.587 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2020-11-20T12:00:15.589 INFO:teuthology.orchestra.run.smithi079:> sudo yum clean all 2020-11-20T12:00:15.834 INFO:teuthology.orchestra.run.smithi044.stdout:42 files removed 2020-11-20T12:00:15.864 INFO:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-20T12:00:15.866 INFO:teuthology.orchestra.run.smithi079.stdout:42 files removed 2020-11-20T12:00:15.895 INFO:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-20T12:00:16.490 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 286 kB/s | 72 kB 00:00 2020-11-20T12:00:16.493 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 291 kB/s | 72 kB 00:00 2020-11-20T12:00:16.730 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 72 kB/s | 15 kB 00:00 2020-11-20T12:00:16.733 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 71 kB/s | 15 kB 00:00 2020-11-20T12:00:16.958 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-11-20T12:00:16.964 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-11-20T12:00:17.154 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-8 - AppStream 40 MB/s | 7.0 MB 00:00 2020-11-20T12:00:17.185 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-8 - AppStream 35 MB/s | 7.0 MB 00:00 2020-11-20T12:00:18.340 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-20T12:00:18.448 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-8 - Base 22 MB/s | 2.2 MB 00:00 2020-11-20T12:00:18.775 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-8 - Extras 87 kB/s | 5.5 kB 00:00 2020-11-20T12:00:18.899 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-8 - Extras 85 kB/s | 5.5 kB 00:00 2020-11-20T12:00:19.146 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 26 MB/s | 8.3 MB 00:00 2020-11-20T12:00:19.330 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 22 MB/s | 8.3 MB 00:00 2020-11-20T12:00:20.895 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 5.1 MB/s | 23 kB 00:00 2020-11-20T12:00:21.225 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 4.5 MB/s | 23 kB 00:00 2020-11-20T12:00:21.809 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T12:00:21.810 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T12:00:21.815 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.20.0-3.el8.x86_64 is already installed. 2020-11-20T12:00:21.816 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-4.18.0-15.el8.x86_64 is already installed. 2020-11-20T12:00:21.816 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.32.1-17.el8.x86_64 is already installed. 2020-11-20T12:00:21.817 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-11-20T12:00:21.817 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-11-20T12:00:21.819 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-11-20T12:00:21.819 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-11-20T12:00:21.950 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2020-11-20T12:00:21.977 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2020-11-20T12:00:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: Package ArchVersion Repository Size 2020-11-20T12:00:21.978 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2020-11-20T12:00:21.979 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2020-11-20T12:00:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_642:16.0.0-7397.gb86e691b.el8 ceph 6.4 k 2020-11-20T12:00:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_642:16.0.0-7397.gb86e691b.el8 ceph 6.0 M 2020-11-20T12:00:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_642:16.0.0-7397.gb86e691b.el8 ceph 632 k 2020-11-20T12:00:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_642:16.0.0-7397.gb86e691b.el8 ceph 204 k 2020-11-20T12:00:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_642:16.0.0-7397.gb86e691b.el8 ceph 1.5 M 2020-11-20T12:00:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_642:16.0.0-7397.gb86e691b.el8 ceph 12 M 2020-11-20T12:00:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_642:16.0.0-7397.gb86e691b.el8 ceph 48 M 2020-11-20T12:00:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_642:16.0.0-7397.gb86e691b.el8 ceph 24 k 2020-11-20T12:00:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_642:16.0.0-7397.gb86e691b.el8 ceph 562 k 2020-11-20T12:00:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_642:16.0.0-7397.gb86e691b.el8 ceph 131 k 2020-11-20T12:00:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_642:16.0.0-7397.gb86e691b.el8 ceph 204 k 2020-11-20T12:00:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_642:16.0.0-7397.gb86e691b.el8 ceph 375 k 2020-11-20T12:00:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_642:16.0.0-7397.gb86e691b.el8 ceph 353 k 2020-11-20T12:00:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_642:16.0.0-7397.gb86e691b.el8 ceph 113 k 2020-11-20T12:00:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_642:16.0.0-7397.gb86e691b.el8 ceph 88 k 2020-11-20T12:00:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_642:16.0.0-7397.gb86e691b.el8 ceph 3.2 M 2020-11-20T12:00:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_642:16.0.0-7397.gb86e691b.el8 ceph 171 k 2020-11-20T12:00:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 66 k 2020-11-20T12:00:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 3.2 M 2020-11-20T12:00:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 8.6 M 2020-11-20T12:00:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 27 k 2020-11-20T12:00:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 57 k 2020-11-20T12:00:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-20T12:00:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-20T12:00:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-20T12:00:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-20T12:00:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-20T12:00:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-20T12:00:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-20T12:00:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-20T12:00:21.992 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2020-11-20T12:00:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_642:16.0.0-7397.gb86e691b.el8 ceph 3.7 M 2020-11-20T12:00:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_642:16.0.0-7397.gb86e691b.el8 ceph 2.4 M 2020-11-20T12:00:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager x86_641:1.20.0-5.el8_1 CentOS-Base 2.1 M 2020-11-20T12:00:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm x86_641:1.20.0-5.el8_1 CentOS-Base 1.7 M 2020-11-20T12:00:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team x86_641:1.20.0-5.el8_1 CentOS-Base 133 k 2020-11-20T12:00:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui x86_641:1.20.0-5.el8_1 CentOS-Base 308 k 2020-11-20T12:00:21.994 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2020-11-20T12:00:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_642:16.0.0-7397.gb86e691b.el8 ceph 23 M 2020-11-20T12:00:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_642:16.0.0-7397.gb86e691b.el8 ceph 2.1 M 2020-11-20T12:00:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_642:16.0.0-7397.gb86e691b.el8 ceph 5.0 M 2020-11-20T12:00:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_642:16.0.0-7397.gb86e691b.el8 ceph 18 M 2020-11-20T12:00:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_642:16.0.0-7397.gb86e691b.el8 ceph 25 k 2020-11-20T12:00:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_642:16.0.0-7397.gb86e691b.el8 ceph 486 k 2020-11-20T12:00:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_642:16.0.0-7397.gb86e691b.el8 ceph 3.7 M 2020-11-20T12:00:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_642:16.0.0-7397.gb86e691b.el8 ceph 40 k 2020-11-20T12:00:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_642:16.0.0-7397.gb86e691b.el8 ceph 73 k 2020-11-20T12:00:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 24 k 2020-11-20T12:00:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 198 k 2020-11-20T12:00:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 8.8 k 2020-11-20T12:00:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-20T12:00:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-20T12:00:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-20T12:00:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-20T12:00:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-20T12:00:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-20T12:00:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-20T12:00:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-20T12:00:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-20T12:00:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-20T12:00:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-20T12:00:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-20T12:00:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-20T12:00:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-20T12:00:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-20T12:00:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-20T12:00:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-20T12:00:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-20T12:00:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-20T12:00:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-20T12:00:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-20T12:00:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-20T12:00:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-20T12:00:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-20T12:00:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-20T12:00:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-20T12:00:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-20T12:00:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-20T12:00:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-20T12:00:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-20T12:00:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-20T12:00:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-20T12:00:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-20T12:00:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-20T12:00:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-20T12:00:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-20T12:00:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-20T12:00:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-20T12:00:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-20T12:00:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-20T12:00:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-20T12:00:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-20T12:00:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-20T12:00:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-20T12:00:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-20T12:00:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-20T12:00:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-20T12:00:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-20T12:00:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-20T12:00:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-20T12:00:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-20T12:00:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-20T12:00:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-20T12:00:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-20T12:00:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-20T12:00:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-20T12:00:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-20T12:00:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-20T12:00:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-20T12:00:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-20T12:00:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-20T12:00:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-20T12:00:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-20T12:00:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-20T12:00:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-20T12:00:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-20T12:00:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-20T12:00:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-20T12:00:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-20T12:00:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-20T12:00:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-20T12:00:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-20T12:00:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-20T12:00:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-20T12:00:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-20T12:00:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-20T12:00:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-20T12:00:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-20T12:00:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-20T12:00:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-20T12:00:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-20T12:00:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-20T12:00:22.014 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2020-11-20T12:00:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-20T12:00:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-20T12:00:22.015 INFO:teuthology.orchestra.run.smithi079.stdout:Downgrading: 2020-11-20T12:00:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-20T12:00:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-20T12:00:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-20T12:00:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-20T12:00:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-20T12:00:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:00:22.016 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2020-11-20T12:00:22.016 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2020-11-20T12:00:22.017 INFO:teuthology.orchestra.run.smithi079.stdout:Install 126 Packages 2020-11-20T12:00:22.017 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 6 Packages 2020-11-20T12:00:22.017 INFO:teuthology.orchestra.run.smithi079.stdout:Downgrade 5 Packages 2020-11-20T12:00:22.017 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:00:22.018 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 202 M 2020-11-20T12:00:22.018 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2020-11-20T12:00:22.041 INFO:teuthology.orchestra.run.smithi079.stdout:(1/137): krb5-libs-1.17-9.el8.x86_64.rpm 18 MB/s | 851 kB 00:00 2020-11-20T12:00:22.042 INFO:teuthology.orchestra.run.smithi079.stdout:(2/137): libkadm5-1.17-9.el8.x86_64.rpm 3.8 MB/s | 184 kB 00:00 2020-11-20T12:00:22.054 INFO:teuthology.orchestra.run.smithi079.stdout:(3/137): krb5-workstation-1.17-9.el8.x86_64.rpm 16 MB/s | 940 kB 00:00 2020-11-20T12:00:22.057 INFO:teuthology.orchestra.run.smithi079.stdout:(4/137): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 41 MB/s | 686 kB 00:00 2020-11-20T12:00:22.073 INFO:teuthology.orchestra.run.smithi079.stdout:(5/137): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 47 MB/s | 1.5 MB 00:00 2020-11-20T12:00:22.154 INFO:teuthology.orchestra.run.smithi079.stdout:(6/137): ceph-16.0.0-7397.gb86e691b.el8.x86_64. 64 kB/s | 6.4 kB 00:00 2020-11-20T12:00:22.197 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T12:00:22.197 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T12:00:22.201 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.20.0-3.el8.x86_64 is already installed. 2020-11-20T12:00:22.202 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-4.18.0-15.el8.x86_64 is already installed. 2020-11-20T12:00:22.203 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.32.1-17.el8.x86_64 is already installed. 2020-11-20T12:00:22.203 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-11-20T12:00:22.204 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-11-20T12:00:22.204 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-11-20T12:00:22.205 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-11-20T12:00:22.297 INFO:teuthology.orchestra.run.smithi079.stdout:(7/137): ceph-fuse-16.0.0-7397.gb86e691b.el8.x8 4.3 MB/s | 632 kB 00:00 2020-11-20T12:00:22.329 INFO:teuthology.orchestra.run.smithi079.stdout:(8/137): ceph-immutable-object-cache-16.0.0-739 6.6 MB/s | 204 kB 00:00 2020-11-20T12:00:22.345 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2020-11-20T12:00:22.369 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2020-11-20T12:00:22.372 INFO:teuthology.orchestra.run.smithi044.stdout: Package ArchVersion Repository Size 2020-11-20T12:00:22.373 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2020-11-20T12:00:22.373 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2020-11-20T12:00:22.373 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_642:16.0.0-7397.gb86e691b.el8 ceph 6.4 k 2020-11-20T12:00:22.373 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_642:16.0.0-7397.gb86e691b.el8 ceph 6.0 M 2020-11-20T12:00:22.374 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_642:16.0.0-7397.gb86e691b.el8 ceph 632 k 2020-11-20T12:00:22.374 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_642:16.0.0-7397.gb86e691b.el8 ceph 204 k 2020-11-20T12:00:22.374 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_642:16.0.0-7397.gb86e691b.el8 ceph 1.5 M 2020-11-20T12:00:22.374 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_642:16.0.0-7397.gb86e691b.el8 ceph 12 M 2020-11-20T12:00:22.375 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_642:16.0.0-7397.gb86e691b.el8 ceph 48 M 2020-11-20T12:00:22.375 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_642:16.0.0-7397.gb86e691b.el8 ceph 24 k 2020-11-20T12:00:22.375 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_642:16.0.0-7397.gb86e691b.el8 ceph 562 k 2020-11-20T12:00:22.375 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_642:16.0.0-7397.gb86e691b.el8 ceph 131 k 2020-11-20T12:00:22.375 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_642:16.0.0-7397.gb86e691b.el8 ceph 204 k 2020-11-20T12:00:22.376 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_642:16.0.0-7397.gb86e691b.el8 ceph 375 k 2020-11-20T12:00:22.376 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_642:16.0.0-7397.gb86e691b.el8 ceph 353 k 2020-11-20T12:00:22.376 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_642:16.0.0-7397.gb86e691b.el8 ceph 113 k 2020-11-20T12:00:22.376 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_642:16.0.0-7397.gb86e691b.el8 ceph 88 k 2020-11-20T12:00:22.376 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_642:16.0.0-7397.gb86e691b.el8 ceph 3.2 M 2020-11-20T12:00:22.377 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_642:16.0.0-7397.gb86e691b.el8 ceph 171 k 2020-11-20T12:00:22.377 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 66 k 2020-11-20T12:00:22.377 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 3.2 M 2020-11-20T12:00:22.377 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 8.6 M 2020-11-20T12:00:22.378 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 27 k 2020-11-20T12:00:22.378 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 57 k 2020-11-20T12:00:22.378 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-20T12:00:22.378 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-20T12:00:22.378 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-20T12:00:22.379 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-20T12:00:22.379 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-20T12:00:22.379 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-20T12:00:22.379 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-20T12:00:22.379 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-20T12:00:22.380 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2020-11-20T12:00:22.380 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_642:16.0.0-7397.gb86e691b.el8 ceph 3.7 M 2020-11-20T12:00:22.380 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_642:16.0.0-7397.gb86e691b.el8 ceph 2.4 M 2020-11-20T12:00:22.380 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager x86_641:1.20.0-5.el8_1 CentOS-Base 2.1 M 2020-11-20T12:00:22.380 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm x86_641:1.20.0-5.el8_1 CentOS-Base 1.7 M 2020-11-20T12:00:22.381 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team x86_641:1.20.0-5.el8_1 CentOS-Base 133 k 2020-11-20T12:00:22.381 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui x86_641:1.20.0-5.el8_1 CentOS-Base 308 k 2020-11-20T12:00:22.381 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2020-11-20T12:00:22.381 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_642:16.0.0-7397.gb86e691b.el8 ceph 23 M 2020-11-20T12:00:22.382 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_642:16.0.0-7397.gb86e691b.el8 ceph 2.1 M 2020-11-20T12:00:22.382 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_642:16.0.0-7397.gb86e691b.el8 ceph 5.0 M 2020-11-20T12:00:22.382 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_642:16.0.0-7397.gb86e691b.el8 ceph 18 M 2020-11-20T12:00:22.382 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_642:16.0.0-7397.gb86e691b.el8 ceph 25 k 2020-11-20T12:00:22.382 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_642:16.0.0-7397.gb86e691b.el8 ceph 486 k 2020-11-20T12:00:22.383 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_642:16.0.0-7397.gb86e691b.el8 ceph 3.7 M 2020-11-20T12:00:22.383 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_642:16.0.0-7397.gb86e691b.el8 ceph 40 k 2020-11-20T12:00:22.383 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_642:16.0.0-7397.gb86e691b.el8 ceph 73 k 2020-11-20T12:00:22.383 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 24 k 2020-11-20T12:00:22.383 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 198 k 2020-11-20T12:00:22.384 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch2:16.0.0-7397.gb86e691b.el8 ceph-noarch 8.8 k 2020-11-20T12:00:22.384 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-20T12:00:22.384 INFO:teuthology.orchestra.run.smithi044.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-20T12:00:22.384 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-20T12:00:22.384 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-20T12:00:22.385 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-20T12:00:22.385 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-20T12:00:22.385 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-20T12:00:22.385 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-20T12:00:22.385 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-20T12:00:22.386 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-20T12:00:22.386 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-20T12:00:22.386 INFO:teuthology.orchestra.run.smithi044.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-20T12:00:22.386 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-20T12:00:22.386 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-20T12:00:22.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-20T12:00:22.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-20T12:00:22.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-20T12:00:22.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-20T12:00:22.387 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-20T12:00:22.388 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-20T12:00:22.388 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-20T12:00:22.388 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-20T12:00:22.388 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-20T12:00:22.388 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-20T12:00:22.389 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-20T12:00:22.389 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-20T12:00:22.389 INFO:teuthology.orchestra.run.smithi044.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-20T12:00:22.389 INFO:teuthology.orchestra.run.smithi044.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-20T12:00:22.390 INFO:teuthology.orchestra.run.smithi044.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-20T12:00:22.390 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-20T12:00:22.390 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-20T12:00:22.390 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-20T12:00:22.390 INFO:teuthology.orchestra.run.smithi044.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-20T12:00:22.391 INFO:teuthology.orchestra.run.smithi044.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-20T12:00:22.391 INFO:teuthology.orchestra.run.smithi044.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-20T12:00:22.391 INFO:teuthology.orchestra.run.smithi044.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-20T12:00:22.391 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-20T12:00:22.391 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-20T12:00:22.392 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-20T12:00:22.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-20T12:00:22.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-20T12:00:22.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-20T12:00:22.392 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-20T12:00:22.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-20T12:00:22.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-20T12:00:22.393 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-20T12:00:22.393 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-20T12:00:22.393 INFO:teuthology.orchestra.run.smithi044.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-20T12:00:22.394 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-20T12:00:22.394 INFO:teuthology.orchestra.run.smithi044.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-20T12:00:22.394 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-20T12:00:22.394 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-20T12:00:22.394 INFO:teuthology.orchestra.run.smithi044.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-20T12:00:22.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-20T12:00:22.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-20T12:00:22.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-20T12:00:22.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-20T12:00:22.395 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-20T12:00:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-20T12:00:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-20T12:00:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-20T12:00:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-20T12:00:22.396 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-20T12:00:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-20T12:00:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-20T12:00:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-20T12:00:22.397 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-20T12:00:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-20T12:00:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-20T12:00:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-20T12:00:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-20T12:00:22.398 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-20T12:00:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-20T12:00:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-20T12:00:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-20T12:00:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-20T12:00:22.399 INFO:teuthology.orchestra.run.smithi044.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-20T12:00:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-20T12:00:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-20T12:00:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-20T12:00:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-20T12:00:22.400 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-20T12:00:22.401 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2020-11-20T12:00:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-20T12:00:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-20T12:00:22.401 INFO:teuthology.orchestra.run.smithi044.stdout:Downgrading: 2020-11-20T12:00:22.401 INFO:teuthology.orchestra.run.smithi044.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-20T12:00:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-20T12:00:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-20T12:00:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-20T12:00:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-20T12:00:22.402 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:00:22.403 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2020-11-20T12:00:22.403 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2020-11-20T12:00:22.403 INFO:teuthology.orchestra.run.smithi044.stdout:Install 126 Packages 2020-11-20T12:00:22.403 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 6 Packages 2020-11-20T12:00:22.403 INFO:teuthology.orchestra.run.smithi044.stdout:Downgrade 5 Packages 2020-11-20T12:00:22.404 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:00:22.404 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 202 M 2020-11-20T12:00:22.405 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2020-11-20T12:00:22.573 INFO:teuthology.orchestra.run.smithi079.stdout:(9/137): ceph-mds-16.0.0-7397.gb86e691b.el8.x86 8.8 MB/s | 2.1 MB 00:00 2020-11-20T12:00:22.769 INFO:teuthology.orchestra.run.smithi079.stdout:(10/137): ceph-mgr-16.0.0-7397.gb86e691b.el8.x8 7.8 MB/s | 1.5 MB 00:00 2020-11-20T12:00:22.806 INFO:teuthology.orchestra.run.smithi079.stdout:(11/137): ceph-base-16.0.0-7397.gb86e691b.el8.x 8.0 MB/s | 6.0 MB 00:00 2020-11-20T12:00:22.826 INFO:teuthology.orchestra.run.smithi044.stdout:(1/137): krb5-libs-1.17-9.el8.x86_64.rpm 1.9 MB/s | 851 kB 00:00 2020-11-20T12:00:22.835 INFO:teuthology.orchestra.run.smithi044.stdout:(2/137): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 77 MB/s | 686 kB 00:00 2020-11-20T12:00:22.854 INFO:teuthology.orchestra.run.smithi044.stdout:(3/137): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 80 MB/s | 1.5 MB 00:00 2020-11-20T12:00:23.128 INFO:teuthology.orchestra.run.smithi044.stdout:(4/137): ceph-16.0.0-7397.gb86e691b.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2020-11-20T12:00:23.338 INFO:teuthology.orchestra.run.smithi079.stdout:(12/137): ceph-mon-16.0.0-7397.gb86e691b.el8.x8 8.7 MB/s | 5.0 MB 00:00 2020-11-20T12:00:23.831 INFO:teuthology.orchestra.run.smithi044.stdout:(5/137): krb5-workstation-1.17-9.el8.x86_64.rpm 650 kB/s | 940 kB 00:01 2020-11-20T12:00:24.212 INFO:teuthology.orchestra.run.smithi044.stdout:(6/137): libkadm5-1.17-9.el8.x86_64.rpm 101 kB/s | 184 kB 00:01 2020-11-20T12:00:24.631 INFO:teuthology.orchestra.run.smithi079.stdout:(13/137): ceph-common-16.0.0-7397.gb86e691b.el8 9.0 MB/s | 23 MB 00:02 2020-11-20T12:00:24.767 INFO:teuthology.orchestra.run.smithi079.stdout:(14/137): ceph-osd-16.0.0-7397.gb86e691b.el8.x8 9.1 MB/s | 18 MB 00:01 2020-11-20T12:00:24.768 INFO:teuthology.orchestra.run.smithi079.stdout:(15/137): ceph-selinux-16.0.0-7397.gb86e691b.el 181 kB/s | 25 kB 00:00 2020-11-20T12:00:24.807 INFO:teuthology.orchestra.run.smithi044.stdout:(7/137): ceph-fuse-16.0.0-7397.gb86e691b.el8.x8 1.0 MB/s | 632 kB 00:00 2020-11-20T12:00:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:(16/137): ceph-radosgw-16.0.0-7397.gb86e691b.el 7.7 MB/s | 12 MB 00:01 2020-11-20T12:00:24.873 INFO:teuthology.orchestra.run.smithi079.stdout:(17/137): libcephfs-devel-16.0.0-7397.gb86e691b 228 kB/s | 24 kB 00:00 2020-11-20T12:00:24.889 INFO:teuthology.orchestra.run.smithi044.stdout:(8/137): ceph-immutable-object-cache-16.0.0-739 2.4 MB/s | 204 kB 00:00 2020-11-20T12:00:24.902 INFO:teuthology.orchestra.run.smithi079.stdout:(18/137): librados-devel-16.0.0-7397.gb86e691b. 4.2 MB/s | 131 kB 00:00 2020-11-20T12:00:24.927 INFO:teuthology.orchestra.run.smithi079.stdout:(19/137): libcephfs2-16.0.0-7397.gb86e691b.el8. 9.9 MB/s | 562 kB 00:00 2020-11-20T12:00:24.975 INFO:teuthology.orchestra.run.smithi079.stdout:(20/137): libradosstriper1-16.0.0-7397.gb86e691 6.5 MB/s | 486 kB 00:00 2020-11-20T12:00:25.035 INFO:teuthology.orchestra.run.smithi079.stdout:(21/137): python3-ceph-argparse-16.0.0-7397.gb8 666 kB/s | 40 kB 00:00 2020-11-20T12:00:25.099 INFO:teuthology.orchestra.run.smithi079.stdout:(22/137): python3-ceph-common-16.0.0-7397.gb86e 1.1 MB/s | 73 kB 00:00 2020-11-20T12:00:25.169 INFO:teuthology.orchestra.run.smithi079.stdout:(23/137): python3-cephfs-16.0.0-7397.gb86e691b. 2.8 MB/s | 204 kB 00:00 2020-11-20T12:00:25.242 INFO:teuthology.orchestra.run.smithi079.stdout:(24/137): python3-rados-16.0.0-7397.gb86e691b.e 5.1 MB/s | 375 kB 00:00 2020-11-20T12:00:25.317 INFO:teuthology.orchestra.run.smithi079.stdout:(25/137): python3-rbd-16.0.0-7397.gb86e691b.el8 4.6 MB/s | 353 kB 00:00 2020-11-20T12:00:25.341 INFO:teuthology.orchestra.run.smithi079.stdout:(26/137): librgw2-16.0.0-7397.gb86e691b.el8.x86 9.0 MB/s | 3.7 MB 00:00 2020-11-20T12:00:25.374 INFO:teuthology.orchestra.run.smithi079.stdout:(27/137): python3-rgw-16.0.0-7397.gb86e691b.el8 2.0 MB/s | 113 kB 00:00 2020-11-20T12:00:25.390 INFO:teuthology.orchestra.run.smithi079.stdout:(28/137): rbd-fuse-16.0.0-7397.gb86e691b.el8.x8 1.8 MB/s | 88 kB 00:00 2020-11-20T12:00:25.462 INFO:teuthology.orchestra.run.smithi079.stdout:(29/137): rbd-nbd-16.0.0-7397.gb86e691b.el8.x86 2.3 MB/s | 171 kB 00:00 2020-11-20T12:00:25.531 INFO:teuthology.orchestra.run.smithi079.stdout:(30/137): ceph-grafana-dashboards-16.0.0-7397.g 353 kB/s | 24 kB 00:00 2020-11-20T12:00:25.601 INFO:teuthology.orchestra.run.smithi079.stdout:(31/137): ceph-mgr-cephadm-16.0.0-7397.gb86e691 946 kB/s | 66 kB 00:00 2020-11-20T12:00:25.727 INFO:teuthology.orchestra.run.smithi079.stdout:(32/137): rbd-mirror-16.0.0-7397.gb86e691b.el8. 9.0 MB/s | 3.2 MB 00:00 2020-11-20T12:00:26.182 INFO:teuthology.orchestra.run.smithi044.stdout:(9/137): ceph-mds-16.0.0-7397.gb86e691b.el8.x86 1.7 MB/s | 2.1 MB 00:01 2020-11-20T12:00:26.185 INFO:teuthology.orchestra.run.smithi079.stdout:(33/137): ceph-mgr-dashboard-16.0.0-7397.gb86e6 5.4 MB/s | 3.2 MB 00:00 2020-11-20T12:00:26.257 INFO:teuthology.orchestra.run.smithi079.stdout:(34/137): ceph-mgr-modules-core-16.0.0-7397.gb8 2.7 MB/s | 198 kB 00:00 2020-11-20T12:00:26.317 INFO:teuthology.orchestra.run.smithi079.stdout:(35/137): ceph-mgr-rook-16.0.0-7397.gb86e691b.e 450 kB/s | 27 kB 00:00 2020-11-20T12:00:26.378 INFO:teuthology.orchestra.run.smithi079.stdout:(36/137): ceph-prometheus-alerts-16.0.0-7397.gb 146 kB/s | 8.8 kB 00:00 2020-11-20T12:00:26.442 INFO:teuthology.orchestra.run.smithi079.stdout:(37/137): cephadm-16.0.0-7397.gb86e691b.el8.noa 925 kB/s | 57 kB 00:00 2020-11-20T12:00:26.450 INFO:teuthology.orchestra.run.smithi079.stdout:(38/137): automake-1.16.1-6.el8.noarch.rpm 72 MB/s | 713 kB 00:00 2020-11-20T12:00:26.465 INFO:teuthology.orchestra.run.smithi079.stdout:(39/137): bison-3.0.4-10.el8.x86_64.rpm 45 MB/s | 688 kB 00:00 2020-11-20T12:00:26.470 INFO:teuthology.orchestra.run.smithi079.stdout:(40/137): flex-2.6.1-9.el8.x86_64.rpm 72 MB/s | 320 kB 00:00 2020-11-20T12:00:26.472 INFO:teuthology.orchestra.run.smithi079.stdout:(41/137): jq-1.5-12.el8.x86_64.rpm 58 MB/s | 161 kB 00:00 2020-11-20T12:00:26.477 INFO:teuthology.orchestra.run.smithi079.stdout:(42/137): librdkafka-0.11.4-1.el8.x86_64.rpm 73 MB/s | 353 kB 00:00 2020-11-20T12:00:26.487 INFO:teuthology.orchestra.run.smithi079.stdout:(43/137): libtool-2.4.6-25.el8.x86_64.rpm 79 MB/s | 709 kB 00:00 2020-11-20T12:00:26.490 INFO:teuthology.orchestra.run.smithi079.stdout:(44/137): lttng-ust-2.8.1-11.el8.x86_64.rpm 67 MB/s | 259 kB 00:00 2020-11-20T12:00:26.493 INFO:teuthology.orchestra.run.smithi079.stdout:(45/137): oniguruma-6.8.2-1.el8.x86_64.rpm 63 MB/s | 188 kB 00:00 2020-11-20T12:00:26.532 INFO:teuthology.orchestra.run.smithi079.stdout:(46/137): ceph-mgr-diskprediction-local-16.0.0- 11 MB/s | 8.6 MB 00:00 2020-11-20T12:00:26.571 INFO:teuthology.orchestra.run.smithi079.stdout:(47/137): openblas-0.3.3-2.el8.x86_64.rpm 57 MB/s | 4.3 MB 00:00 2020-11-20T12:00:26.647 INFO:teuthology.orchestra.run.smithi079.stdout:(48/137): python3-babel-2.5.1-5.el8.noarch.rpm 61 MB/s | 4.8 MB 00:00 2020-11-20T12:00:26.652 INFO:teuthology.orchestra.run.smithi079.stdout:(49/137): python3-html5lib-0.999999999-6.el8.no 39 MB/s | 214 kB 00:00 2020-11-20T12:00:26.663 INFO:teuthology.orchestra.run.smithi079.stdout:(50/137): python3-jinja2-2.10.1-2.el8_0.noarch. 53 MB/s | 538 kB 00:00 2020-11-20T12:00:26.663 INFO:teuthology.orchestra.run.smithi044.stdout:(10/137): ceph-base-16.0.0-7397.gb86e691b.el8.x 1.7 MB/s | 6.0 MB 00:03 2020-11-20T12:00:26.666 INFO:teuthology.orchestra.run.smithi079.stdout:(51/137): python3-jsonpatch-1.21-2.el8.noarch.r 7.0 MB/s | 27 kB 00:00 2020-11-20T12:00:26.670 INFO:teuthology.orchestra.run.smithi079.stdout:(52/137): python3-jsonpointer-1.10-11.el8.noarc 5.0 MB/s | 20 kB 00:00 2020-11-20T12:00:26.702 INFO:teuthology.orchestra.run.smithi079.stdout:(53/137): python3-lxml-4.2.3-1.el8.x86_64.rpm 56 MB/s | 1.5 MB 00:00 2020-11-20T12:00:26.711 INFO:teuthology.orchestra.run.smithi079.stdout:(54/137): openblas-threads-0.3.3-2.el8.x86_64.r 24 MB/s | 4.4 MB 00:00 2020-11-20T12:00:26.712 INFO:teuthology.orchestra.run.smithi079.stdout:(55/137): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2020-11-20T12:00:26.713 INFO:teuthology.orchestra.run.smithi079.stdout:(56/137): python3-markupsafe-0.23-19.el8.x86_64 18 MB/s | 39 kB 00:00 2020-11-20T12:00:26.719 INFO:teuthology.orchestra.run.smithi079.stdout:(57/137): python3-numpy-f2py-1.14.3-9.el8.x86_6 55 MB/s | 225 kB 00:00 2020-11-20T12:00:26.720 INFO:teuthology.orchestra.run.smithi079.stdout:(58/137): python3-prettytable-0.7.2-14.el8.noar 13 MB/s | 44 kB 00:00 2020-11-20T12:00:26.726 INFO:teuthology.orchestra.run.smithi079.stdout:(59/137): python3-pyasn1-0.3.7-6.el8.noarch.rpm 24 MB/s | 126 kB 00:00 2020-11-20T12:00:26.731 INFO:teuthology.orchestra.run.smithi079.stdout:(60/137): python3-pyasn1-modules-0.3.7-6.el8.no 20 MB/s | 110 kB 00:00 2020-11-20T12:00:26.738 INFO:teuthology.orchestra.run.smithi079.stdout:(61/137): python3-pytz-2017.2-9.el8.noarch.rpm 8.4 MB/s | 54 kB 00:00 2020-11-20T12:00:26.761 INFO:teuthology.orchestra.run.smithi044.stdout:(11/137): ceph-mgr-16.0.0-7397.gb86e691b.el8.x8 2.7 MB/s | 1.5 MB 00:00 2020-11-20T12:00:26.780 INFO:teuthology.orchestra.run.smithi079.stdout:(62/137): python3-numpy-1.14.3-9.el8.x86_64.rpm 56 MB/s | 3.7 MB 00:00 2020-11-20T12:00:26.782 INFO:teuthology.orchestra.run.smithi079.stdout:(63/137): python3-webencodings-0.5.1-6.el8.noar 8.9 MB/s | 27 kB 00:00 2020-11-20T12:00:26.794 INFO:teuthology.orchestra.run.smithi079.stdout:(64/137): python3-werkzeug-0.12.2-4.el8.noarch. 40 MB/s | 457 kB 00:00 2020-11-20T12:00:26.805 INFO:teuthology.orchestra.run.smithi079.stdout:(65/137): socat-1.7.3.2-6.el8.x86_64.rpm 27 MB/s | 298 kB 00:00 2020-11-20T12:00:26.812 INFO:teuthology.orchestra.run.smithi079.stdout:(66/137): elfutils-libelf-devel-0.176-5.el8.x86 7.0 MB/s | 54 kB 00:00 2020-11-20T12:00:26.818 INFO:teuthology.orchestra.run.smithi079.stdout:(67/137): fuse-2.9.7-12.el8.x86_64.rpm 14 MB/s | 82 kB 00:00 2020-11-20T12:00:26.822 INFO:teuthology.orchestra.run.smithi079.stdout:(68/137): fuse-common-3.2.1-12.el8.x86_64.rpm 5.6 MB/s | 21 kB 00:00 2020-11-20T12:00:26.828 INFO:teuthology.orchestra.run.smithi079.stdout:(69/137): keyutils-libs-devel-1.5.10-6.el8.x86_ 11 MB/s | 48 kB 00:00 2020-11-20T12:00:26.842 INFO:teuthology.orchestra.run.smithi079.stdout:(70/137): krb5-devel-1.17-9.el8.x86_64.rpm 34 MB/s | 548 kB 00:00 2020-11-20T12:00:26.851 INFO:teuthology.orchestra.run.smithi079.stdout:(71/137): libaio-devel-0.3.112-1.el8.x86_64.rpm 2.2 MB/s | 19 kB 00:00 2020-11-20T12:00:26.857 INFO:teuthology.orchestra.run.smithi079.stdout:(72/137): libcom_err-devel-1.44.6-3.el8.x86_64. 6.8 MB/s | 38 kB 00:00 2020-11-20T12:00:26.875 INFO:teuthology.orchestra.run.smithi079.stdout:(73/137): libgfortran-8.3.1-4.5.el8.x86_64.rpm 34 MB/s | 638 kB 00:00 2020-11-20T12:00:26.886 INFO:teuthology.orchestra.run.smithi079.stdout:(74/137): libquadmath-8.3.1-4.5.el8.x86_64.rpm 16 MB/s | 169 kB 00:00 2020-11-20T12:00:26.895 INFO:teuthology.orchestra.run.smithi079.stdout:(75/137): librabbitmq-0.9.0-1.el8.x86_64.rpm 5.2 MB/s | 47 kB 00:00 2020-11-20T12:00:26.903 INFO:teuthology.orchestra.run.smithi079.stdout:(76/137): libselinux-devel-2.9-2.1.el8.x86_64.r 23 MB/s | 199 kB 00:00 2020-11-20T12:00:26.913 INFO:teuthology.orchestra.run.smithi079.stdout:(77/137): libsepol-devel-2.9-1.el8.x86_64.rpm 11 MB/s | 86 kB 00:00 2020-11-20T12:00:26.920 INFO:teuthology.orchestra.run.smithi079.stdout:(78/137): libuuid-devel-2.32.1-17.el8.x86_64.rp 10 MB/s | 95 kB 00:00 2020-11-20T12:00:26.929 INFO:teuthology.orchestra.run.smithi079.stdout:(79/137): libverto-devel-0.3.0-5.el8.x86_64.rpm 2.0 MB/s | 18 kB 00:00 2020-11-20T12:00:26.941 INFO:teuthology.orchestra.run.smithi079.stdout:(80/137): libxslt-1.1.32-3.el8.x86_64.rpm 22 MB/s | 249 kB 00:00 2020-11-20T12:00:27.056 INFO:teuthology.orchestra.run.smithi079.stdout:(81/137): python3-scipy-1.0.0-20.module_el8.1.0 46 MB/s | 15 MB 00:00 2020-11-20T12:00:27.069 INFO:teuthology.orchestra.run.smithi079.stdout:(82/137): pcre2-devel-10.32-1.el8.x86_64.rpm 44 MB/s | 605 kB 00:00 2020-11-20T12:00:27.075 INFO:teuthology.orchestra.run.smithi079.stdout:(83/137): openssl-devel-1.1.1c-2.el8_1.1.x86_64 17 MB/s | 2.3 MB 00:00 2020-11-20T12:00:27.076 INFO:teuthology.orchestra.run.smithi079.stdout:(84/137): pcre2-utf16-10.32-1.el8.x86_64.rpm 28 MB/s | 228 kB 00:00 2020-11-20T12:00:27.079 INFO:teuthology.orchestra.run.smithi079.stdout:(85/137): pcre2-utf32-10.32-1.el8.x86_64.rpm 59 MB/s | 220 kB 00:00 2020-11-20T12:00:27.080 INFO:teuthology.orchestra.run.smithi079.stdout:(86/137): python3-chardet-3.0.4-7.el8.noarch.rp 53 MB/s | 195 kB 00:00 2020-11-20T12:00:27.081 INFO:teuthology.orchestra.run.smithi079.stdout:(87/137): python3-jwt-1.6.1-2.el8.noarch.rpm 20 MB/s | 43 kB 00:00 2020-11-20T12:00:27.083 INFO:teuthology.orchestra.run.smithi079.stdout:(88/137): python3-oauthlib-2.1.0-1.el8.noarch.r 57 MB/s | 155 kB 00:00 2020-11-20T12:00:27.084 INFO:teuthology.orchestra.run.smithi079.stdout:(89/137): python3-pysocks-1.6.8-3.el8.noarch.rp 13 MB/s | 34 kB 00:00 2020-11-20T12:00:27.086 INFO:teuthology.orchestra.run.smithi079.stdout:(90/137): python3-requests-2.20.0-2.1.el8_1.noa 57 MB/s | 123 kB 00:00 2020-11-20T12:00:27.086 INFO:teuthology.orchestra.run.smithi079.stdout:(91/137): python3-requests-oauthlib-1.0.0-1.el8 19 MB/s | 43 kB 00:00 2020-11-20T12:00:27.089 INFO:teuthology.orchestra.run.smithi079.stdout:(92/137): python3-urllib3-1.24.2-2.el8.noarch.r 52 MB/s | 176 kB 00:00 2020-11-20T12:00:27.090 INFO:teuthology.orchestra.run.smithi079.stdout:(93/137): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 19 MB/s | 69 kB 00:00 2020-11-20T12:00:27.091 INFO:teuthology.orchestra.run.smithi079.stdout:(94/137): zlib-devel-1.2.11-10.el8.x86_64.rpm 39 MB/s | 56 kB 00:00 2020-11-20T12:00:27.188 INFO:teuthology.orchestra.run.smithi079.stdout:(95/137): fmt-6.2.1-1.el8.x86_64.rpm 1.2 MB/s | 116 kB 00:00 2020-11-20T12:00:27.208 INFO:teuthology.orchestra.run.smithi079.stdout:(96/137): gperftools-libs-2.7-6.el8.x86_64.rpm 2.6 MB/s | 315 kB 00:00 2020-11-20T12:00:27.227 INFO:teuthology.orchestra.run.smithi079.stdout:(97/137): leveldb-1.22-1.el8.x86_64.rpm 4.5 MB/s | 181 kB 00:00 2020-11-20T12:00:27.228 INFO:teuthology.orchestra.run.smithi079.stdout:(98/137): liboath-2.6.2-3.el8.x86_64.rpm 2.9 MB/s | 59 kB 00:00 2020-11-20T12:00:27.247 INFO:teuthology.orchestra.run.smithi079.stdout:(99/137): libunwind-1.3.1-3.el8.x86_64.rpm 3.7 MB/s | 75 kB 00:00 2020-11-20T12:00:27.248 INFO:teuthology.orchestra.run.smithi079.stdout:(100/137): python3-apipkg-1.5-6.el8.noarch.rpm 1.0 MB/s | 21 kB 00:00 2020-11-20T12:00:27.267 INFO:teuthology.orchestra.run.smithi079.stdout:(101/137): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.2 MB/s | 44 kB 00:00 2020-11-20T12:00:27.269 INFO:teuthology.orchestra.run.smithi079.stdout:(102/137): python3-beautifulsoup4-4.6.3-2.el8.1 8.5 MB/s | 185 kB 00:00 2020-11-20T12:00:27.288 INFO:teuthology.orchestra.run.smithi079.stdout:(103/137): python3-cachetools-3.1.1-4.el8.noarc 1.6 MB/s | 33 kB 00:00 2020-11-20T12:00:27.289 INFO:teuthology.orchestra.run.smithi079.stdout:(104/137): python3-certifi-2018.10.15-7.el8.noa 833 kB/s | 16 kB 00:00 2020-11-20T12:00:27.315 INFO:teuthology.orchestra.run.smithi079.stdout:(105/137): python3-cheroot-8.2.1-1.el8.noarch.r 5.2 MB/s | 145 kB 00:00 2020-11-20T12:00:27.329 INFO:teuthology.orchestra.run.smithi079.stdout:(106/137): python3-cherrypy-18.4.0-1.el8.noarch 9.5 MB/s | 384 kB 00:00 2020-11-20T12:00:27.335 INFO:teuthology.orchestra.run.smithi079.stdout:(107/137): python3-cssselect-0.9.2-10.el8.noarc 2.0 MB/s | 40 kB 00:00 2020-11-20T12:00:27.351 INFO:teuthology.orchestra.run.smithi079.stdout:(108/137): python3-execnet-1.7.1-1.el8.noarch.r 16 MB/s | 355 kB 00:00 2020-11-20T12:00:27.355 INFO:teuthology.orchestra.run.smithi079.stdout:(109/137): python3-google-auth-1.1.1-10.el8.noa 3.9 MB/s | 81 kB 00:00 2020-11-20T12:00:27.370 INFO:teuthology.orchestra.run.smithi079.stdout:(110/137): python3-jaraco-6.2-6.el8.noarch.rpm 571 kB/s | 11 kB 00:00 2020-11-20T12:00:27.375 INFO:teuthology.orchestra.run.smithi079.stdout:(111/137): python3-jaraco-functools-2.0-4.el8.n 902 kB/s | 18 kB 00:00 2020-11-20T12:00:27.396 INFO:teuthology.orchestra.run.smithi079.stdout:(112/137): python3-logutils-0.3.5-11.el8.noarch 2.3 MB/s | 49 kB 00:00 2020-11-20T12:00:27.416 INFO:teuthology.orchestra.run.smithi079.stdout:(113/137): python3-more-itertools-7.2.0-3.el8.n 2.9 MB/s | 59 kB 00:00 2020-11-20T12:00:27.432 INFO:teuthology.orchestra.run.smithi079.stdout:(114/137): python3-kubernetes-11.0.0-2.el8.noar 19 MB/s | 1.2 MB 00:00 2020-11-20T12:00:27.439 INFO:teuthology.orchestra.run.smithi079.stdout:(115/137): python3-pecan-1.3.2-9.el8.noarch.rpm 13 MB/s | 283 kB 00:00 2020-11-20T12:00:27.452 INFO:teuthology.orchestra.run.smithi079.stdout:(116/137): python3-portend-2.6-1.el8.noarch.rpm 792 kB/s | 16 kB 00:00 2020-11-20T12:00:27.458 INFO:teuthology.orchestra.run.smithi079.stdout:(117/137): python3-remoto-1.1.4-4.el8.noarch.rp 2.3 MB/s | 48 kB 00:00 2020-11-20T12:00:27.472 INFO:teuthology.orchestra.run.smithi079.stdout:(118/137): python3-repoze-lru-0.7-6.el8.noarch. 1.6 MB/s | 34 kB 00:00 2020-11-20T12:00:27.479 INFO:teuthology.orchestra.run.smithi079.stdout:(119/137): python3-routes-2.4.1-12.el8.noarch.r 9.1 MB/s | 196 kB 00:00 2020-11-20T12:00:27.492 INFO:teuthology.orchestra.run.smithi079.stdout:(120/137): python3-rsa-3.4.2-15.el8.noarch.rpm 3.6 MB/s | 73 kB 00:00 2020-11-20T12:00:27.503 INFO:teuthology.orchestra.run.smithi079.stdout:(121/137): python3-simplegeneric-0.8.1-17.el8.n 944 kB/s | 19 kB 00:00 2020-11-20T12:00:27.512 INFO:teuthology.orchestra.run.smithi079.stdout:(122/137): python3-singledispatch-3.4.0.3-18.el 1.2 MB/s | 24 kB 00:00 2020-11-20T12:00:27.519 INFO:teuthology.orchestra.run.smithi079.stdout:(123/137): python3-tempora-1.14.1-5.el8.noarch. 1.4 MB/s | 29 kB 00:00 2020-11-20T12:00:27.532 INFO:teuthology.orchestra.run.smithi079.stdout:(124/137): python3-trustme-0.6.0-4.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2020-11-20T12:00:27.541 INFO:teuthology.orchestra.run.smithi079.stdout:(125/137): python3-waitress-1.2.1-2.el8.1.noarc 11 MB/s | 241 kB 00:00 2020-11-20T12:00:27.553 INFO:teuthology.orchestra.run.smithi079.stdout:(126/137): python3-webob-1.8.5-1.el8.1.noarch.r 12 MB/s | 251 kB 00:00 2020-11-20T12:00:27.561 INFO:teuthology.orchestra.run.smithi079.stdout:(127/137): python3-websocket-client-0.56.0-5.el 3.0 MB/s | 61 kB 00:00 2020-11-20T12:00:27.573 INFO:teuthology.orchestra.run.smithi079.stdout:(128/137): python3-webtest-2.0.33-1.el8.noarch. 4.1 MB/s | 85 kB 00:00 2020-11-20T12:00:27.583 INFO:teuthology.orchestra.run.smithi079.stdout:(129/137): python3-zc-lockfile-2.0-2.el8.noarch 1.1 MB/s | 23 kB 00:00 2020-11-20T12:00:27.594 INFO:teuthology.orchestra.run.smithi079.stdout:(130/137): xmlstarlet-1.6.1-11.el8.x86_64.rpm 3.3 MB/s | 69 kB 00:00 2020-11-20T12:00:28.063 INFO:teuthology.orchestra.run.smithi079.stdout:(131/137): librbd1-16.0.0-7397.gb86e691b.el8.x8 5.1 MB/s | 2.4 MB 00:00 2020-11-20T12:00:28.090 INFO:teuthology.orchestra.run.smithi079.stdout:(132/137): NetworkManager-1.20.0-5.el8_1.x86_64 82 MB/s | 2.1 MB 00:00 2020-11-20T12:00:28.110 INFO:teuthology.orchestra.run.smithi079.stdout:(133/137): NetworkManager-libnm-1.20.0-5.el8_1. 84 MB/s | 1.7 MB 00:00 2020-11-20T12:00:28.113 INFO:teuthology.orchestra.run.smithi079.stdout:(134/137): NetworkManager-team-1.20.0-5.el8_1.x 55 MB/s | 133 kB 00:00 2020-11-20T12:00:28.117 INFO:teuthology.orchestra.run.smithi079.stdout:(135/137): NetworkManager-tui-1.20.0-5.el8_1.x8 70 MB/s | 308 kB 00:00 2020-11-20T12:00:28.217 INFO:teuthology.orchestra.run.smithi079.stdout:(136/137): librados2-16.0.0-7397.gb86e691b.el8. 5.8 MB/s | 3.7 MB 00:00 2020-11-20T12:00:28.554 INFO:teuthology.orchestra.run.smithi044.stdout:(12/137): ceph-mon-16.0.0-7397.gb86e691b.el8.x8 2.6 MB/s | 5.0 MB 00:01 2020-11-20T12:00:28.888 INFO:teuthology.orchestra.run.smithi079.stdout:(137/137): ceph-test-16.0.0-7397.gb86e691b.el8. 12 MB/s | 48 MB 00:04 2020-11-20T12:00:28.893 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2020-11-20T12:00:28.893 INFO:teuthology.orchestra.run.smithi079.stdout:Total 29 MB/s | 202 MB 00:06 2020-11-20T12:00:28.965 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2020-11-20T12:00:29.138 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2020-11-20T12:00:29.138 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2020-11-20T12:00:29.562 INFO:teuthology.orchestra.run.smithi044.stdout:(13/137): ceph-common-16.0.0-7397.gb86e691b.el8 4.0 MB/s | 23 MB 00:05 2020-11-20T12:00:29.587 INFO:teuthology.orchestra.run.smithi044.stdout:(14/137): ceph-selinux-16.0.0-7397.gb86e691b.el 1.0 MB/s | 25 kB 00:00 2020-11-20T12:00:30.531 INFO:teuthology.orchestra.run.smithi044.stdout:(15/137): ceph-osd-16.0.0-7397.gb86e691b.el8.x8 4.8 MB/s | 18 MB 00:03 2020-11-20T12:00:30.557 INFO:teuthology.orchestra.run.smithi044.stdout:(16/137): libcephfs-devel-16.0.0-7397.gb86e691b 925 kB/s | 24 kB 00:00 2020-11-20T12:00:30.608 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2020-11-20T12:00:30.609 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2020-11-20T12:00:30.651 INFO:teuthology.orchestra.run.smithi044.stdout:(17/137): libcephfs2-16.0.0-7397.gb86e691b.el8. 5.8 MB/s | 562 kB 00:00 2020-11-20T12:00:30.676 INFO:teuthology.orchestra.run.smithi044.stdout:(18/137): librados-devel-16.0.0-7397.gb86e691b. 5.2 MB/s | 131 kB 00:00 2020-11-20T12:00:30.755 INFO:teuthology.orchestra.run.smithi044.stdout:(19/137): ceph-radosgw-16.0.0-7397.gb86e691b.el 5.4 MB/s | 12 MB 00:02 2020-11-20T12:00:30.827 INFO:teuthology.orchestra.run.smithi044.stdout:(20/137): libradosstriper1-16.0.0-7397.gb86e691 3.1 MB/s | 486 kB 00:00 2020-11-20T12:00:30.852 INFO:teuthology.orchestra.run.smithi044.stdout:(21/137): python3-ceph-argparse-16.0.0-7397.gb8 1.6 MB/s | 40 kB 00:00 2020-11-20T12:00:30.876 INFO:teuthology.orchestra.run.smithi044.stdout:(22/137): python3-ceph-common-16.0.0-7397.gb86e 3.0 MB/s | 73 kB 00:00 2020-11-20T12:00:30.907 INFO:teuthology.orchestra.run.smithi044.stdout:(23/137): python3-cephfs-16.0.0-7397.gb86e691b. 7.7 MB/s | 204 kB 00:00 2020-11-20T12:00:30.951 INFO:teuthology.orchestra.run.smithi044.stdout:(24/137): python3-rados-16.0.0-7397.gb86e691b.e 7.5 MB/s | 375 kB 00:00 2020-11-20T12:00:31.000 INFO:teuthology.orchestra.run.smithi044.stdout:(25/137): python3-rbd-16.0.0-7397.gb86e691b.el8 7.1 MB/s | 353 kB 00:00 2020-11-20T12:00:31.025 INFO:teuthology.orchestra.run.smithi044.stdout:(26/137): python3-rgw-16.0.0-7397.gb86e691b.el8 4.5 MB/s | 113 kB 00:00 2020-11-20T12:00:31.049 INFO:teuthology.orchestra.run.smithi044.stdout:(27/137): rbd-fuse-16.0.0-7397.gb86e691b.el8.x8 3.5 MB/s | 88 kB 00:00 2020-11-20T12:00:31.290 INFO:teuthology.orchestra.run.smithi044.stdout:(28/137): librgw2-16.0.0-7397.gb86e691b.el8.x86 6.9 MB/s | 3.7 MB 00:00 2020-11-20T12:00:31.323 INFO:teuthology.orchestra.run.smithi044.stdout:(29/137): rbd-nbd-16.0.0-7397.gb86e691b.el8.x86 5.9 MB/s | 171 kB 00:00 2020-11-20T12:00:31.343 INFO:teuthology.orchestra.run.smithi044.stdout:(30/137): ceph-grafana-dashboards-16.0.0-7397.g 1.0 MB/s | 24 kB 00:00 2020-11-20T12:00:31.366 INFO:teuthology.orchestra.run.smithi044.stdout:(31/137): ceph-mgr-cephadm-16.0.0-7397.gb86e691 2.7 MB/s | 66 kB 00:00 2020-11-20T12:00:31.436 INFO:teuthology.orchestra.run.smithi044.stdout:(32/137): rbd-mirror-16.0.0-7397.gb86e691b.el8. 8.3 MB/s | 3.2 MB 00:00 2020-11-20T12:00:31.765 INFO:teuthology.orchestra.run.smithi044.stdout:(33/137): ceph-mgr-dashboard-16.0.0-7397.gb86e6 7.9 MB/s | 3.2 MB 00:00 2020-11-20T12:00:31.898 INFO:teuthology.orchestra.run.smithi044.stdout:(34/137): ceph-mgr-modules-core-16.0.0-7397.gb8 7.2 MB/s | 198 kB 00:00 2020-11-20T12:00:31.899 INFO:teuthology.orchestra.run.smithi044.stdout:(35/137): ceph-mgr-rook-16.0.0-7397.gb86e691b.e 1.1 MB/s | 27 kB 00:00 2020-11-20T12:00:31.899 INFO:teuthology.orchestra.run.smithi044.stdout:(36/137): ceph-prometheus-alerts-16.0.0-7397.gb 386 kB/s | 8.8 kB 00:00 2020-11-20T12:00:31.899 INFO:teuthology.orchestra.run.smithi044.stdout:(37/137): cephadm-16.0.0-7397.gb86e691b.el8.noa 2.4 MB/s | 57 kB 00:00 2020-11-20T12:00:31.903 INFO:teuthology.orchestra.run.smithi044.stdout:(38/137): automake-1.16.1-6.el8.noarch.rpm 17 MB/s | 713 kB 00:00 2020-11-20T12:00:31.913 INFO:teuthology.orchestra.run.smithi044.stdout:(39/137): bison-3.0.4-10.el8.x86_64.rpm 73 MB/s | 688 kB 00:00 2020-11-20T12:00:31.917 INFO:teuthology.orchestra.run.smithi044.stdout:(40/137): flex-2.6.1-9.el8.x86_64.rpm 69 MB/s | 320 kB 00:00 2020-11-20T12:00:31.922 INFO:teuthology.orchestra.run.smithi044.stdout:(41/137): jq-1.5-12.el8.x86_64.rpm 55 MB/s | 161 kB 00:00 2020-11-20T12:00:31.925 INFO:teuthology.orchestra.run.smithi044.stdout:(42/137): librdkafka-0.11.4-1.el8.x86_64.rpm 71 MB/s | 353 kB 00:00 2020-11-20T12:00:31.934 INFO:teuthology.orchestra.run.smithi044.stdout:(43/137): libtool-2.4.6-25.el8.x86_64.rpm 77 MB/s | 709 kB 00:00 2020-11-20T12:00:31.938 INFO:teuthology.orchestra.run.smithi044.stdout:(44/137): lttng-ust-2.8.1-11.el8.x86_64.rpm 65 MB/s | 259 kB 00:00 2020-11-20T12:00:31.941 INFO:teuthology.orchestra.run.smithi044.stdout:(45/137): oniguruma-6.8.2-1.el8.x86_64.rpm 62 MB/s | 188 kB 00:00 2020-11-20T12:00:31.993 INFO:teuthology.orchestra.run.smithi044.stdout:(46/137): openblas-0.3.3-2.el8.x86_64.rpm 84 MB/s | 4.3 MB 00:00 2020-11-20T12:00:32.046 INFO:teuthology.orchestra.run.smithi044.stdout:(47/137): openblas-threads-0.3.3-2.el8.x86_64.r 84 MB/s | 4.4 MB 00:00 2020-11-20T12:00:32.103 INFO:teuthology.orchestra.run.smithi044.stdout:(48/137): python3-babel-2.5.1-5.el8.noarch.rpm 84 MB/s | 4.8 MB 00:00 2020-11-20T12:00:32.107 INFO:teuthology.orchestra.run.smithi044.stdout:(49/137): python3-html5lib-0.999999999-6.el8.no 61 MB/s | 214 kB 00:00 2020-11-20T12:00:32.114 INFO:teuthology.orchestra.run.smithi044.stdout:(50/137): python3-jinja2-2.10.1-2.el8_0.noarch. 73 MB/s | 538 kB 00:00 2020-11-20T12:00:32.115 INFO:teuthology.orchestra.run.smithi044.stdout:(51/137): python3-jsonpatch-1.21-2.el8.noarch.r 22 MB/s | 27 kB 00:00 2020-11-20T12:00:32.116 INFO:teuthology.orchestra.run.smithi044.stdout:(52/137): python3-jsonpointer-1.10-11.el8.noarc 16 MB/s | 20 kB 00:00 2020-11-20T12:00:32.135 INFO:teuthology.orchestra.run.smithi044.stdout:(53/137): python3-lxml-4.2.3-1.el8.x86_64.rpm 81 MB/s | 1.5 MB 00:00 2020-11-20T12:00:32.138 INFO:teuthology.orchestra.run.smithi044.stdout:(54/137): python3-mako-1.0.6-13.el8.noarch.rpm 56 MB/s | 157 kB 00:00 2020-11-20T12:00:32.139 INFO:teuthology.orchestra.run.smithi044.stdout:(55/137): python3-markupsafe-0.23-19.el8.x86_64 25 MB/s | 39 kB 00:00 2020-11-20T12:00:32.185 INFO:teuthology.orchestra.run.smithi044.stdout:(56/137): python3-numpy-1.14.3-9.el8.x86_64.rpm 83 MB/s | 3.7 MB 00:00 2020-11-20T12:00:32.189 INFO:teuthology.orchestra.run.smithi044.stdout:(57/137): python3-numpy-f2py-1.14.3-9.el8.x86_6 61 MB/s | 225 kB 00:00 2020-11-20T12:00:32.192 INFO:teuthology.orchestra.run.smithi044.stdout:(58/137): python3-prettytable-0.7.2-14.el8.noar 12 MB/s | 44 kB 00:00 2020-11-20T12:00:32.195 INFO:teuthology.orchestra.run.smithi044.stdout:(59/137): python3-pyasn1-0.3.7-6.el8.noarch.rpm 53 MB/s | 126 kB 00:00 2020-11-20T12:00:32.198 INFO:teuthology.orchestra.run.smithi044.stdout:(60/137): python3-pyasn1-modules-0.3.7-6.el8.no 48 MB/s | 110 kB 00:00 2020-11-20T12:00:32.199 INFO:teuthology.orchestra.run.smithi044.stdout:(61/137): python3-pytz-2017.2-9.el8.noarch.rpm 32 MB/s | 54 kB 00:00 2020-11-20T12:00:32.427 INFO:teuthology.orchestra.run.smithi044.stdout:(62/137): python3-scipy-1.0.0-20.module_el8.1.0 64 MB/s | 15 MB 00:00 2020-11-20T12:00:32.429 INFO:teuthology.orchestra.run.smithi044.stdout:(63/137): python3-webencodings-0.5.1-6.el8.noar 18 MB/s | 27 kB 00:00 2020-11-20T12:00:32.454 INFO:teuthology.orchestra.run.smithi044.stdout:(64/137): ceph-mgr-diskprediction-local-16.0.0- 8.4 MB/s | 8.6 MB 00:01 2020-11-20T12:00:32.463 INFO:teuthology.orchestra.run.smithi044.stdout:(65/137): python3-werkzeug-0.12.2-4.el8.noarch. 13 MB/s | 457 kB 00:00 2020-11-20T12:00:32.465 INFO:teuthology.orchestra.run.smithi044.stdout:(66/137): elfutils-libelf-devel-0.176-5.el8.x86 35 MB/s | 54 kB 00:00 2020-11-20T12:00:32.474 INFO:teuthology.orchestra.run.smithi044.stdout:(67/137): fuse-2.9.7-12.el8.x86_64.rpm 8.5 MB/s | 82 kB 00:00 2020-11-20T12:00:32.476 INFO:teuthology.orchestra.run.smithi044.stdout:(68/137): fuse-common-3.2.1-12.el8.x86_64.rpm 11 MB/s | 21 kB 00:00 2020-11-20T12:00:32.478 INFO:teuthology.orchestra.run.smithi044.stdout:(69/137): keyutils-libs-devel-1.5.10-6.el8.x86_ 36 MB/s | 48 kB 00:00 2020-11-20T12:00:32.485 INFO:teuthology.orchestra.run.smithi044.stdout:(70/137): krb5-devel-1.17-9.el8.x86_64.rpm 77 MB/s | 548 kB 00:00 2020-11-20T12:00:32.486 INFO:teuthology.orchestra.run.smithi044.stdout:(71/137): libaio-devel-0.3.112-1.el8.x86_64.rpm 13 MB/s | 19 kB 00:00 2020-11-20T12:00:32.488 INFO:teuthology.orchestra.run.smithi044.stdout:(72/137): libcom_err-devel-1.44.6-3.el8.x86_64. 31 MB/s | 38 kB 00:00 2020-11-20T12:00:32.497 INFO:teuthology.orchestra.run.smithi044.stdout:(73/137): libgfortran-8.3.1-4.5.el8.x86_64.rpm 72 MB/s | 638 kB 00:00 2020-11-20T12:00:32.500 INFO:teuthology.orchestra.run.smithi044.stdout:(74/137): libquadmath-8.3.1-4.5.el8.x86_64.rpm 48 MB/s | 169 kB 00:00 2020-11-20T12:00:32.502 INFO:teuthology.orchestra.run.smithi044.stdout:(75/137): socat-1.7.3.2-6.el8.x86_64.rpm 6.1 MB/s | 298 kB 00:00 2020-11-20T12:00:32.503 INFO:teuthology.orchestra.run.smithi044.stdout:(76/137): librabbitmq-0.9.0-1.el8.x86_64.rpm 19 MB/s | 47 kB 00:00 2020-11-20T12:00:32.505 INFO:teuthology.orchestra.run.smithi044.stdout:(77/137): libsepol-devel-2.9-1.el8.x86_64.rpm 39 MB/s | 86 kB 00:00 2020-11-20T12:00:32.510 INFO:teuthology.orchestra.run.smithi044.stdout:(78/137): libselinux-devel-2.9-2.1.el8.x86_64.r 37 MB/s | 199 kB 00:00 2020-11-20T12:00:32.512 INFO:teuthology.orchestra.run.smithi044.stdout:(79/137): libuuid-devel-2.32.1-17.el8.x86_64.rp 31 MB/s | 95 kB 00:00 2020-11-20T12:00:32.512 INFO:teuthology.orchestra.run.smithi044.stdout:(80/137): libverto-devel-0.3.0-5.el8.x86_64.rpm 16 MB/s | 18 kB 00:00 2020-11-20T12:00:32.513 INFO:teuthology.orchestra.run.smithi044.stdout:(81/137): libxslt-1.1.32-3.el8.x86_64.rpm 62 MB/s | 249 kB 00:00 2020-11-20T12:00:32.522 INFO:teuthology.orchestra.run.smithi044.stdout:(82/137): pcre2-devel-10.32-1.el8.x86_64.rpm 69 MB/s | 605 kB 00:00 2020-11-20T12:00:32.526 INFO:teuthology.orchestra.run.smithi044.stdout:(83/137): pcre2-utf16-10.32-1.el8.x86_64.rpm 53 MB/s | 228 kB 00:00 2020-11-20T12:00:32.530 INFO:teuthology.orchestra.run.smithi044.stdout:(84/137): pcre2-utf32-10.32-1.el8.x86_64.rpm 56 MB/s | 220 kB 00:00 2020-11-20T12:00:32.534 INFO:teuthology.orchestra.run.smithi044.stdout:(85/137): python3-chardet-3.0.4-7.el8.noarch.rp 54 MB/s | 195 kB 00:00 2020-11-20T12:00:32.535 INFO:teuthology.orchestra.run.smithi044.stdout:(86/137): python3-jwt-1.6.1-2.el8.noarch.rpm 23 MB/s | 43 kB 00:00 2020-11-20T12:00:32.539 INFO:teuthology.orchestra.run.smithi044.stdout:(87/137): python3-oauthlib-2.1.0-1.el8.noarch.r 47 MB/s | 155 kB 00:00 2020-11-20T12:00:32.541 INFO:teuthology.orchestra.run.smithi044.stdout:(88/137): python3-pysocks-1.6.8-3.el8.noarch.rp 20 MB/s | 34 kB 00:00 2020-11-20T12:00:32.544 INFO:teuthology.orchestra.run.smithi044.stdout:(89/137): python3-requests-2.20.0-2.1.el8_1.noa 42 MB/s | 123 kB 00:00 2020-11-20T12:00:32.546 INFO:teuthology.orchestra.run.smithi044.stdout:(90/137): python3-requests-oauthlib-1.0.0-1.el8 23 MB/s | 43 kB 00:00 2020-11-20T12:00:32.550 INFO:teuthology.orchestra.run.smithi044.stdout:(91/137): python3-urllib3-1.24.2-2.el8.noarch.r 45 MB/s | 176 kB 00:00 2020-11-20T12:00:32.553 INFO:teuthology.orchestra.run.smithi044.stdout:(92/137): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 27 MB/s | 69 kB 00:00 2020-11-20T12:00:32.555 INFO:teuthology.orchestra.run.smithi044.stdout:(93/137): zlib-devel-1.2.11-10.el8.x86_64.rpm 25 MB/s | 56 kB 00:00 2020-11-20T12:00:32.563 INFO:teuthology.orchestra.run.smithi044.stdout:(94/137): openssl-devel-1.1.1c-2.el8_1.1.x86_64 43 MB/s | 2.3 MB 00:00 2020-11-20T12:00:32.636 INFO:teuthology.orchestra.run.smithi044.stdout:(95/137): fmt-6.2.1-1.el8.x86_64.rpm 1.4 MB/s | 116 kB 00:00 2020-11-20T12:00:32.669 INFO:teuthology.orchestra.run.smithi044.stdout:(96/137): leveldb-1.22-1.el8.x86_64.rpm 5.4 MB/s | 181 kB 00:00 2020-11-20T12:00:32.679 INFO:teuthology.orchestra.run.smithi044.stdout:(97/137): gperftools-libs-2.7-6.el8.x86_64.rpm 2.6 MB/s | 315 kB 00:00 2020-11-20T12:00:32.686 INFO:teuthology.orchestra.run.smithi044.stdout:(98/137): liboath-2.6.2-3.el8.x86_64.rpm 3.5 MB/s | 59 kB 00:00 2020-11-20T12:00:32.701 INFO:teuthology.orchestra.run.smithi044.stdout:(99/137): libunwind-1.3.1-3.el8.x86_64.rpm 3.5 MB/s | 75 kB 00:00 2020-11-20T12:00:32.703 INFO:teuthology.orchestra.run.smithi044.stdout:(100/137): python3-apipkg-1.5-6.el8.noarch.rpm 1.2 MB/s | 21 kB 00:00 2020-11-20T12:00:32.721 INFO:teuthology.orchestra.run.smithi044.stdout:(101/137): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.1 MB/s | 44 kB 00:00 2020-11-20T12:00:32.722 INFO:teuthology.orchestra.run.smithi044.stdout:(102/137): python3-beautifulsoup4-4.6.3-2.el8.1 10 MB/s | 185 kB 00:00 2020-11-20T12:00:32.739 INFO:teuthology.orchestra.run.smithi044.stdout:(103/137): python3-certifi-2018.10.15-7.el8.noa 983 kB/s | 16 kB 00:00 2020-11-20T12:00:32.741 INFO:teuthology.orchestra.run.smithi044.stdout:(104/137): python3-cachetools-3.1.1-4.el8.noarc 1.6 MB/s | 33 kB 00:00 2020-11-20T12:00:32.756 INFO:teuthology.orchestra.run.smithi044.stdout:(105/137): python3-cheroot-8.2.1-1.el8.noarch.r 8.2 MB/s | 145 kB 00:00 2020-11-20T12:00:32.773 INFO:teuthology.orchestra.run.smithi044.stdout:(106/137): python3-cssselect-0.9.2-10.el8.noarc 2.4 MB/s | 40 kB 00:00 2020-11-20T12:00:32.781 INFO:teuthology.orchestra.run.smithi044.stdout:(107/137): python3-cherrypy-18.4.0-1.el8.noarch 9.5 MB/s | 384 kB 00:00 2020-11-20T12:00:32.792 INFO:teuthology.orchestra.run.smithi044.stdout:(108/137): python3-execnet-1.7.1-1.el8.noarch.r 18 MB/s | 355 kB 00:00 2020-11-20T12:00:32.801 INFO:teuthology.orchestra.run.smithi044.stdout:(109/137): python3-google-auth-1.1.1-10.el8.noa 3.9 MB/s | 81 kB 00:00 2020-11-20T12:00:32.810 INFO:teuthology.orchestra.run.smithi044.stdout:(110/137): python3-jaraco-6.2-6.el8.noarch.rpm 681 kB/s | 11 kB 00:00 2020-11-20T12:00:32.821 INFO:teuthology.orchestra.run.smithi044.stdout:(111/137): python3-jaraco-functools-2.0-4.el8.n 886 kB/s | 18 kB 00:00 2020-11-20T12:00:32.841 INFO:teuthology.orchestra.run.smithi044.stdout:(112/137): python3-logutils-0.3.5-11.el8.noarch 2.4 MB/s | 49 kB 00:00 2020-11-20T12:00:32.862 INFO:teuthology.orchestra.run.smithi044.stdout:(113/137): python3-kubernetes-11.0.0-2.el8.noar 22 MB/s | 1.2 MB 00:00 2020-11-20T12:00:32.863 INFO:teuthology.orchestra.run.smithi044.stdout:(114/137): python3-more-itertools-7.2.0-3.el8.n 2.9 MB/s | 59 kB 00:00 2020-11-20T12:00:32.880 INFO:teuthology.orchestra.run.smithi044.stdout:(115/137): python3-pecan-1.3.2-9.el8.noarch.rpm 15 MB/s | 283 kB 00:00 2020-11-20T12:00:32.883 INFO:teuthology.orchestra.run.smithi044.stdout:(116/137): python3-portend-2.6-1.el8.noarch.rpm 799 kB/s | 16 kB 00:00 2020-11-20T12:00:32.896 INFO:teuthology.orchestra.run.smithi044.stdout:(117/137): python3-remoto-1.1.4-4.el8.noarch.rp 2.8 MB/s | 48 kB 00:00 2020-11-20T12:00:32.901 INFO:teuthology.orchestra.run.smithi044.stdout:(118/137): python3-repoze-lru-0.7-6.el8.noarch. 1.7 MB/s | 34 kB 00:00 2020-11-20T12:00:32.914 INFO:teuthology.orchestra.run.smithi044.stdout:(119/137): python3-routes-2.4.1-12.el8.noarch.r 11 MB/s | 196 kB 00:00 2020-11-20T12:00:32.922 INFO:teuthology.orchestra.run.smithi044.stdout:(120/137): python3-rsa-3.4.2-15.el8.noarch.rpm 3.6 MB/s | 73 kB 00:00 2020-11-20T12:00:32.930 INFO:teuthology.orchestra.run.smithi044.stdout:(121/137): python3-simplegeneric-0.8.1-17.el8.n 1.1 MB/s | 19 kB 00:00 2020-11-20T12:00:32.940 INFO:teuthology.orchestra.run.smithi044.stdout:(122/137): python3-singledispatch-3.4.0.3-18.el 1.2 MB/s | 24 kB 00:00 2020-11-20T12:00:32.946 INFO:teuthology.orchestra.run.smithi044.stdout:(123/137): python3-tempora-1.14.1-5.el8.noarch. 1.7 MB/s | 29 kB 00:00 2020-11-20T12:00:32.960 INFO:teuthology.orchestra.run.smithi044.stdout:(124/137): python3-trustme-0.6.0-4.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2020-11-20T12:00:32.965 INFO:teuthology.orchestra.run.smithi044.stdout:(125/137): python3-waitress-1.2.1-2.el8.1.noarc 13 MB/s | 241 kB 00:00 2020-11-20T12:00:32.982 INFO:teuthology.orchestra.run.smithi044.stdout:(126/137): python3-webob-1.8.5-1.el8.1.noarch.r 11 MB/s | 251 kB 00:00 2020-11-20T12:00:32.983 INFO:teuthology.orchestra.run.smithi044.stdout:(127/137): python3-websocket-client-0.56.0-5.el 3.4 MB/s | 61 kB 00:00 2020-11-20T12:00:32.999 INFO:teuthology.orchestra.run.smithi044.stdout:(128/137): python3-webtest-2.0.33-1.el8.noarch. 4.9 MB/s | 85 kB 00:00 2020-11-20T12:00:33.002 INFO:teuthology.orchestra.run.smithi044.stdout:(129/137): python3-zc-lockfile-2.0-2.el8.noarch 1.1 MB/s | 23 kB 00:00 2020-11-20T12:00:33.016 INFO:teuthology.orchestra.run.smithi044.stdout:(130/137): xmlstarlet-1.6.1-11.el8.x86_64.rpm 4.0 MB/s | 69 kB 00:00 2020-11-20T12:00:33.316 INFO:teuthology.orchestra.run.smithi044.stdout:(131/137): librbd1-16.0.0-7397.gb86e691b.el8.x8 8.1 MB/s | 2.4 MB 00:00 2020-11-20T12:00:33.341 INFO:teuthology.orchestra.run.smithi044.stdout:(132/137): NetworkManager-1.20.0-5.el8_1.x86_64 82 MB/s | 2.1 MB 00:00 2020-11-20T12:00:33.362 INFO:teuthology.orchestra.run.smithi044.stdout:(133/137): NetworkManager-libnm-1.20.0-5.el8_1. 82 MB/s | 1.7 MB 00:00 2020-11-20T12:00:33.366 INFO:teuthology.orchestra.run.smithi044.stdout:(134/137): NetworkManager-team-1.20.0-5.el8_1.x 52 MB/s | 133 kB 00:00 2020-11-20T12:00:33.371 INFO:teuthology.orchestra.run.smithi044.stdout:(135/137): NetworkManager-tui-1.20.0-5.el8_1.x8 69 MB/s | 308 kB 00:00 2020-11-20T12:00:33.536 INFO:teuthology.orchestra.run.smithi044.stdout:(136/137): librados2-16.0.0-7397.gb86e691b.el8. 7.0 MB/s | 3.7 MB 00:00 2020-11-20T12:00:34.243 INFO:teuthology.orchestra.run.smithi044.stdout:(137/137): ceph-test-16.0.0-7397.gb86e691b.el8. 10 MB/s | 48 MB 00:04 2020-11-20T12:00:34.248 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2020-11-20T12:00:34.248 INFO:teuthology.orchestra.run.smithi044.stdout:Total 17 MB/s | 202 MB 00:11 2020-11-20T12:00:34.326 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2020-11-20T12:00:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2020-11-20T12:00:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2020-11-20T12:00:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2020-11-20T12:00:35.875 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/1 2020-11-20T12:00:36.082 INFO:teuthology.orchestra.run.smithi079.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-11-20T12:00:36.093 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2020-11-20T12:00:36.094 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2020-11-20T12:00:36.136 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-11-20T12:00:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/148 2020-11-20T12:00:36.438 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/148 2020-11-20T12:00:36.662 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/148 2020-11-20T12:00:36.813 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/148 2020-11-20T12:00:36.871 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/148 2020-11-20T12:00:37.083 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/148 2020-11-20T12:00:37.247 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-11-20T12:00:37.651 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-11-20T12:00:38.708 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 8/148 2020-11-20T12:00:38.708 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 8/148 2020-11-20T12:00:38.708 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 9/148 2020-11-20T12:00:38.709 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 9/148 2020-11-20T12:00:38.709 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 10/148 2020-11-20T12:00:38.709 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 10/148 2020-11-20T12:00:39.166 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:16.0.0-7397.gb86e691b.el8.x86_64 11/148 2020-11-20T12:00:39.371 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 12/148 2020-11-20T12:00:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 12/148 2020-11-20T12:00:39.784 INFO:teuthology.orchestra.run.smithi079.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/148 2020-11-20T12:00:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/148 2020-11-20T12:00:40.134 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/148 2020-11-20T12:00:40.364 INFO:teuthology.orchestra.run.smithi079.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/148 2020-11-20T12:00:40.513 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86 17/148 2020-11-20T12:00:40.853 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86 17/148 2020-11-20T12:00:41.204 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-11-20T12:00:41.254 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-11-20T12:00:41.489 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2020-11-20T12:00:41.555 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-11-20T12:00:41.704 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/1 2020-11-20T12:00:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-11-20T12:00:41.888 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-11-20T12:00:41.942 INFO:teuthology.orchestra.run.smithi044.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-11-20T12:00:42.019 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/148 2020-11-20T12:00:42.126 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/148 2020-11-20T12:00:42.191 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/148 2020-11-20T12:00:42.280 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 21/148 2020-11-20T12:00:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/148 2020-11-20T12:00:42.470 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 22/148 2020-11-20T12:00:42.593 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/148 2020-11-20T12:00:42.693 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 23/148 2020-11-20T12:00:42.750 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/148 2020-11-20T12:00:42.821 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/148 2020-11-20T12:00:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-11-20T12:00:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-11-20T12:00:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/148 2020-11-20T12:00:43.187 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-11-20T12:00:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-11-20T12:00:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/148 2020-11-20T12:00:43.710 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-11-20T12:00:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.3-2.el8.x86_64 26/148 2020-11-20T12:00:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 8/148 2020-11-20T12:00:44.180 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 8/148 2020-11-20T12:00:44.400 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 26/148 2020-11-20T12:00:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 9/148 2020-11-20T12:00:44.540 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 9/148 2020-11-20T12:00:44.545 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-11-20T12:00:44.708 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 10/148 2020-11-20T12:00:44.808 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 10/148 2020-11-20T12:00:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-11-20T12:00:45.342 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:16.0.0-7397.gb86e691b.el8.x86_64 11/148 2020-11-20T12:00:45.516 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 12/148 2020-11-20T12:00:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 28/148 2020-11-20T12:00:45.753 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 12/148 2020-11-20T12:00:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/148 2020-11-20T12:00:45.965 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 30/148 2020-11-20T12:00:45.996 INFO:teuthology.orchestra.run.smithi044.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/148 2020-11-20T12:00:46.181 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/148 2020-11-20T12:00:46.195 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/148 2020-11-20T12:00:46.391 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:16.0.0-7397.gb86e691b.el8. 32/148 2020-11-20T12:00:46.401 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/148 2020-11-20T12:00:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:16.0.0-7397.gb86e691b.el 33/148 2020-11-20T12:00:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/148 2020-11-20T12:00:46.764 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:16.0.0-7397.gb86e691b.el8.x86_6 34/148 2020-11-20T12:00:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86 17/148 2020-11-20T12:00:47.185 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86 17/148 2020-11-20T12:00:47.474 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-11-20T12:00:47.527 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 18/148 2020-11-20T12:00:47.688 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 35/148 2020-11-20T12:00:47.858 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-11-20T12:00:47.993 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 36/148 2020-11-20T12:00:48.083 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-11-20T12:00:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.20.0-5.el8_1.x86_64 19/148 2020-11-20T12:00:48.283 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 37/148 2020-11-20T12:00:48.447 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/148 2020-11-20T12:00:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 38/148 2020-11-20T12:00:48.585 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 21/148 2020-11-20T12:00:48.783 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 22/148 2020-11-20T12:00:48.947 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 23/148 2020-11-20T12:00:49.116 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-11-20T12:00:49.260 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 24/148 2020-11-20T12:00:49.407 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-11-20T12:00:49.957 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 25/148 2020-11-20T12:00:50.132 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.3-2.el8.x86_64 26/148 2020-11-20T12:00:50.462 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 38/148 2020-11-20T12:00:50.690 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 26/148 2020-11-20T12:00:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 39/148 2020-11-20T12:00:50.815 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-11-20T12:00:51.015 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 40/148 2020-11-20T12:00:51.254 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 41/148 2020-11-20T12:00:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 42/148 2020-11-20T12:00:51.581 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 27/148 2020-11-20T12:00:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 43/148 2020-11-20T12:00:51.900 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:16.0.0-7397.gb86e691b.el8.x86_64 44/148 2020-11-20T12:00:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 28/148 2020-11-20T12:00:52.081 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:16.0.0-7397.gb86e691b.el8.x86_64 45/148 2020-11-20T12:00:52.302 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:16.0.0-7397.gb86e691b.el8.x86_6 46/148 2020-11-20T12:00:52.459 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 47/148 2020-11-20T12:00:52.540 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/148 2020-11-20T12:00:52.687 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 48/148 2020-11-20T12:00:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 30/148 2020-11-20T12:00:52.914 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 49/148 2020-11-20T12:00:52.977 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/148 2020-11-20T12:00:53.167 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 50/148 2020-11-20T12:00:53.187 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:16.0.0-7397.gb86e691b.el8. 32/148 2020-11-20T12:00:53.316 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/148 2020-11-20T12:00:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:16.0.0-7397.gb86e691b.el 33/148 2020-11-20T12:00:53.472 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/148 2020-11-20T12:00:53.578 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:16.0.0-7397.gb86e691b.el8.x86_6 34/148 2020-11-20T12:00:53.662 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/148 2020-11-20T12:00:53.878 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/148 2020-11-20T12:00:54.112 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/148 2020-11-20T12:00:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 35/148 2020-11-20T12:00:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 56/148 2020-11-20T12:00:54.921 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 36/148 2020-11-20T12:00:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 57/148 2020-11-20T12:00:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 37/148 2020-11-20T12:00:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 58/148 2020-11-20T12:00:55.440 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 38/148 2020-11-20T12:00:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 59/148 2020-11-20T12:00:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/148 2020-11-20T12:00:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/148 2020-11-20T12:00:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 62/148 2020-11-20T12:00:56.224 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 63/148 2020-11-20T12:00:56.423 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 64/148 2020-11-20T12:00:56.641 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 65/148 2020-11-20T12:00:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 66/148 2020-11-20T12:00:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/148 2020-11-20T12:00:57.367 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 68/148 2020-11-20T12:00:57.515 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 38/148 2020-11-20T12:00:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 69/148 2020-11-20T12:00:57.819 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 70/148 2020-11-20T12:00:57.842 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 39/148 2020-11-20T12:00:57.989 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/148 2020-11-20T12:00:58.066 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 40/148 2020-11-20T12:00:58.158 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 72/148 2020-11-20T12:00:58.303 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 41/148 2020-11-20T12:00:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 73/148 2020-11-20T12:00:58.564 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 42/148 2020-11-20T12:00:58.628 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 74/148 2020-11-20T12:00:58.768 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 43/148 2020-11-20T12:00:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 75/148 2020-11-20T12:00:58.983 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:16.0.0-7397.gb86e691b.el8.x86_64 44/148 2020-11-20T12:00:59.072 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 76/148 2020-11-20T12:00:59.191 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:16.0.0-7397.gb86e691b.el8.x86_64 45/148 2020-11-20T12:00:59.360 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:16.0.0-7397.gb86e691b.el8.x86_6 46/148 2020-11-20T12:00:59.548 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 47/148 2020-11-20T12:00:59.764 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 48/148 2020-11-20T12:01:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/148 2020-11-20T12:01:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 78/148 2020-11-20T12:01:00.635 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 49/148 2020-11-20T12:01:00.636 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 50/148 2020-11-20T12:01:00.636 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/148 2020-11-20T12:01:00.645 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/148 2020-11-20T12:01:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 79/148 2020-11-20T12:01:00.793 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/148 2020-11-20T12:01:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 80/148 2020-11-20T12:01:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/148 2020-11-20T12:01:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 81/148 2020-11-20T12:01:01.212 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/148 2020-11-20T12:01:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 82/148 2020-11-20T12:01:01.393 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 56/148 2020-11-20T12:01:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 83/148 2020-11-20T12:01:01.529 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 57/148 2020-11-20T12:01:01.758 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 58/148 2020-11-20T12:01:01.771 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 84/148 2020-11-20T12:01:01.977 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 59/148 2020-11-20T12:01:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 85/148 2020-11-20T12:01:02.170 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 86/148 2020-11-20T12:01:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/148 2020-11-20T12:01:02.401 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 87/148 2020-11-20T12:01:02.468 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/148 2020-11-20T12:01:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 88/148 2020-11-20T12:01:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 89/148 2020-11-20T12:01:02.636 INFO:teuthology.orchestra.run.smithi079.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-20T12:01:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:01:02.684 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 62/148 2020-11-20T12:01:02.864 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 63/148 2020-11-20T12:01:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.7-12.el8.x86_64 90/148 2020-11-20T12:01:03.073 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 64/148 2020-11-20T12:01:03.217 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 91/148 2020-11-20T12:01:03.268 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 65/148 2020-11-20T12:01:03.469 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 92/148 2020-11-20T12:01:03.517 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 66/148 2020-11-20T12:01:03.747 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/148 2020-11-20T12:01:03.826 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 93/148 2020-11-20T12:01:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 94/148 2020-11-20T12:01:04.539 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 68/148 2020-11-20T12:01:04.539 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 69/148 2020-11-20T12:01:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 95/148 2020-11-20T12:01:04.628 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 70/148 2020-11-20T12:01:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/148 2020-11-20T12:01:04.829 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/148 2020-11-20T12:01:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 97/148 2020-11-20T12:01:05.031 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 72/148 2020-11-20T12:01:05.256 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 73/148 2020-11-20T12:01:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 74/148 2020-11-20T12:01:05.756 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 75/148 2020-11-20T12:01:05.930 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 76/148 2020-11-20T12:01:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/148 2020-11-20T12:01:07.338 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 78/148 2020-11-20T12:01:07.543 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 79/148 2020-11-20T12:01:07.853 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 80/148 2020-11-20T12:01:08.066 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 81/148 2020-11-20T12:01:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 82/148 2020-11-20T12:01:08.425 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 83/148 2020-11-20T12:01:08.692 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 84/148 2020-11-20T12:01:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 98/148 2020-11-20T12:01:08.901 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 85/148 2020-11-20T12:01:09.077 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 86/148 2020-11-20T12:01:09.149 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 98/148 2020-11-20T12:01:09.319 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 87/148 2020-11-20T12:01:09.557 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 88/148 2020-11-20T12:01:09.558 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 89/148 2020-11-20T12:01:09.558 INFO:teuthology.orchestra.run.smithi044.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-20T12:01:09.558 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:01:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 98/148 2020-11-20T12:01:09.935 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.7-12.el8.x86_64 90/148 2020-11-20T12:01:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 99/148 2020-11-20T12:01:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 99/148 2020-11-20T12:01:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 100/148 2020-11-20T12:01:10.536 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 91/148 2020-11-20T12:01:10.536 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 92/148 2020-11-20T12:01:10.886 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 93/148 2020-11-20T12:01:11.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 94/148 2020-11-20T12:01:11.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 95/148 2020-11-20T12:01:11.850 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/148 2020-11-20T12:01:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 97/148 2020-11-20T12:01:16.021 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 98/148 2020-11-20T12:01:16.271 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 98/148 2020-11-20T12:01:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 98/148 2020-11-20T12:01:17.363 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 99/148 2020-11-20T12:01:17.473 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 99/148 2020-11-20T12:01:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 100/148 2020-11-20T12:01:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 100/148 2020-11-20T12:01:27.142 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2020-11-20T12:01:27.142 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2020-11-20T12:01:27.142 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2020-11-20T12:01:27.142 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /run 2020-11-20T12:01:27.142 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2020-11-20T12:01:27.143 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2020-11-20T12:01:27.143 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2020-11-20T12:01:27.143 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2020-11-20T12:01:27.143 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2020-11-20T12:01:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:01:27.981 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 101/148 2020-11-20T12:01:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 101/148 2020-11-20T12:01:28.621 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 102/148 2020-11-20T12:01:30.131 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 102/148 2020-11-20T12:01:30.346 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 103/148 2020-11-20T12:01:30.376 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 103/148 2020-11-20T12:01:30.550 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/148 2020-11-20T12:01:30.779 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/148 2020-11-20T12:01:30.935 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 106/148 2020-11-20T12:01:31.000 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 106/148 2020-11-20T12:01:31.279 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : jq-1.5-12.el8.x86_64 107/148 2020-11-20T12:01:31.510 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.1-6.el8.noarch 108/148 2020-11-20T12:01:32.942 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 109/148 2020-11-20T12:01:33.229 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 109/148 2020-11-20T12:01:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-7397.gb86e691b.e 110/148 2020-11-20T12:01:34.145 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-7397.gb86e691b. 111/148 2020-11-20T12:01:34.334 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noa 112/148 2020-11-20T12:01:35.039 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 100/148 2020-11-20T12:01:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2020-11-20T12:01:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2020-11-20T12:01:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /dev 2020-11-20T12:01:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /run 2020-11-20T12:01:35.040 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2020-11-20T12:01:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2020-11-20T12:01:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2020-11-20T12:01:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2020-11-20T12:01:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /dev 2020-11-20T12:01:35.041 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:01:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noa 112/148 2020-11-20T12:01:35.435 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 101/148 2020-11-20T12:01:35.452 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.n 113/148 2020-11-20T12:01:35.902 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 101/148 2020-11-20T12:01:36.490 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.n 113/148 2020-11-20T12:01:36.543 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 102/148 2020-11-20T12:01:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86 114/148 2020-11-20T12:01:36.745 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86 114/148 2020-11-20T12:01:37.086 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-7397.gb86e691b.el 115/148 2020-11-20T12:01:37.327 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 116/148 2020-11-20T12:01:37.355 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 116/148 2020-11-20T12:01:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 117/148 2020-11-20T12:01:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 117/148 2020-11-20T12:01:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:16.0.0-7397.gb86e691b.el8.x86_64 118/148 2020-11-20T12:01:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/148 2020-11-20T12:01:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 102/148 2020-11-20T12:01:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 103/148 2020-11-20T12:01:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 103/148 2020-11-20T12:01:38.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/148 2020-11-20T12:01:38.788 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/148 2020-11-20T12:01:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 106/148 2020-11-20T12:01:39.058 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 106/148 2020-11-20T12:01:39.386 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : jq-1.5-12.el8.x86_64 107/148 2020-11-20T12:01:39.659 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.1-6.el8.noarch 108/148 2020-11-20T12:01:41.090 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 109/148 2020-11-20T12:01:41.268 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 109/148 2020-11-20T12:01:41.400 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-7397.gb86e691b.e 110/148 2020-11-20T12:01:41.621 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-7397.gb86e691b. 111/148 2020-11-20T12:01:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/148 2020-11-20T12:01:42.409 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noa 112/148 2020-11-20T12:01:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noa 112/148 2020-11-20T12:01:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:16.0.0-7397.gb86e691b.el8.x86_64 120/148 2020-11-20T12:01:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.n 113/148 2020-11-20T12:01:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 121/148 2020-11-20T12:01:44.167 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 121/148 2020-11-20T12:01:44.432 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 122/148 2020-11-20T12:01:44.506 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 122/148 2020-11-20T12:01:44.724 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.n 113/148 2020-11-20T12:01:44.968 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86 114/148 2020-11-20T12:01:45.321 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86 114/148 2020-11-20T12:01:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 123/148 2020-11-20T12:01:45.561 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-7397.gb86e691b.el 115/148 2020-11-20T12:01:45.759 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 116/148 2020-11-20T12:01:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 124/148 2020-11-20T12:01:45.788 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 116/148 2020-11-20T12:01:45.943 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 125/148 2020-11-20T12:01:45.943 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 117/148 2020-11-20T12:01:45.956 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 117/148 2020-11-20T12:01:46.127 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:16.0.0-7397.gb86e691b.el8.x86_64 118/148 2020-11-20T12:01:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:16.0.0-7397.gb86e691b.el8.x86_ 126/148 2020-11-20T12:01:46.283 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/148 2020-11-20T12:01:46.285 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 127/148 2020-11-20T12:01:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 128/148 2020-11-20T12:01:46.917 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 129/148 2020-11-20T12:01:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 130/148 2020-11-20T12:01:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 131/148 2020-11-20T12:01:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:16.0.0-7397.gb86e691b.el8.x86_64 132/148 2020-11-20T12:01:48.007 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-7397.gb86e6 133/148 2020-11-20T12:01:48.115 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-7397.gb86e6 133/148 2020-11-20T12:01:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 134/148 2020-11-20T12:01:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 135/148 2020-11-20T12:01:48.705 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.1-9.el8.x86_64 136/148 2020-11-20T12:01:48.784 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 136/148 2020-11-20T12:01:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.0.4-10.el8.x86_64 137/148 2020-11-20T12:01:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 137/148 2020-11-20T12:01:49.216 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 138/148 2020-11-20T12:01:49.374 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-tui-1:1.20.0-3.el8.x86_64 139/148 2020-11-20T12:01:49.518 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 140/148 2020-11-20T12:01:49.701 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 141/148 2020-11-20T12:01:49.925 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 142/148 2020-11-20T12:01:49.926 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-11-20T12:01:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-11-20T12:01:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-team-1:1.20.0-3.el8.x86_64 144/148 2020-11-20T12:01:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-11-20T12:01:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-11-20T12:01:50.551 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-11-20T12:01:50.551 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-11-20T12:01:50.825 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-11-20T12:01:50.826 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 147/148 2020-11-20T12:01:50.935 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/148 2020-11-20T12:01:51.064 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 147/148 2020-11-20T12:01:51.064 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-11-20T12:01:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-11-20T12:01:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/148 2020-11-20T12:01:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/148 2020-11-20T12:01:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/148 2020-11-20T12:01:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/148 2020-11-20T12:01:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/148 2020-11-20T12:01:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/148 2020-11-20T12:01:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/148 2020-11-20T12:01:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/148 2020-11-20T12:01:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/148 2020-11-20T12:01:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/148 2020-11-20T12:01:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:16.0.0-7397.gb86e691b.el8.x86_64 11/148 2020-11-20T12:01:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 12/148 2020-11-20T12:01:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 13/148 2020-11-20T12:01:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 14/148 2020-11-20T12:01:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-7397.gb86e6 15/148 2020-11-20T12:01:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 16/148 2020-11-20T12:01:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 17/148 2020-11-20T12:01:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 18/148 2020-11-20T12:01:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 19/148 2020-11-20T12:01:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 20/148 2020-11-20T12:01:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 21/148 2020-11-20T12:01:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:16.0.0-7397.gb86e691b.el8.x86_64 22/148 2020-11-20T12:01:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:16.0.0-7397.gb86e691b.el8.x86_ 23/148 2020-11-20T12:01:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 24/148 2020-11-20T12:01:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:16.0.0-7397.gb86e691b.el8.x86_6 25/148 2020-11-20T12:01:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86 26/148 2020-11-20T12:01:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 27/148 2020-11-20T12:01:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:16.0.0-7397.gb86e691b.el 28/148 2020-11-20T12:01:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:16.0.0-7397.gb86e691b.el8. 29/148 2020-11-20T12:01:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:16.0.0-7397.gb86e691b.el8.x86_6 30/148 2020-11-20T12:01:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:16.0.0-7397.gb86e691b.el8.x86_64 31/148 2020-11-20T12:01:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:16.0.0-7397.gb86e691b.el8.x86_64 32/148 2020-11-20T12:01:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:16.0.0-7397.gb86e691b.el8.x86_64 33/148 2020-11-20T12:01:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 34/148 2020-11-20T12:01:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 35/148 2020-11-20T12:01:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:16.0.0-7397.gb86e691b.el8.x86_64 36/148 2020-11-20T12:01:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-7397.gb86e691b. 37/148 2020-11-20T12:01:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noa 38/148 2020-11-20T12:01:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.n 39/148 2020-11-20T12:01:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86 40/148 2020-11-20T12:01:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-7397.gb86e691b.el 41/148 2020-11-20T12:01:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 42/148 2020-11-20T12:01:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-7397.gb86e691b.e 43/148 2020-11-20T12:01:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 44/148 2020-11-20T12:01:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/148 2020-11-20T12:01:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/148 2020-11-20T12:01:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/148 2020-11-20T12:01:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : jq-1.5-12.el8.x86_64 48/148 2020-11-20T12:01:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/148 2020-11-20T12:01:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/148 2020-11-20T12:01:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/148 2020-11-20T12:01:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/148 2020-11-20T12:01:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/148 2020-11-20T12:01:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/148 2020-11-20T12:01:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/148 2020-11-20T12:01:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/148 2020-11-20T12:01:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/148 2020-11-20T12:01:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/148 2020-11-20T12:01:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/148 2020-11-20T12:01:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/148 2020-11-20T12:01:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/148 2020-11-20T12:01:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/148 2020-11-20T12:01:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/148 2020-11-20T12:01:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/148 2020-11-20T12:01:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/148 2020-11-20T12:01:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/148 2020-11-20T12:01:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/148 2020-11-20T12:01:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/148 2020-11-20T12:01:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/148 2020-11-20T12:01:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/148 2020-11-20T12:01:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/148 2020-11-20T12:01:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/148 2020-11-20T12:01:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/148 2020-11-20T12:01:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/148 2020-11-20T12:01:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/148 2020-11-20T12:01:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/148 2020-11-20T12:01:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/148 2020-11-20T12:01:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/148 2020-11-20T12:01:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/148 2020-11-20T12:01:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/148 2020-11-20T12:01:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/148 2020-11-20T12:01:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/148 2020-11-20T12:01:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/148 2020-11-20T12:01:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/148 2020-11-20T12:01:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/148 2020-11-20T12:01:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/148 2020-11-20T12:01:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/148 2020-11-20T12:01:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/148 2020-11-20T12:01:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/148 2020-11-20T12:01:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/148 2020-11-20T12:01:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/148 2020-11-20T12:01:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/148 2020-11-20T12:01:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/148 2020-11-20T12:01:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/148 2020-11-20T12:01:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/148 2020-11-20T12:01:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/148 2020-11-20T12:01:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/148 2020-11-20T12:01:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/148 2020-11-20T12:01:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/148 2020-11-20T12:01:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/148 2020-11-20T12:01:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/148 2020-11-20T12:01:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/148 2020-11-20T12:01:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/148 2020-11-20T12:01:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/148 2020-11-20T12:01:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/148 2020-11-20T12:01:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/148 2020-11-20T12:01:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/148 2020-11-20T12:01:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/148 2020-11-20T12:01:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/148 2020-11-20T12:01:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/148 2020-11-20T12:01:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/148 2020-11-20T12:01:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/148 2020-11-20T12:01:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/148 2020-11-20T12:01:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/148 2020-11-20T12:01:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/148 2020-11-20T12:01:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/148 2020-11-20T12:01:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/148 2020-11-20T12:01:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/148 2020-11-20T12:01:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/148 2020-11-20T12:01:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/148 2020-11-20T12:01:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/148 2020-11-20T12:01:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/148 2020-11-20T12:01:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/148 2020-11-20T12:01:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/148 2020-11-20T12:01:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/148 2020-11-20T12:01:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/148 2020-11-20T12:01:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/148 2020-11-20T12:01:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/148 2020-11-20T12:01:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/148 2020-11-20T12:01:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/148 2020-11-20T12:01:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/148 2020-11-20T12:01:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/148 2020-11-20T12:01:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/148 2020-11-20T12:01:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/148 2020-11-20T12:01:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/148 2020-11-20T12:01:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/148 2020-11-20T12:01:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 137/148 2020-11-20T12:01:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/148 2020-11-20T12:01:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 139/148 2020-11-20T12:01:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/148 2020-11-20T12:01:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.20.0-5.el8_1.x86_64 141/148 2020-11-20T12:01:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-1:1.20.0-3.el8.x86_64 142/148 2020-11-20T12:01:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 143/148 2020-11-20T12:01:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 144/148 2020-11-20T12:01:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 145/148 2020-11-20T12:01:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-team-1:1.20.0-3.el8.x86_64 146/148 2020-11-20T12:01:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 147/148 2020-11-20T12:01:52.318 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:16.0.0-7397.gb86e691b.el8.x86_64 120/148 2020-11-20T12:01:52.539 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 121/148 2020-11-20T12:01:52.932 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 121/148 2020-11-20T12:01:53.208 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 122/148 2020-11-20T12:01:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 122/148 2020-11-20T12:01:53.688 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-tui-1:1.20.0-3.el8.x86_64 148/148 2020-11-20T12:01:53.688 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:01:53.688 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2020-11-20T12:01:53.688 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.689 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.689 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:01:53.689 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:01:53.689 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-team-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:01:53.690 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:01:53.690 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:01:53.690 INFO:teuthology.orchestra.run.smithi079.stdout:Downgraded: 2020-11-20T12:01:53.690 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-20T12:01:53.690 INFO:teuthology.orchestra.run.smithi079.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T12:01:53.691 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T12:01:53.691 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:01:53.691 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2020-11-20T12:01:53.691 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.693 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.693 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.693 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.693 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.693 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.694 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.694 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.695 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.695 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.695 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.695 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.695 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.696 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.696 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.697 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.0.4-10.el8.x86_64 2020-11-20T12:01:53.697 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.1-9.el8.x86_64 2020-11-20T12:01:53.697 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-20T12:01:53.697 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-20T12:01:53.697 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-20T12:01:53.698 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-20T12:01:53.698 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T12:01:53.698 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-20T12:01:53.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-20T12:01:53.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-20T12:01:53.699 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.699 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.699 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.699 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.699 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.700 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.700 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:01:53.700 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.701 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.701 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:01:53.701 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.1-6.el8.noarch 2020-11-20T12:01:53.701 INFO:teuthology.orchestra.run.smithi079.stdout: jq-1.5-12.el8.x86_64 2020-11-20T12:01:53.702 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-20T12:01:53.702 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-20T12:01:53.702 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-20T12:01:53.702 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-20T12:01:53.703 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-20T12:01:53.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-20T12:01:53.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-20T12:01:53.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-20T12:01:53.703 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-20T12:01:53.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-20T12:01:53.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-20T12:01:53.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-20T12:01:53.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-20T12:01:53.704 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-20T12:01:53.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-20T12:01:53.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-20T12:01:53.705 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-20T12:01:53.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-20T12:01:53.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-20T12:01:53.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-20T12:01:53.706 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-20T12:01:53.706 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-20T12:01:53.707 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-20T12:01:53.707 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-20T12:01:53.707 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-20T12:01:53.707 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-20T12:01:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-20T12:01:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-20T12:01:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-20T12:01:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-20T12:01:53.708 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-20T12:01:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-20T12:01:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-20T12:01:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-20T12:01:53.709 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-20T12:01:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-20T12:01:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-20T12:01:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-20T12:01:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-20T12:01:53.710 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-20T12:01:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-20T12:01:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-20T12:01:53.711 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-20T12:01:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-20T12:01:53.712 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-20T12:01:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-20T12:01:53.713 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-20T12:01:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: leveldb-1.22-1.el8.x86_64 2020-11-20T12:01:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-20T12:01:53.714 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-20T12:01:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-20T12:01:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-20T12:01:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-20T12:01:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-20T12:01:53.715 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-20T12:01:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-20T12:01:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-20T12:01:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-20T12:01:53.716 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-20T12:01:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-20T12:01:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-20T12:01:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-20T12:01:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-20T12:01:53.717 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-20T12:01:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-20T12:01:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-2.6-1.el8.noarch 2020-11-20T12:01:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-20T12:01:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-20T12:01:53.718 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-20T12:01:53.719 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-20T12:01:53.719 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-20T12:01:53.719 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-20T12:01:53.719 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-20T12:01:53.720 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-20T12:01:53.720 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-20T12:01:53.720 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-20T12:01:53.720 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-20T12:01:53.720 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-20T12:01:53.721 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-20T12:01:53.721 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-20T12:01:53.721 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:01:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2020-11-20T12:01:53.841 DEBUG:teuthology.parallel:result is None 2020-11-20T12:01:54.238 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 123/148 2020-11-20T12:01:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 124/148 2020-11-20T12:01:54.694 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 125/148 2020-11-20T12:01:54.885 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:16.0.0-7397.gb86e691b.el8.x86_ 126/148 2020-11-20T12:01:55.061 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 127/148 2020-11-20T12:01:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 128/148 2020-11-20T12:01:55.711 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 129/148 2020-11-20T12:01:56.099 INFO:teuthology.orchestra.run.smithi044.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 130/148 2020-11-20T12:01:56.366 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 131/148 2020-11-20T12:01:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:16.0.0-7397.gb86e691b.el8.x86_64 132/148 2020-11-20T12:01:56.849 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-7397.gb86e6 133/148 2020-11-20T12:01:56.965 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-7397.gb86e6 133/148 2020-11-20T12:01:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 134/148 2020-11-20T12:01:57.399 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 135/148 2020-11-20T12:01:57.598 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.1-9.el8.x86_64 136/148 2020-11-20T12:01:57.682 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 136/148 2020-11-20T12:01:57.882 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.0.4-10.el8.x86_64 137/148 2020-11-20T12:01:57.913 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 137/148 2020-11-20T12:01:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 138/148 2020-11-20T12:01:58.250 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-tui-1:1.20.0-3.el8.x86_64 139/148 2020-11-20T12:01:58.412 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 140/148 2020-11-20T12:01:58.578 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 141/148 2020-11-20T12:01:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 142/148 2020-11-20T12:01:58.934 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-11-20T12:01:59.676 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 143/148 2020-11-20T12:01:59.817 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-team-1:1.20.0-3.el8.x86_64 144/148 2020-11-20T12:01:59.835 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-11-20T12:01:59.836 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-11-20T12:02:00.119 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.20.0-3.el8.x86_64 145/148 2020-11-20T12:02:00.120 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-11-20T12:02:00.343 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.20.0-3.el8.x86_64 146/148 2020-11-20T12:02:00.344 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 147/148 2020-11-20T12:02:00.610 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 147/148 2020-11-20T12:02:00.610 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-11-20T12:02:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 148/148 2020-11-20T12:02:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/148 2020-11-20T12:02:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/148 2020-11-20T12:02:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/148 2020-11-20T12:02:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/148 2020-11-20T12:02:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/148 2020-11-20T12:02:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/148 2020-11-20T12:02:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/148 2020-11-20T12:02:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/148 2020-11-20T12:02:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/148 2020-11-20T12:02:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/148 2020-11-20T12:02:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:16.0.0-7397.gb86e691b.el8.x86_64 11/148 2020-11-20T12:02:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 12/148 2020-11-20T12:02:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 13/148 2020-11-20T12:02:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 14/148 2020-11-20T12:02:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-7397.gb86e6 15/148 2020-11-20T12:02:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 16/148 2020-11-20T12:02:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 17/148 2020-11-20T12:02:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 18/148 2020-11-20T12:02:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 19/148 2020-11-20T12:02:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 20/148 2020-11-20T12:02:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 21/148 2020-11-20T12:02:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:16.0.0-7397.gb86e691b.el8.x86_64 22/148 2020-11-20T12:02:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:16.0.0-7397.gb86e691b.el8.x86_ 23/148 2020-11-20T12:02:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 24/148 2020-11-20T12:02:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:16.0.0-7397.gb86e691b.el8.x86_6 25/148 2020-11-20T12:02:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86 26/148 2020-11-20T12:02:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 27/148 2020-11-20T12:02:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:16.0.0-7397.gb86e691b.el 28/148 2020-11-20T12:02:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:16.0.0-7397.gb86e691b.el8. 29/148 2020-11-20T12:02:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:16.0.0-7397.gb86e691b.el8.x86_6 30/148 2020-11-20T12:02:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:16.0.0-7397.gb86e691b.el8.x86_64 31/148 2020-11-20T12:02:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:16.0.0-7397.gb86e691b.el8.x86_64 32/148 2020-11-20T12:02:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:16.0.0-7397.gb86e691b.el8.x86_64 33/148 2020-11-20T12:02:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 34/148 2020-11-20T12:02:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 35/148 2020-11-20T12:02:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:16.0.0-7397.gb86e691b.el8.x86_64 36/148 2020-11-20T12:02:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-7397.gb86e691b. 37/148 2020-11-20T12:02:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noa 38/148 2020-11-20T12:02:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.n 39/148 2020-11-20T12:02:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86 40/148 2020-11-20T12:02:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-7397.gb86e691b.el 41/148 2020-11-20T12:02:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 42/148 2020-11-20T12:02:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-7397.gb86e691b.e 43/148 2020-11-20T12:02:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 44/148 2020-11-20T12:02:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/148 2020-11-20T12:02:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/148 2020-11-20T12:02:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/148 2020-11-20T12:02:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : jq-1.5-12.el8.x86_64 48/148 2020-11-20T12:02:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/148 2020-11-20T12:02:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/148 2020-11-20T12:02:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/148 2020-11-20T12:02:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/148 2020-11-20T12:02:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/148 2020-11-20T12:02:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/148 2020-11-20T12:02:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/148 2020-11-20T12:02:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/148 2020-11-20T12:02:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/148 2020-11-20T12:02:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/148 2020-11-20T12:02:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/148 2020-11-20T12:02:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/148 2020-11-20T12:02:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/148 2020-11-20T12:02:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/148 2020-11-20T12:02:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/148 2020-11-20T12:02:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/148 2020-11-20T12:02:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/148 2020-11-20T12:02:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/148 2020-11-20T12:02:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/148 2020-11-20T12:02:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/148 2020-11-20T12:02:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/148 2020-11-20T12:02:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/148 2020-11-20T12:02:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/148 2020-11-20T12:02:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/148 2020-11-20T12:02:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/148 2020-11-20T12:02:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/148 2020-11-20T12:02:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/148 2020-11-20T12:02:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/148 2020-11-20T12:02:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/148 2020-11-20T12:02:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/148 2020-11-20T12:02:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/148 2020-11-20T12:02:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/148 2020-11-20T12:02:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/148 2020-11-20T12:02:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/148 2020-11-20T12:02:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/148 2020-11-20T12:02:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/148 2020-11-20T12:02:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/148 2020-11-20T12:02:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/148 2020-11-20T12:02:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/148 2020-11-20T12:02:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/148 2020-11-20T12:02:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/148 2020-11-20T12:02:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/148 2020-11-20T12:02:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/148 2020-11-20T12:02:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/148 2020-11-20T12:02:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/148 2020-11-20T12:02:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/148 2020-11-20T12:02:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/148 2020-11-20T12:02:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/148 2020-11-20T12:02:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/148 2020-11-20T12:02:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/148 2020-11-20T12:02:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/148 2020-11-20T12:02:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/148 2020-11-20T12:02:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/148 2020-11-20T12:02:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/148 2020-11-20T12:02:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/148 2020-11-20T12:02:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/148 2020-11-20T12:02:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/148 2020-11-20T12:02:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/148 2020-11-20T12:02:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/148 2020-11-20T12:02:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/148 2020-11-20T12:02:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/148 2020-11-20T12:02:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/148 2020-11-20T12:02:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/148 2020-11-20T12:02:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/148 2020-11-20T12:02:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/148 2020-11-20T12:02:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/148 2020-11-20T12:02:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/148 2020-11-20T12:02:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/148 2020-11-20T12:02:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/148 2020-11-20T12:02:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/148 2020-11-20T12:02:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/148 2020-11-20T12:02:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/148 2020-11-20T12:02:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/148 2020-11-20T12:02:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/148 2020-11-20T12:02:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/148 2020-11-20T12:02:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/148 2020-11-20T12:02:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/148 2020-11-20T12:02:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/148 2020-11-20T12:02:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/148 2020-11-20T12:02:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/148 2020-11-20T12:02:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/148 2020-11-20T12:02:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/148 2020-11-20T12:02:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/148 2020-11-20T12:02:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/148 2020-11-20T12:02:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/148 2020-11-20T12:02:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/148 2020-11-20T12:02:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/148 2020-11-20T12:02:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/148 2020-11-20T12:02:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 137/148 2020-11-20T12:02:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/148 2020-11-20T12:02:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 139/148 2020-11-20T12:02:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/148 2020-11-20T12:02:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.20.0-5.el8_1.x86_64 141/148 2020-11-20T12:02:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.20.0-3.el8.x86_64 142/148 2020-11-20T12:02:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 143/148 2020-11-20T12:02:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.20.0-3.el8.x86_64 144/148 2020-11-20T12:02:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.20.0-5.el8_1.x86_64 145/148 2020-11-20T12:02:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.20.0-3.el8.x86_64 146/148 2020-11-20T12:02:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 147/148 2020-11-20T12:02:03.477 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.20.0-3.el8.x86_64 148/148 2020-11-20T12:02:03.477 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:03.477 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2020-11-20T12:02:03.477 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.478 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.478 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:02:03.478 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:02:03.478 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:02:03.479 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui-1:1.20.0-5.el8_1.x86_64 2020-11-20T12:02:03.479 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:03.479 INFO:teuthology.orchestra.run.smithi044.stdout:Downgraded: 2020-11-20T12:02:03.479 INFO:teuthology.orchestra.run.smithi044.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-20T12:02:03.479 INFO:teuthology.orchestra.run.smithi044.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T12:02:03.480 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T12:02:03.480 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:03.480 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2020-11-20T12:02:03.480 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.481 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.481 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.481 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.481 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.482 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.482 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.482 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.482 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.483 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.483 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.483 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.483 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.484 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.484 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.484 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.484 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.485 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.485 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.485 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.485 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.485 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.486 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.0.4-10.el8.x86_64 2020-11-20T12:02:03.486 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.1-9.el8.x86_64 2020-11-20T12:02:03.486 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-20T12:02:03.486 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-20T12:02:03.487 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-20T12:02:03.487 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-20T12:02:03.487 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T12:02:03.487 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-20T12:02:03.487 INFO:teuthology.orchestra.run.smithi044.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-20T12:02:03.488 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-20T12:02:03.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.488 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.489 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.489 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.489 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.489 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.490 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:16.0.0-7397.gb86e691b.el8.x86_64 2020-11-20T12:02:03.490 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.490 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.490 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:16.0.0-7397.gb86e691b.el8.noarch 2020-11-20T12:02:03.490 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.1-6.el8.noarch 2020-11-20T12:02:03.491 INFO:teuthology.orchestra.run.smithi044.stdout: jq-1.5-12.el8.x86_64 2020-11-20T12:02:03.491 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-20T12:02:03.491 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-20T12:02:03.492 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-20T12:02:03.492 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-20T12:02:03.492 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-20T12:02:03.492 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-20T12:02:03.492 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-20T12:02:03.493 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-20T12:02:03.493 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-20T12:02:03.493 INFO:teuthology.orchestra.run.smithi044.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-20T12:02:03.493 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-20T12:02:03.493 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-20T12:02:03.494 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-20T12:02:03.494 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-20T12:02:03.494 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-20T12:02:03.494 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-20T12:02:03.495 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-20T12:02:03.495 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-20T12:02:03.495 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-20T12:02:03.495 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-20T12:02:03.495 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-20T12:02:03.496 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-20T12:02:03.496 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-20T12:02:03.496 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-20T12:02:03.496 INFO:teuthology.orchestra.run.smithi044.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-20T12:02:03.496 INFO:teuthology.orchestra.run.smithi044.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-20T12:02:03.497 INFO:teuthology.orchestra.run.smithi044.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-20T12:02:03.497 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-20T12:02:03.497 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-20T12:02:03.497 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-20T12:02:03.498 INFO:teuthology.orchestra.run.smithi044.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-20T12:02:03.498 INFO:teuthology.orchestra.run.smithi044.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-20T12:02:03.498 INFO:teuthology.orchestra.run.smithi044.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-20T12:02:03.498 INFO:teuthology.orchestra.run.smithi044.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-20T12:02:03.499 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-20T12:02:03.499 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-20T12:02:03.499 INFO:teuthology.orchestra.run.smithi044.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-20T12:02:03.499 INFO:teuthology.orchestra.run.smithi044.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-20T12:02:03.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-20T12:02:03.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-20T12:02:03.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-20T12:02:03.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-20T12:02:03.500 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-20T12:02:03.501 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-20T12:02:03.501 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-20T12:02:03.501 INFO:teuthology.orchestra.run.smithi044.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-20T12:02:03.501 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-20T12:02:03.502 INFO:teuthology.orchestra.run.smithi044.stdout: leveldb-1.22-1.el8.x86_64 2020-11-20T12:02:03.502 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-20T12:02:03.502 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-20T12:02:03.502 INFO:teuthology.orchestra.run.smithi044.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-20T12:02:03.502 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-20T12:02:03.503 INFO:teuthology.orchestra.run.smithi044.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-20T12:02:03.503 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-20T12:02:03.503 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-20T12:02:03.503 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-20T12:02:03.503 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-20T12:02:03.504 INFO:teuthology.orchestra.run.smithi044.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-20T12:02:03.504 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-20T12:02:03.504 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-20T12:02:03.504 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-20T12:02:03.505 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-20T12:02:03.505 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-20T12:02:03.505 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-20T12:02:03.505 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-20T12:02:03.505 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-2.6-1.el8.noarch 2020-11-20T12:02:03.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-20T12:02:03.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-20T12:02:03.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-20T12:02:03.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-20T12:02:03.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-20T12:02:03.507 INFO:teuthology.orchestra.run.smithi044.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-20T12:02:03.507 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-20T12:02:03.507 INFO:teuthology.orchestra.run.smithi044.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-20T12:02:03.507 INFO:teuthology.orchestra.run.smithi044.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-20T12:02:03.507 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-20T12:02:03.508 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-20T12:02:03.508 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-20T12:02:03.508 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-20T12:02:03.508 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-20T12:02:03.509 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:03.509 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2020-11-20T12:02:03.687 DEBUG:teuthology.parallel:result is None 2020-11-20T12:02:03.688 INFO:teuthology.orchestra.run.smithi044:> sudo lsb_release -is 2020-11-20T12:02:03.742 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS 2020-11-20T12:02:03.743 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T12:02:03.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:02:03.905 INFO:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-20T12:02:03.925 INFO:teuthology.orchestra.run.smithi044.stdout:16.0.0-7397.gb86e691b.el8 2020-11-20T12:02:03.926 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7397.gb86e691b.el8 2020-11-20T12:02:03.926 INFO:teuthology.task.install:The correct ceph version 16.0.0-7397.gb86e691b is installed. 2020-11-20T12:02:03.927 INFO:teuthology.orchestra.run.smithi079:> sudo lsb_release -is 2020-11-20T12:02:03.981 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS 2020-11-20T12:02:03.982 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T12:02:03.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:02:04.146 INFO:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-20T12:02:04.171 INFO:teuthology.orchestra.run.smithi079.stdout:16.0.0-7397.gb86e691b.el8 2020-11-20T12:02:04.171 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7397.gb86e691b.el8 2020-11-20T12:02:04.172 INFO:teuthology.task.install:The correct ceph version 16.0.0-7397.gb86e691b is installed. 2020-11-20T12:02:04.172 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-20T12:02:04.172 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:04.173 INFO:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:02:04.186 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:02:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:24+1 records in 2020-11-20T12:02:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:24+1 records out 2020-11-20T12:02:04.203 INFO:teuthology.orchestra.run.smithi044.stderr:12667 bytes (13 kB, 12 KiB) copied, 8.216e-05 s, 154 MB/s 2020-11-20T12:02:04.204 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:04.205 INFO:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:02:04.225 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:02:04.239 INFO:teuthology.orchestra.run.smithi079.stderr:24+1 records in 2020-11-20T12:02:04.240 INFO:teuthology.orchestra.run.smithi079.stderr:24+1 records out 2020-11-20T12:02:04.240 INFO:teuthology.orchestra.run.smithi079.stderr:12667 bytes (13 kB, 12 KiB) copied, 7.5433e-05 s, 168 MB/s 2020-11-20T12:02:04.241 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-20T12:02:04.241 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:04.241 INFO:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:02:04.257 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:02:04.273 INFO:teuthology.orchestra.run.smithi044.stderr:5+1 records in 2020-11-20T12:02:04.273 INFO:teuthology.orchestra.run.smithi044.stderr:5+1 records out 2020-11-20T12:02:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 3.9947e-05 s, 76.6 MB/s 2020-11-20T12:02:04.274 INFO:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T12:02:04.345 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:04.345 INFO:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:02:04.360 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:02:04.374 INFO:teuthology.orchestra.run.smithi079.stderr:5+1 records in 2020-11-20T12:02:04.375 INFO:teuthology.orchestra.run.smithi079.stderr:5+1 records out 2020-11-20T12:02:04.375 INFO:teuthology.orchestra.run.smithi079.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 3.7204e-05 s, 82.2 MB/s 2020-11-20T12:02:04.376 INFO:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T12:02:04.446 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-20T12:02:04.447 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:04.447 INFO:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:02:04.467 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:02:04.483 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:04.483 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:04.483 INFO:teuthology.orchestra.run.smithi044.stderr:299 bytes copied, 2.7615e-05 s, 10.8 MB/s 2020-11-20T12:02:04.484 INFO:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T12:02:04.561 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:04.561 INFO:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:02:04.579 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:02:04.593 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:04.593 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:04.594 INFO:teuthology.orchestra.run.smithi079.stderr:299 bytes copied, 2.7944e-05 s, 10.7 MB/s 2020-11-20T12:02:04.595 INFO:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T12:02:04.668 INFO:teuthology.run_tasks:Running task ceph... 2020-11-20T12:02:04.724 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-20T12:02:04.724 INFO:teuthology.orchestra.run.smithi044:> sudo chmod 777 /var/log/ceph 2020-11-20T12:02:04.730 INFO:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2020-11-20T12:02:04.759 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-20T12:02:04.760 INFO:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T12:02:04.802 INFO:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T12:02:04.830 INFO:tasks.ceph:Creating extra log directories... 2020-11-20T12:02:04.831 INFO:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T12:02:04.873 INFO:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T12:02:04.922 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-11-20T12:02:04.923 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-11-20T12:02:04.923 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-11-20T12:02:04.924 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:04.924 INFO:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:02:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:02:04.957 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:04.957 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:04.958 INFO:teuthology.orchestra.run.smithi044.stderr:408 bytes copied, 2.7898e-05 s, 14.6 MB/s 2020-11-20T12:02:04.958 INFO:teuthology.orchestra.run.smithi044:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T12:02:05.079 INFO:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-11-20T12:02:05.111 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:05.112 INFO:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:02:05.125 INFO:teuthology.orchestra.run.smithi079.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:02:05.126 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:05.127 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:05.127 INFO:teuthology.orchestra.run.smithi079.stderr:408 bytes copied, 2.9037e-05 s, 14.1 MB/s 2020-11-20T12:02:05.127 INFO:teuthology.orchestra.run.smithi079:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T12:02:05.239 INFO:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-11-20T12:02:05.275 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-20T12:02:05.275 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-20T12:02:05.276 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640321', 'branch': 'wip-pdonnell-testing-20201120.053228', 'description': 'fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '5640321', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.1', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20201120.053228'}, 'ceph': {'cephfs': {'session_timeout': 300}, 'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20201120.053228', 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 95, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20201120.053228', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}}, {'ceph': None}, {'ceph-fuse': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['fs/snaps']}}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-20_08:39:42', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.6017', 'targets': {'smithi044.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCl10bOspedT7YS1oXeglqpT7ktwelK/Q86t0MBR9xm+UURVA6meBVFcIoDhear3I1rgltd1bdrxS9r9Hfsa1bQKI3WlFhgTKTiP8NF79KRFkLSiipP/SvWhvB6CaHD8N2nsJRAEUOwCoN+6RveN8CwAlai2yd2peVAn52cTAWTdd95pEBQOTcf78KnLJeImNAnNW10ohBwCETu7hOtSk4jXtCZQ83DrMmNB5qTTXjIB51nveB1JlDBR+YRjP0EMOH6fAmaYXTghScevPj9qBJKfUR39OHhur6G0iFOWE7mvIlqzzGpxQgnmC4EqDIJDWzdZSYZ1IzXVAC/K7DtvdLI1cQ0upAxk+PS+vd0K28v3MfKVP5Kf768yi3BkHgJjJ4xLjT3c5ksIR9B90CTp5kiKubasFDtjLkqdvkVHpECHsW0LfiouJFbi0C6eFNRIK/g54sxGy41bn9UnWkKeYFpFtsI8SWquEWHmSeqwQv2nXbirdng5mgQ7IDj3luMCoc=', 'smithi079.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDqNQM/Nvn6E1TgaiyIBJo1T3KqoP8jOjDbIL5PQZ5jSXzxK+uhcb7SJn8FlZzY809zObUtoCCy25mGYaJOhe+LHjXt+tiYRYH7jmRXbRiqOhH5cuqLY1MV0c8qgsyTloDv89aRlcmF2VkORG7A+OCAVaMdU4q8FJskpgdcCE8pQtrInXJiqPtD8DcRsIc/JCpYAx/gzQxaax14KfpTqfkYIfQsSBhO02hh+HbleSUxuyNJ/v0okBwkeQcsJn63lit7vzabqMUwYntl37jGXnelZvcyxZbXg9xCGwAvpp1RT/Wi3DjlUO4Hfm9z518KHeXPomVzFK1GuUXmWbDGwmv/N7vTlLOQVOASjeTkORatueGvn0KR4oxeDPC5ukZU5a/tyVoIHC76BRaUmTSkFHVDjlF9Zj64wvAs112rx/bY0FYbzFeFhRzXWUOqSg3xW2CH6LbZXMid41VxpcDSqIIV37+FpeSacnbHemSY4wCa4AGazHRX1LK34C83+xcr0GM='}} 2020-11-20T12:02:05.277 INFO:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T12:02:05.279 INFO:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T12:02:05.320 INFO:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T12:02:05.333 INFO:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T12:02:05.388 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:05.388 INFO:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:02:05.416 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:02:05.417 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/vg_nvme/lv_1 2020-11-20T12:02:05.418 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/vg_nvme/lv_2 2020-11-20T12:02:05.418 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/vg_nvme/lv_3 2020-11-20T12:02:05.418 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/vg_nvme/lv_4 2020-11-20T12:02:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:05.418 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:05.419 INFO:teuthology.orchestra.run.smithi044.stderr:72 bytes copied, 4.9922e-05 s, 1.4 MB/s 2020-11-20T12:02:05.419 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T12:02:05.419 INFO:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T12:02:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T12:02:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:05.494 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 59359 Links: 1 2020-11-20T12:02:05.495 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:05.495 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:05.495 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2020-11-20 12:02:01.175578917 +0000 2020-11-20T12:02:05.495 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2020-11-20 12:00:00.752016449 +0000 2020-11-20T12:02:05.496 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2020-11-20 12:00:00.752016449 +0000 2020-11-20T12:02:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2020-11-20T12:02:05.510 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2020-11-20T12:02:05.510 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2020-11-20T12:02:05.511 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000235731 s, 2.2 MB/s 2020-11-20T12:02:05.513 INFO:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T12:02:05.605 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T12:02:05.605 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:05.605 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 60412 Links: 1 2020-11-20T12:02:05.605 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:05.606 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:05.606 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2020-11-20 12:02:01.175578917 +0000 2020-11-20T12:02:05.606 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2020-11-20 12:00:01.122010466 +0000 2020-11-20T12:02:05.606 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2020-11-20 12:00:01.122010466 +0000 2020-11-20T12:02:05.606 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2020-11-20T12:02:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2020-11-20T12:02:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2020-11-20T12:02:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.0002035 s, 2.5 MB/s 2020-11-20T12:02:05.628 INFO:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T12:02:05.709 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T12:02:05.710 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:05.710 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 61009 Links: 1 2020-11-20T12:02:05.710 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:05.711 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:05.711 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2020-11-20 12:02:01.175578917 +0000 2020-11-20T12:02:05.712 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2020-11-20 12:00:01.485004596 +0000 2020-11-20T12:02:05.712 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2020-11-20 12:00:01.485004596 +0000 2020-11-20T12:02:05.713 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2020-11-20T12:02:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2020-11-20T12:02:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2020-11-20T12:02:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000207453 s, 2.5 MB/s 2020-11-20T12:02:05.727 INFO:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T12:02:05.798 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T12:02:05.798 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:05.798 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 66616 Links: 1 2020-11-20T12:02:05.798 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:05.799 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:05.799 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2020-11-20 12:02:01.175578917 +0000 2020-11-20T12:02:05.799 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2020-11-20 12:00:01.824999097 +0000 2020-11-20T12:02:05.799 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2020-11-20 12:00:01.824999097 +0000 2020-11-20T12:02:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2020-11-20T12:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2020-11-20T12:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2020-11-20T12:02:05.817 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000176259 s, 2.9 MB/s 2020-11-20T12:02:05.818 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:05.818 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:05.818 INFO:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:02:05.834 INFO:teuthology.orchestra.run.smithi079.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:02:05.835 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/vg_nvme/lv_1 2020-11-20T12:02:05.835 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/vg_nvme/lv_2 2020-11-20T12:02:05.835 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/vg_nvme/lv_3 2020-11-20T12:02:05.836 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/vg_nvme/lv_4 2020-11-20T12:02:05.836 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:05.836 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:05.836 INFO:teuthology.orchestra.run.smithi079.stderr:72 bytes copied, 3.2272e-05 s, 2.2 MB/s 2020-11-20T12:02:05.837 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T12:02:05.837 INFO:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T12:02:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T12:02:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:05.892 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 59009 Links: 1 2020-11-20T12:02:05.893 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:05.893 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:05.893 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2020-11-20 12:01:51.563104651 +0000 2020-11-20T12:02:05.893 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2020-11-20 11:59:57.712292263 +0000 2020-11-20T12:02:05.894 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2020-11-20 11:59:57.712292263 +0000 2020-11-20T12:02:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2020-11-20T12:02:05.906 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2020-11-20T12:02:05.906 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2020-11-20T12:02:05.906 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.00021905 s, 2.3 MB/s 2020-11-20T12:02:05.909 INFO:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T12:02:05.964 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T12:02:05.964 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:05.964 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 58064 Links: 1 2020-11-20T12:02:05.965 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:05.965 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:05.965 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2020-11-20 12:01:51.563104651 +0000 2020-11-20T12:02:05.965 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2020-11-20 11:59:58.043284657 +0000 2020-11-20T12:02:05.965 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2020-11-20 11:59:58.043284657 +0000 2020-11-20T12:02:05.966 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2020-11-20T12:02:05.977 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2020-11-20T12:02:05.977 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2020-11-20T12:02:05.978 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.00023111 s, 2.2 MB/s 2020-11-20T12:02:05.980 INFO:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T12:02:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T12:02:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:06.035 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 62892 Links: 1 2020-11-20T12:02:06.035 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:06.036 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:06.036 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2020-11-20 12:01:51.563104651 +0000 2020-11-20T12:02:06.036 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2020-11-20 11:59:58.415276109 +0000 2020-11-20T12:02:06.036 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2020-11-20 11:59:58.415276109 +0000 2020-11-20T12:02:06.036 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2020-11-20T12:02:06.048 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2020-11-20T12:02:06.048 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2020-11-20T12:02:06.048 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000212875 s, 2.4 MB/s 2020-11-20T12:02:06.051 INFO:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T12:02:06.105 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T12:02:06.106 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:02:06.106 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 61358 Links: 1 2020-11-20T12:02:06.106 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:02:06.106 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T12:02:06.106 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2020-11-20 12:01:51.563104651 +0000 2020-11-20T12:02:06.107 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2020-11-20 11:59:58.777267791 +0000 2020-11-20T12:02:06.107 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2020-11-20 11:59:58.777267791 +0000 2020-11-20T12:02:06.107 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2020-11-20T12:02:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2020-11-20T12:02:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2020-11-20T12:02:06.119 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000210586 s, 2.4 MB/s 2020-11-20T12:02:06.121 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:06.121 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T12:02:06.122 INFO:tasks.ceph:Generating config... 2020-11-20T12:02:06.123 INFO:tasks.ceph:[client] client mount timeout = 600 2020-11-20T12:02:06.123 INFO:tasks.ceph:[client] debug client = 20 2020-11-20T12:02:06.124 INFO:tasks.ceph:[client] debug ms = 1 2020-11-20T12:02:06.124 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-11-20T12:02:06.124 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-11-20T12:02:06.124 INFO:tasks.ceph:[mds] debug mds = 20 2020-11-20T12:02:06.125 INFO:tasks.ceph:[mds] debug ms = 1 2020-11-20T12:02:06.125 INFO:tasks.ceph:[mds] mds bal frag = True 2020-11-20T12:02:06.125 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-11-20T12:02:06.126 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-11-20T12:02:06.126 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-11-20T12:02:06.126 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-11-20T12:02:06.126 INFO:tasks.ceph:[mds] mds debug frag = True 2020-11-20T12:02:06.127 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-11-20T12:02:06.127 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-11-20T12:02:06.127 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-11-20T12:02:06.128 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-11-20T12:02:06.128 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-11-20T12:02:06.128 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-11-20T12:02:06.128 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-20T12:02:06.129 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-20T12:02:06.129 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-20T12:02:06.129 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-20T12:02:06.129 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-20T12:02:06.130 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-11-20T12:02:06.130 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-20T12:02:06.130 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2020-11-20T12:02:06.130 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-20T12:02:06.131 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-20T12:02:06.131 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-20T12:02:06.131 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-20T12:02:06.132 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-20T12:02:06.132 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-20T12:02:06.132 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-20T12:02:06.132 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-20T12:02:06.133 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-20T12:02:06.133 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-20T12:02:06.133 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-20T12:02:06.134 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-20T12:02:06.134 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2020-11-20T12:02:06.134 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-20T12:02:06.134 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-11-20T12:02:06.135 INFO:tasks.ceph:Setting up mon.a... 2020-11-20T12:02:06.135 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-20T12:02:06.193 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.keyring 2020-11-20T12:02:06.195 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-20T12:02:06.241 INFO:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:02:06.271 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.44'), ('mon.b', '172.21.15.79'), ('mon.c', '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]')] 2020-11-20T12:02:06.272 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.44,172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}} 2020-11-20T12:02:06.273 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:06.274 INFO:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-20T12:02:06.324 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-20T12:02:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:6+1 records in 2020-11-20T12:02:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:6+1 records out 2020-11-20T12:02:06.326 INFO:teuthology.orchestra.run.smithi044.stderr:3486 bytes (3.5 kB, 3.4 KiB) copied, 4.9824e-05 s, 70.0 MB/s 2020-11-20T12:02:06.326 INFO:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.44 --add b 172.21.15.79 --addv c '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:06.406 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:06.407 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: generated fsid 0437e350-ccfe-424c-9e91-59ff58800c77 2020-11-20T12:02:06.407 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 0 2020-11-20T12:02:06.408 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 0437e350-ccfe-424c-9e91-59ff58800c77 2020-11-20T12:02:06.408 INFO:teuthology.orchestra.run.smithi044.stdout:last_changed 2020-11-20T12:02:06.403922+0000 2020-11-20T12:02:06.408 INFO:teuthology.orchestra.run.smithi044.stdout:created 2020-11-20T12:02:06.403922+0000 2020-11-20T12:02:06.409 INFO:teuthology.orchestra.run.smithi044.stdout:min_mon_release 0 (unknown) 2020-11-20T12:02:06.410 INFO:teuthology.orchestra.run.smithi044.stdout:election_strategy: 1 2020-11-20T12:02:06.410 INFO:teuthology.orchestra.run.smithi044.stdout:0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2020-11-20T12:02:06.410 INFO:teuthology.orchestra.run.smithi044.stdout:1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.b 2020-11-20T12:02:06.411 INFO:teuthology.orchestra.run.smithi044.stdout:2: [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] mon.c 2020-11-20T12:02:06.411 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-20T12:02:06.413 INFO:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-11-20T12:02:06.467 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0437e350-ccfe-424c-9e91-59ff58800c77... 2020-11-20T12:02:06.468 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:02:06.510 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:02:06.565 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2020-11-20T12:02:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2020-11-20T12:02:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:02:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2020-11-20T12:02:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2020-11-20T12:02:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2020-11-20T12:02:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2020-11-20T12:02:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2020-11-20T12:02:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2020-11-20T12:02:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2020-11-20T12:02:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:02:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2020-11-20T12:02:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:02:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:02:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2020-11-20T12:02:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2020-11-20T12:02:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2020-11-20T12:02:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:02:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:02:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2020-11-20T12:02:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:02:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2020-11-20T12:02:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:02:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:02:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2020-11-20T12:02:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:02:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2020-11-20T12:02:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2020-11-20T12:02:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2020-11-20T12:02:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2020-11-20T12:02:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2020-11-20T12:02:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.44,172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790]" 2020-11-20T12:02:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = 0437e350-ccfe-424c-9e91-59ff58800c77 2020-11-20T12:02:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.577 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2020-11-20T12:02:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2020-11-20T12:02:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2020-11-20T12:02:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2020-11-20T12:02:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2020-11-20T12:02:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2020-11-20T12:02:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2020-11-20T12:02:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2020-11-20T12:02:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2020-11-20T12:02:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2020-11-20T12:02:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2020-11-20T12:02:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:02:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2020-11-20T12:02:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2020-11-20T12:02:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:02:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2020-11-20T12:02:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2020-11-20T12:02:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2020-11-20T12:02:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2020-11-20T12:02:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore compression mode = aggressive 2020-11-20T12:02:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2020-11-20T12:02:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 20 2020-11-20T12:02:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 20 2020-11-20T12:02:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: debug filestore = 20 2020-11-20T12:02:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: debug journal = 20 2020-11-20T12:02:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 20 2020-11-20T12:02:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 25 2020-11-20T12:02:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 10 2020-11-20T12:02:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:02:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2020-11-20T12:02:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:02:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:02:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: osd max markdown count = 1000 2020-11-20T12:02:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2020-11-20T12:02:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2020-11-20T12:02:06.588 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.588 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2020-11-20T12:02:06.588 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2020-11-20T12:02:06.588 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2020-11-20T12:02:06.588 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2020-11-20T12:02:06.589 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2020-11-20T12:02:06.589 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:02:06.589 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:02:06.589 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2020-11-20T12:02:06.589 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.590 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2020-11-20T12:02:06.590 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2020-11-20T12:02:06.590 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2020-11-20T12:02:06.590 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2020-11-20T12:02:06.590 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2020-11-20T12:02:06.591 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2020-11-20T12:02:06.591 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2020-11-20T12:02:06.591 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:02:06.591 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2020-11-20T12:02:06.592 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2020-11-20T12:02:06.592 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:02:06.592 INFO:teuthology.orchestra.run.smithi079.stdout: mon op complaint time = 120 2020-11-20T12:02:06.592 INFO:teuthology.orchestra.run.smithi079.stdout: 2020-11-20T12:02:06.592 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2020-11-20T12:02:06.593 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2020-11-20T12:02:06.593 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2020-11-20T12:02:06.593 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2020-11-20T12:02:06.593 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:02:06.594 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:02:06.594 INFO:teuthology.orchestra.run.smithi079.stdout: client mount timeout = 600 2020-11-20T12:02:06.594 INFO:teuthology.orchestra.run.smithi079.stdout: debug client = 20 2020-11-20T12:02:06.594 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2020-11-20T12:02:06.594 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 15m 2020-11-20T12:02:06.595 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 15m 2020-11-20T12:02:06.595 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2020-11-20T12:02:06.595 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2020-11-20T12:02:06.595 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2020-11-20T12:02:06.595 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2020-11-20T12:02:06.596 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2020-11-20T12:02:06.596 INFO:teuthology.orchestra.run.smithi079.stdout:[mds] 2020-11-20T12:02:06.596 INFO:teuthology.orchestra.run.smithi079.stdout: debug mds = 20 2020-11-20T12:02:06.596 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2020-11-20T12:02:06.597 INFO:teuthology.orchestra.run.smithi079.stdout: mds bal frag = True 2020-11-20T12:02:06.597 INFO:teuthology.orchestra.run.smithi079.stdout: mds bal fragment size max = 10000 2020-11-20T12:02:06.597 INFO:teuthology.orchestra.run.smithi079.stdout: mds bal merge size = 5 2020-11-20T12:02:06.597 INFO:teuthology.orchestra.run.smithi079.stdout: mds bal split bits = 3 2020-11-20T12:02:06.598 INFO:teuthology.orchestra.run.smithi079.stdout: mds bal split size = 100 2020-11-20T12:02:06.598 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug frag = True 2020-11-20T12:02:06.598 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug scatterstat = True 2020-11-20T12:02:06.598 INFO:teuthology.orchestra.run.smithi079.stdout: mds op complaint time = 180 2020-11-20T12:02:06.598 INFO:teuthology.orchestra.run.smithi079.stdout: mds verify scatter = True 2020-11-20T12:02:06.599 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2020-11-20T12:02:06.599 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 15m 2020-11-20T12:02:06.599 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 15m 2020-11-20T12:02:06.600 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2020-11-20T12:02:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: chdir = "" 2020-11-20T12:02:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:02:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: auth supported = cephx 2020-11-20T12:02:06.601 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: filestore xattr use omap = true 2020-11-20T12:02:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: mon clock drift allowed = 1.000 2020-11-20T12:02:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.602 INFO:teuthology.orchestra.run.smithi044.stdout: osd crush chooseleaf type = 0 2020-11-20T12:02:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: auth debug = true 2020-11-20T12:02:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on old message = true 2020-11-20T12:02:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on bug = true 2020-11-20T12:02:06.603 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:02:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: mon pg warn max object skew = 0 2020-11-20T12:02:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:02:06.604 INFO:teuthology.orchestra.run.smithi044.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:02:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default size = 2 2020-11-20T12:02:06.605 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow primary affinity = true 2020-11-20T12:02:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow pg remap = true 2020-11-20T12:02:06.606 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:02:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:02:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on no sortbitwise = false 2020-11-20T12:02:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:02:06.607 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on too few osds = false 2020-11-20T12:02:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:02:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:02:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: mon_allow_pool_size_one = true 2020-11-20T12:02:06.608 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:02:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: osd default data pool replay window = 5 2020-11-20T12:02:06.609 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: mon allow pool delete = true 2020-11-20T12:02:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: mon cluster log file level = debug 2020-11-20T12:02:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: debug asserts on shutdown = true 2020-11-20T12:02:06.610 INFO:teuthology.orchestra.run.smithi044.stdout: mon health detail to clog = false 2020-11-20T12:02:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon host = "172.21.15.44,172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790]" 2020-11-20T12:02:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 0437e350-ccfe-424c-9e91-59ff58800c77 2020-11-20T12:02:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.611 INFO:teuthology.orchestra.run.smithi044.stdout:[osd] 2020-11-20T12:02:06.611 INFO:teuthology.orchestra.run.smithi044.stdout: osd journal size = 100 2020-11-20T12:02:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub load threshold = 5.0 2020-11-20T12:02:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub max interval = 600 2020-11-20T12:02:06.612 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd recover clone overlap = true 2020-11-20T12:02:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd recovery max chunk = 1048576 2020-11-20T12:02:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug shutdown = true 2020-11-20T12:02:06.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug op order = true 2020-11-20T12:02:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify stray on activate = true 2020-11-20T12:02:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: osd open classes on start = true 2020-11-20T12:02:06.614 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug pg log writeout = true 2020-11-20T12:02:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:02:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.615 INFO:teuthology.orchestra.run.smithi044.stdout: osd map max advance = 10 2020-11-20T12:02:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: journal zero on create = true 2020-11-20T12:02:06.616 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:02:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: filestore apply finisher threads = 3 2020-11-20T12:02:06.617 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: bdev debug aio = true 2020-11-20T12:02:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug misdirected ops = true 2020-11-20T12:02:06.618 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore block size = 96636764160 2020-11-20T12:02:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore compression mode = aggressive 2020-11-20T12:02:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore fsck on mount = True 2020-11-20T12:02:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluefs = 20 2020-11-20T12:02:06.619 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluestore = 20 2020-11-20T12:02:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: debug filestore = 20 2020-11-20T12:02:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: debug journal = 20 2020-11-20T12:02:06.620 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 20 2020-11-20T12:02:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: debug osd = 25 2020-11-20T12:02:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: debug rocksdb = 10 2020-11-20T12:02:06.621 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:02:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd full ratio = 0.9 2020-11-20T12:02:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:02:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:02:06.622 INFO:teuthology.orchestra.run.smithi044.stdout: osd max markdown count = 1000 2020-11-20T12:02:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: osd objectstore = bluestore 2020-11-20T12:02:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: osd op complaint time = 180 2020-11-20T12:02:06.623 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.623 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr] 2020-11-20T12:02:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2020-11-20T12:02:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: debug mgr = 20 2020-11-20T12:02:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2020-11-20T12:02:06.624 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2020-11-20T12:02:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:02:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:02:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: mgr/telemetry/nag = false 2020-11-20T12:02:06.625 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.626 INFO:teuthology.orchestra.run.smithi044.stdout:[mon] 2020-11-20T12:02:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2020-11-20T12:02:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2020-11-20T12:02:06.626 INFO:teuthology.orchestra.run.smithi044.stdout: debug paxos = 20 2020-11-20T12:02:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2020-11-20T12:02:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: mon data avail warn = 5 2020-11-20T12:02:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: mon mgr mkfs grace = 240 2020-11-20T12:02:06.627 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:02:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd reporter subtree level = osd 2020-11-20T12:02:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd prime pg temp = true 2020-11-20T12:02:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:02:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: mon op complaint time = 120 2020-11-20T12:02:06.628 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:06.629 INFO:teuthology.orchestra.run.smithi044.stdout:[client] 2020-11-20T12:02:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: rgw cache enabled = true 2020-11-20T12:02:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable ops log = true 2020-11-20T12:02:06.629 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable usage log = true 2020-11-20T12:02:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:02:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:02:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: client mount timeout = 600 2020-11-20T12:02:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: debug client = 20 2020-11-20T12:02:06.630 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2020-11-20T12:02:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: rados mon op timeout = 15m 2020-11-20T12:02:06.631 INFO:teuthology.orchestra.run.smithi044.stdout: rados osd op timeout = 15m 2020-11-20T12:02:06.631 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.a] 2020-11-20T12:02:06.631 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.b] 2020-11-20T12:02:06.632 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.c] 2020-11-20T12:02:06.632 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2020-11-20T12:02:06.632 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2020-11-20T12:02:06.632 INFO:teuthology.orchestra.run.smithi044.stdout:[mds] 2020-11-20T12:02:06.632 INFO:teuthology.orchestra.run.smithi044.stdout: debug mds = 20 2020-11-20T12:02:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2020-11-20T12:02:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal frag = True 2020-11-20T12:02:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal fragment size max = 10000 2020-11-20T12:02:06.633 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal merge size = 5 2020-11-20T12:02:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal split bits = 3 2020-11-20T12:02:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal split size = 100 2020-11-20T12:02:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: mds debug frag = True 2020-11-20T12:02:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: mds debug scatterstat = True 2020-11-20T12:02:06.634 INFO:teuthology.orchestra.run.smithi044.stdout: mds op complaint time = 180 2020-11-20T12:02:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: mds verify scatter = True 2020-11-20T12:02:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: osd op complaint time = 180 2020-11-20T12:02:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: rados mon op timeout = 15m 2020-11-20T12:02:06.635 INFO:teuthology.orchestra.run.smithi044.stdout: rados osd op timeout = 15m 2020-11-20T12:02:06.636 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-20T12:02:06.636 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-20T12:02:06.693 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-20T12:02:06.693 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:06.694 INFO:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-20T12:02:06.707 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-20T12:02:06.708 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.] 2020-11-20T12:02:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQA+sLdf9XoXDhAAwpEzjI4JV81eVy86yTQcRw== 2020-11-20T12:02:06.709 INFO:teuthology.orchestra.run.smithi044.stdout:[client.admin] 2020-11-20T12:02:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQA+sLdfvmsLKRAA71gHfnJLMWjM413P9DHBcw== 2020-11-20T12:02:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: caps mds = "allow *" 2020-11-20T12:02:06.709 INFO:teuthology.orchestra.run.smithi044.stdout: caps mgr = "allow *" 2020-11-20T12:02:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: caps mon = "allow *" 2020-11-20T12:02:06.710 INFO:teuthology.orchestra.run.smithi044.stdout: caps osd = "allow *" 2020-11-20T12:02:06.710 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:06.710 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:06.711 INFO:teuthology.orchestra.run.smithi044.stderr:206 bytes copied, 3.3892e-05 s, 6.1 MB/s 2020-11-20T12:02:06.711 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:06.711 INFO:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-20T12:02:06.761 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-20T12:02:06.763 INFO:teuthology.orchestra.run.smithi044.stdout: �7�P��BL��Y�X� w>��_^>��_^�aXa �,��,bXb �O��OcXc �O��Oabc 2020-11-20T12:02:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:06.764 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:06.765 INFO:teuthology.orchestra.run.smithi044.stderr:409 bytes copied, 3.1707e-05 s, 12.9 MB/s 2020-11-20T12:02:06.765 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi044.front.sepia.ceph.com 2020-11-20T12:02:06.766 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:06.767 INFO:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:02:06.767 INFO:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:02:06.840 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:02:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:06.888 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:06.888 INFO:teuthology.orchestra.run.smithi044.stderr:206 bytes copied, 0.0307334 s, 6.7 kB/s 2020-11-20T12:02:06.889 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:02:06.905 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:06.905 INFO:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:06.975 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:08.309 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:08.309 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:08.309 INFO:teuthology.orchestra.run.smithi044.stderr:409 bytes copied, 0.00320614 s, 128 kB/s 2020-11-20T12:02:08.309 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2020-11-20T12:02:08.310 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:08.310 INFO:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:02:08.311 INFO:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:02:08.332 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:02:08.347 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:08.347 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:08.347 INFO:teuthology.orchestra.run.smithi079.stderr:206 bytes copied, 3.8415e-05 s, 5.4 MB/s 2020-11-20T12:02:08.348 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:02:08.364 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:08.365 INFO:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:08.425 INFO:teuthology.orchestra.run.smithi079.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:08.427 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:08.428 INFO:teuthology.orchestra.run.smithi079.stderr:409 bytes copied, 0.00171564 s, 238 kB/s 2020-11-20T12:02:08.428 INFO:tasks.ceph:Setting up mon nodes... 2020-11-20T12:02:08.428 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-20T12:02:08.429 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T12:02:08.504 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T12:02:08.506 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T12:02:08.576 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T12:02:08.578 INFO:tasks.ceph:Setting up mds nodes... 2020-11-20T12:02:08.578 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T12:02:08.651 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T12:02:08.653 INFO:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-20T12:02:08.689 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T12:02:08.748 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T12:02:08.751 INFO:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-11-20T12:02:08.782 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-20T12:02:08.783 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-20T12:02:08.829 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-20T12:02:08.848 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-20T12:02:08.848 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T12:02:08.849 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-20T12:02:08.881 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:08.882 INFO:tasks.ceph:role: osd.0 2020-11-20T12:02:08.882 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com 2020-11-20T12:02:08.882 INFO:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T12:02:09.404 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:09.404 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:09.405 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:09.405 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 2020-11-20T12:02:09.405 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:09.405 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:09.406 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:09.406 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:09.406 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:09.406 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:09.408 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2020-11-20T12:02:09.408 INFO:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-20T12:02:09.512 INFO:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-20T12:02:09.548 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-20T12:02:09.621 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:09.621 INFO:tasks.ceph:role: osd.1 2020-11-20T12:02:09.621 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com 2020-11-20T12:02:09.622 INFO:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T12:02:10.147 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 2020-11-20T12:02:10.148 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:10.148 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:10.149 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:10.149 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:10.150 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2020-11-20T12:02:10.150 INFO:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-20T12:02:10.204 INFO:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-20T12:02:10.240 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-20T12:02:10.312 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:10.312 INFO:tasks.ceph:role: osd.2 2020-11-20T12:02:10.313 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com 2020-11-20T12:02:10.313 INFO:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T12:02:10.836 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 2020-11-20T12:02:10.837 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:10.838 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:10.838 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:10.839 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:10.839 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2020-11-20T12:02:10.840 INFO:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-20T12:02:10.913 INFO:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-20T12:02:10.949 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-20T12:02:11.021 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:11.022 INFO:tasks.ceph:role: osd.3 2020-11-20T12:02:11.022 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com 2020-11-20T12:02:11.022 INFO:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T12:02:11.555 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:11.555 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:11.555 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:11.555 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 2020-11-20T12:02:11.556 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:11.556 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:11.556 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:11.556 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:11.556 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:11.557 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:11.558 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2020-11-20T12:02:11.558 INFO:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-20T12:02:11.667 INFO:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-20T12:02:11.706 INFO:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:11.794+0000 7f421935aec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-20T12:02:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:11.794+0000 7f421935aec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-20T12:02:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:11.794+0000 7f421935aec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T12:02:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:11.794+0000 7f421935aec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T12:02:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:11.794+0000 7f421935aec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T12:02:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:11.796+0000 7f421935aec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-20T12:02:14.656 INFO:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-20T12:02:14.698 INFO:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:14.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:14.789+0000 7fb404242ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-20T12:02:14.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:14.789+0000 7fb404242ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-20T12:02:14.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:14.789+0000 7fb404242ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T12:02:14.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:14.789+0000 7fb404242ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T12:02:14.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:14.789+0000 7fb404242ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T12:02:14.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:14.790+0000 7fb404242ec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-20T12:02:17.642 INFO:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-20T12:02:17.679 INFO:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:17.769+0000 7f6b152f5ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-20T12:02:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:17.769+0000 7f6b152f5ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-20T12:02:17.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:17.769+0000 7f6b152f5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T12:02:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:17.769+0000 7f6b152f5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T12:02:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:17.769+0000 7f6b152f5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T12:02:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:17.769+0000 7f6b152f5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-20T12:02:20.616 INFO:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-20T12:02:20.653 INFO:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:20.742+0000 7ff1ce4adec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-20T12:02:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:20.742+0000 7ff1ce4adec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-20T12:02:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:20.742+0000 7ff1ce4adec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T12:02:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:20.742+0000 7ff1ce4adec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T12:02:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:20.743+0000 7ff1ce4adec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T12:02:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:20.743+0000 7ff1ce4adec0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-20T12:02:23.595 INFO:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-20T12:02:23.631 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-20T12:02:23.664 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:23.665 INFO:tasks.ceph:role: osd.4 2020-11-20T12:02:23.665 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2020-11-20T12:02:23.666 INFO:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T12:02:24.202 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:24.202 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:24.203 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:24.203 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2020-11-20T12:02:24.203 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:24.203 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:24.204 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:24.204 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:24.204 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:24.204 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:24.205 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2020-11-20T12:02:24.205 INFO:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-20T12:02:24.316 INFO:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-20T12:02:24.355 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-20T12:02:24.425 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:24.426 INFO:tasks.ceph:role: osd.5 2020-11-20T12:02:24.426 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2020-11-20T12:02:24.426 INFO:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T12:02:24.936 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:24.936 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2020-11-20T12:02:24.937 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:24.937 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:24.937 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:24.937 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:24.938 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:24.938 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:24.939 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2020-11-20T12:02:24.939 INFO:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-20T12:02:24.991 INFO:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-20T12:02:25.022 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-20T12:02:25.090 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:25.090 INFO:tasks.ceph:role: osd.6 2020-11-20T12:02:25.091 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2020-11-20T12:02:25.091 INFO:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T12:02:25.620 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:25.621 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:25.621 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:25.621 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2020-11-20T12:02:25.621 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:25.622 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:25.622 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:25.622 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:25.622 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:25.622 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:25.623 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2020-11-20T12:02:25.624 INFO:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-20T12:02:25.695 INFO:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-20T12:02:25.728 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-20T12:02:25.799 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:02:25.799 INFO:tasks.ceph:role: osd.7 2020-11-20T12:02:25.800 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com 2020-11-20T12:02:25.800 INFO:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T12:02:26.330 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:02:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:02:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T12:02:26.331 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2020-11-20T12:02:26.331 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:02:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:02:26.332 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T12:02:26.332 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:02:26.332 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:02:26.332 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:02:26.333 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2020-11-20T12:02:26.334 INFO:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-20T12:02:26.433 INFO:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-20T12:02:26.464 INFO:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:26.547 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:26.544+0000 7fd316eb5ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-20T12:02:26.548 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:26.545+0000 7fd316eb5ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-20T12:02:26.548 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:26.545+0000 7fd316eb5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T12:02:26.549 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:26.545+0000 7fd316eb5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T12:02:26.549 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:26.545+0000 7fd316eb5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T12:02:26.556 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:26.552+0000 7fd316eb5ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-20T12:02:29.411 INFO:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-20T12:02:29.493 INFO:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:29.587 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:29.583+0000 7fd3352f2ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-20T12:02:29.588 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:29.583+0000 7fd3352f2ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-20T12:02:29.589 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:29.583+0000 7fd3352f2ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T12:02:29.589 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:29.583+0000 7fd3352f2ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T12:02:29.590 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:29.583+0000 7fd3352f2ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T12:02:29.590 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:29.584+0000 7fd3352f2ec0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-20T12:02:32.437 INFO:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-20T12:02:32.469 INFO:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:32.559 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:32.556+0000 7fd713fb3ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-20T12:02:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:32.556+0000 7fd713fb3ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-20T12:02:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:32.556+0000 7fd713fb3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T12:02:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:32.556+0000 7fd713fb3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T12:02:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:32.556+0000 7fd713fb3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T12:02:32.561 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:32.557+0000 7fd713fb3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-20T12:02:35.278 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:02:35.283 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:02:35.400 INFO:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-20T12:02:35.431 INFO:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:35.519 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:35.516+0000 7f17656deec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-20T12:02:35.520 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:35.516+0000 7f17656deec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-20T12:02:35.520 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:35.516+0000 7f17656deec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T12:02:35.520 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:35.516+0000 7f17656deec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T12:02:35.520 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:35.516+0000 7f17656deec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T12:02:35.521 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:35.517+0000 7f17656deec0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-20T12:02:38.363 INFO:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-20T12:02:38.399 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-20T12:02:38.399 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:38.400 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-20T12:02:38.418 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-20T12:02:38.438 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.y] 2020-11-20T12:02:38.438 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdfcxHsHRAAW/i0+X0omJCglqCJmLIa9A== 2020-11-20T12:02:38.439 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:38.439 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:38.439 INFO:teuthology.orchestra.run.smithi044.stderr:56 bytes copied, 4.6936e-05 s, 1.2 MB/s 2020-11-20T12:02:38.440 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:38.441 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-20T12:02:38.494 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-20T12:02:38.511 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2020-11-20T12:02:38.512 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdffj2oJhAAVqusvgAHrVD1Sci8tw+2Ng== 2020-11-20T12:02:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:38.512 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:38.513 INFO:teuthology.orchestra.run.smithi044.stderr:56 bytes copied, 4.6133e-05 s, 1.2 MB/s 2020-11-20T12:02:38.514 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:38.514 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-20T12:02:38.568 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-20T12:02:38.586 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2020-11-20T12:02:38.586 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBDsLdf3+9lLxAAapbKScjWLGXP4M3C9232CQ== 2020-11-20T12:02:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:56 bytes copied, 3.7442e-05 s, 1.5 MB/s 2020-11-20T12:02:38.588 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:38.588 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-20T12:02:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-20T12:02:38.660 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2020-11-20T12:02:38.661 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBGsLdfqZUeLxAAsU1E/gd0Vm3I8yDUMoODOw== 2020-11-20T12:02:38.661 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:38.661 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:38.661 INFO:teuthology.orchestra.run.smithi044.stderr:56 bytes copied, 4.2844e-05 s, 1.3 MB/s 2020-11-20T12:02:38.662 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:38.663 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-20T12:02:38.716 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-20T12:02:38.734 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2020-11-20T12:02:38.734 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBJsLdfvrPnLRAAFlru27Qy0BwMOhzcpsg7Jw== 2020-11-20T12:02:38.734 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:38.735 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:38.735 INFO:teuthology.orchestra.run.smithi044.stderr:56 bytes copied, 3.483e-05 s, 1.6 MB/s 2020-11-20T12:02:38.736 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:38.736 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-20T12:02:38.791 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-20T12:02:38.808 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2020-11-20T12:02:38.808 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBMsLdfdElULBAAg29UFQTWSl5lLHNa+5XayA== 2020-11-20T12:02:38.809 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:38.809 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:38.810 INFO:teuthology.orchestra.run.smithi044.stderr:56 bytes copied, 3.6739e-05 s, 1.5 MB/s 2020-11-20T12:02:38.811 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:38.812 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-20T12:02:38.828 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-20T12:02:38.843 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2020-11-20T12:02:38.844 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBAsLdfgIcvIhAAyTDw9xXVvbU8yinUGRDVtw== 2020-11-20T12:02:38.844 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:38.844 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:38.844 INFO:teuthology.orchestra.run.smithi079.stderr:56 bytes copied, 4.5104e-05 s, 1.2 MB/s 2020-11-20T12:02:38.845 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:38.846 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-20T12:02:38.901 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-20T12:02:38.916 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2020-11-20T12:02:38.916 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBAsLdfRRh2LBAAqwM9bROpMJQxFcgMOt9ZaA== 2020-11-20T12:02:38.916 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:38.917 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:38.917 INFO:teuthology.orchestra.run.smithi079.stderr:56 bytes copied, 4.0171e-05 s, 1.4 MB/s 2020-11-20T12:02:38.918 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:38.918 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-20T12:02:38.973 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-20T12:02:38.986 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2020-11-20T12:02:38.986 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBSsLdfOKl/IBAABfgAEFvdelLnBpRxkf36yg== 2020-11-20T12:02:38.987 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:38.987 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:38.987 INFO:teuthology.orchestra.run.smithi079.stderr:56 bytes copied, 3.0091e-05 s, 1.9 MB/s 2020-11-20T12:02:38.988 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:38.988 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-20T12:02:39.042 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-20T12:02:39.056 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2020-11-20T12:02:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBVsLdfO/DSIhAAgsf7hyARZFAgJR0OAQZDrQ== 2020-11-20T12:02:39.057 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:39.057 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:39.057 INFO:teuthology.orchestra.run.smithi079.stderr:56 bytes copied, 3.2136e-05 s, 1.7 MB/s 2020-11-20T12:02:39.058 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:39.059 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-20T12:02:39.112 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-20T12:02:39.126 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.6] 2020-11-20T12:02:39.127 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBYsLdfVqEyIRAAanCfDtqENZ0n1nZ+mA6u2Q== 2020-11-20T12:02:39.127 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:39.127 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:39.128 INFO:teuthology.orchestra.run.smithi079.stderr:56 bytes copied, 3.2549e-05 s, 1.7 MB/s 2020-11-20T12:02:39.128 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:39.128 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-20T12:02:39.181 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-20T12:02:39.194 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.7] 2020-11-20T12:02:39.194 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBbsLdfAkbOHhAAXjmM3sXd/MvorcrqsbYXhw== 2020-11-20T12:02:39.195 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:39.195 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:39.195 INFO:teuthology.orchestra.run.smithi079.stderr:56 bytes copied, 3.2186e-05 s, 1.7 MB/s 2020-11-20T12:02:39.196 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:39.197 INFO:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-20T12:02:39.212 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-20T12:02:39.214 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2020-11-20T12:02:39.214 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdfWXJGMRAA/UfhmUr+jNLyneEY5/oBVA== 2020-11-20T12:02:39.214 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:39.214 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:39.215 INFO:teuthology.orchestra.run.smithi044.stderr:59 bytes copied, 4.2974e-05 s, 1.4 MB/s 2020-11-20T12:02:39.215 INFO:tasks.ceph:Adding keys to all mons... 2020-11-20T12:02:39.215 INFO:teuthology.orchestra.run.smithi044:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T12:02:39.257 INFO:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T12:02:39.284 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.y] 2020-11-20T12:02:39.284 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBAsLdfcxHsHRAAW/i0+X0omJCglqCJmLIa9A== 2020-11-20T12:02:39.284 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2020-11-20T12:02:39.284 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBAsLdffj2oJhAAVqusvgAHrVD1Sci8tw+2Ng== 2020-11-20T12:02:39.285 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2020-11-20T12:02:39.285 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBDsLdf3+9lLxAAapbKScjWLGXP4M3C9232CQ== 2020-11-20T12:02:39.285 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2020-11-20T12:02:39.285 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBGsLdfqZUeLxAAsU1E/gd0Vm3I8yDUMoODOw== 2020-11-20T12:02:39.286 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2020-11-20T12:02:39.286 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBJsLdfvrPnLRAAFlru27Qy0BwMOhzcpsg7Jw== 2020-11-20T12:02:39.286 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2020-11-20T12:02:39.286 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMsLdfdElULBAAg29UFQTWSl5lLHNa+5XayA== 2020-11-20T12:02:39.287 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2020-11-20T12:02:39.287 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBAsLdfgIcvIhAAyTDw9xXVvbU8yinUGRDVtw== 2020-11-20T12:02:39.287 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2020-11-20T12:02:39.287 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBAsLdfRRh2LBAAqwM9bROpMJQxFcgMOt9ZaA== 2020-11-20T12:02:39.287 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2020-11-20T12:02:39.288 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBSsLdfOKl/IBAABfgAEFvdelLnBpRxkf36yg== 2020-11-20T12:02:39.288 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2020-11-20T12:02:39.288 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBVsLdfO/DSIhAAgsf7hyARZFAgJR0OAQZDrQ== 2020-11-20T12:02:39.288 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.6] 2020-11-20T12:02:39.288 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBYsLdfVqEyIRAAanCfDtqENZ0n1nZ+mA6u2Q== 2020-11-20T12:02:39.289 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.7] 2020-11-20T12:02:39.289 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBbsLdfAkbOHhAAXjmM3sXd/MvorcrqsbYXhw== 2020-11-20T12:02:39.289 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2020-11-20T12:02:39.289 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBAsLdfWXJGMRAA/UfhmUr+jNLyneEY5/oBVA== 2020-11-20T12:02:39.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.y] 2020-11-20T12:02:39.291 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdfcxHsHRAAW/i0+X0omJCglqCJmLIa9A== 2020-11-20T12:02:39.291 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2020-11-20T12:02:39.291 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdffj2oJhAAVqusvgAHrVD1Sci8tw+2Ng== 2020-11-20T12:02:39.292 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2020-11-20T12:02:39.292 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBDsLdf3+9lLxAAapbKScjWLGXP4M3C9232CQ== 2020-11-20T12:02:39.292 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2020-11-20T12:02:39.292 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBGsLdfqZUeLxAAsU1E/gd0Vm3I8yDUMoODOw== 2020-11-20T12:02:39.292 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2020-11-20T12:02:39.293 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBJsLdfvrPnLRAAFlru27Qy0BwMOhzcpsg7Jw== 2020-11-20T12:02:39.293 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2020-11-20T12:02:39.293 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBMsLdfdElULBAAg29UFQTWSl5lLHNa+5XayA== 2020-11-20T12:02:39.293 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.x] 2020-11-20T12:02:39.293 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdfgIcvIhAAyTDw9xXVvbU8yinUGRDVtw== 2020-11-20T12:02:39.294 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2020-11-20T12:02:39.294 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdfRRh2LBAAqwM9bROpMJQxFcgMOt9ZaA== 2020-11-20T12:02:39.294 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.4] 2020-11-20T12:02:39.294 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBSsLdfOKl/IBAABfgAEFvdelLnBpRxkf36yg== 2020-11-20T12:02:39.294 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.5] 2020-11-20T12:02:39.295 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBVsLdfO/DSIhAAgsf7hyARZFAgJR0OAQZDrQ== 2020-11-20T12:02:39.295 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.6] 2020-11-20T12:02:39.295 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBYsLdfVqEyIRAAanCfDtqENZ0n1nZ+mA6u2Q== 2020-11-20T12:02:39.295 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.7] 2020-11-20T12:02:39.296 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBbsLdfAkbOHhAAXjmM3sXd/MvorcrqsbYXhw== 2020-11-20T12:02:39.296 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2020-11-20T12:02:39.296 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBAsLdfWXJGMRAA/UfhmUr+jNLyneEY5/oBVA== 2020-11-20T12:02:39.296 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:02:39.333 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:02:39.387 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:02:39.389 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:02:39.444 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.448 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.498 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.502 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.552 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.555 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.605 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.608 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.658 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:02:39.665 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:02:39.716 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:02:39.725 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:02:39.781 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.788 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.840 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.843 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.930 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.933 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.983 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:39.985 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:02:40.035 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:02:40.038 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:02:40.091 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-20T12:02:40.091 INFO:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-20T12:02:40.122 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T12:02:40.220 INFO:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-20T12:02:40.250 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-20T12:02:40.278 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T12:02:40.374 INFO:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-20T12:02:40.401 INFO:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-20T12:02:40.480 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T12:02:40.592 INFO:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-20T12:02:40.624 INFO:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:40.642 INFO:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:02:40.682 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-20T12:02:40.683 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:02:40.683 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:02:40.708 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:02:40.708 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:02:40.709 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:02:40.733 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:02:40.733 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-20T12:02:40.734 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-20T12:02:40.743 INFO:tasks.ceph.mon.c:Started 2020-11-20T12:02:40.743 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-20T12:02:40.743 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-20T12:02:40.744 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-20T12:02:40.752 INFO:tasks.ceph.mgr.y:Started 2020-11-20T12:02:40.753 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-20T12:02:40.753 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-20T12:02:40.759 INFO:tasks.ceph.mgr.x:Started 2020-11-20T12:02:40.760 INFO:tasks.ceph:Setting crush tunables to default 2020-11-20T12:02:40.760 INFO:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd crush tunables default 2020-11-20T12:02:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.910+0000 7fa17be70700 1 Processor -- start 2020-11-20T12:02:40.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa17be70700 1 -- start start 2020-11-20T12:02:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa17be70700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa17413b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa17be70700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17413abb0 con 0x7fa1740ab050 2020-11-20T12:02:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa17be70700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17413ae50 con 0x7fa1740abbf0 2020-11-20T12:02:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa17be70700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740a8770 con 0x7fa1740ac9f0 2020-11-20T12:02:40.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa179c0c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa1740ab050 0x7fa1740a9410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39232/0 (socket says 172.21.15.44:39232) 2020-11-20T12:02:40.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa179c0c700 1 -- 172.21.15.44:0/1584832630 learned_addr learned my addr 172.21.15.44:0/1584832630 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:02:40.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.911+0000 7fa17a40d700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa17413b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:40.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.912+0000 7fa17a40d700 1 -- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 msgr2=0x7fa17413b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:02:40.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.912+0000 7fa17a40d700 1 -- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 msgr2=0x7fa17413b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:02:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.912+0000 7fa17a40d700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa17413b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:02:40.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:40.912+0000 7fa17a40d700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa17413b590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:02:41.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.053+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311974807 0 0) 0x7fa17413abb0 con 0x7fa1740ab050 2020-11-20T12:02:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.053+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa160003240 con 0x7fa1740ab050 2020-11-20T12:02:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.053+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498720071 0 0) 0x7fa17413ae50 con 0x7fa1740abbf0 2020-11-20T12:02:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.053+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa17413abb0 con 0x7fa1740abbf0 2020-11-20T12:02:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.053+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 645367787 0 0) 0x7fa160003240 con 0x7fa1740ab050 2020-11-20T12:02:41.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.053+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 msgr2=0x7fa17413b590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:02:41.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa17413b590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 >> v1:172.21.15.79:6789/0 conn(0x7fa1740abbf0 legacy=0x7fa1740ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17413b0f0 con 0x7fa1740ab050 2020-11-20T12:02:41.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fa168001660 con 0x7fa1740ab050 2020-11-20T12:02:41.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 >> v1:172.21.15.44:6789/0 conn(0x7fa1740ab050 legacy=0x7fa1740a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1600038f0 0x7fa160003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1600044e0 0x7fa160004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa160004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160005340 con 0x7fa1600038f0 2020-11-20T12:02:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16000b730 con 0x7fa1600044e0 2020-11-20T12:02:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16000b8a0 con 0x7fa1740ac9f0 2020-11-20T12:02:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa179c0c700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1600038f0 0x7fa160003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa17a40d700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa160004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa17940b700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1600044e0 0x7fa160004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa179c0c700 1 -- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 msgr2=0x7fa160004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa179c0c700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1740ac9f0 0x7fa160004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa179c0c700 1 -- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1600044e0 msgr2=0x7fa160004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa179c0c700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1600044e0 0x7fa160004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa179c0c700 1 -- 172.21.15.44:0/1584832630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160003240 con 0x7fa1600038f0 2020-11-20T12:02:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa17940b700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1600044e0 0x7fa160004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:02:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa179c0c700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1600038f0 0x7fa160003cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fa16800c420 tx=0x7fa1740aae20).ready entity=mon.0 client_cookie=96296da29b1744b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680070d0 con 0x7fa1600038f0 2020-11-20T12:02:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa168007230 con 0x7fa1600038f0 2020-11-20T12:02:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.054+0000 7fa178c0a700 1 -- 172.21.15.44:0/1584832630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168005710 con 0x7fa1600038f0 2020-11-20T12:02:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.055+0000 7fa17be70700 1 -- 172.21.15.44:0/1584832630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1600038f0 msgr2=0x7fa160003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.055+0000 7fa17be70700 1 --2- 172.21.15.44:0/1584832630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1600038f0 0x7fa160003cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fa16800c420 tx=0x7fa1740aae20).stop 2020-11-20T12:02:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.055+0000 7fa17a40d700 1 -- 172.21.15.44:0/1584832630 reap_dead start 2020-11-20T12:02:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.055+0000 7fa17be70700 1 -- 172.21.15.44:0/1584832630 shutdown_connections 2020-11-20T12:02:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.055+0000 7fa17be70700 1 -- 172.21.15.44:0/1584832630 >> 172.21.15.44:0/1584832630 conn(0x7fa174060170 msgr2=0x7fa174060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.056+0000 7fa17be70700 1 -- 172.21.15.44:0/1584832630 shutdown_connections 2020-11-20T12:02:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.056+0000 7fa17be70700 1 -- 172.21.15.44:0/1584832630 wait complete. 2020-11-20T12:02:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.057+0000 7fa17be70700 1 Processor -- start 2020-11-20T12:02:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.057+0000 7fa17be70700 1 -- start start 2020-11-20T12:02:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17be70700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1600038f0 0x7fa17413dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17be70700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1740a75b0 0x7fa1740a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17be70700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa17409ecc0 0x7fa1740a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17be70700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740a7000 con 0x7fa1740a75b0 2020-11-20T12:02:41.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17be70700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740a7ac0 con 0x7fa17409ecc0 2020-11-20T12:02:41.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17be70700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740a7d60 con 0x7fa1600038f0 2020-11-20T12:02:41.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1740a75b0 0x7fa1740a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1740a75b0 0x7fa1740a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35512/0 (socket says 172.21.15.44:35512) 2020-11-20T12:02:41.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa179c0c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1600038f0 0x7fa17413dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 -- 172.21.15.44:0/3184116859 learned_addr learned my addr 172.21.15.44:0/3184116859 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17a40d700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa17409ecc0 0x7fa1740a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 -- 172.21.15.44:0/3184116859 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1600038f0 msgr2=0x7fa17413dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1600038f0 0x7fa17413dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 -- 172.21.15.44:0/3184116859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa17409ecc0 msgr2=0x7fa1740a6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa17409ecc0 0x7fa1740a6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 -- 172.21.15.44:0/3184116859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740a8000 con 0x7fa1740a75b0 2020-11-20T12:02:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.058+0000 7fa17940b700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1740a75b0 0x7fa1740a6620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fa16400ee90 tx=0x7fa164009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.059+0000 7fa171ffb700 1 -- 172.21.15.44:0/3184116859 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164003320 con 0x7fa1740a75b0 2020-11-20T12:02:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.059+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17413e900 con 0x7fa1740a75b0 2020-11-20T12:02:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.060+0000 7fa171ffb700 1 -- 172.21.15.44:0/3184116859 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164003480 con 0x7fa1740a75b0 2020-11-20T12:02:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.060+0000 7fa171ffb700 1 -- 172.21.15.44:0/3184116859 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16400cb20 con 0x7fa1740a75b0 2020-11-20T12:02:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.060+0000 7fa171ffb700 1 -- 172.21.15.44:0/3184116859 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fa164006ad0 con 0x7fa1740a75b0 2020-11-20T12:02:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.060+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1741375a0 con 0x7fa1740a75b0 2020-11-20T12:02:41.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.060+0000 7fa171ffb700 1 -- 172.21.15.44:0/3184116859 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fa164007200 con 0x7fa1740a75b0 2020-11-20T12:02:41.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.061+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1740a6490 con 0x7fa1740a75b0 2020-11-20T12:02:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.064+0000 7fa171ffb700 1 -- 172.21.15.44:0/3184116859 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fa164007480 con 0x7fa1740a75b0 2020-11-20T12:02:41.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.148+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa1740696f0 con 0x7fa1740a75b0 2020-11-20T12:02:41.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.151+0000 7fa171ffb700 1 -- 172.21.15.44:0/3184116859 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa164007620 con 0x7fa1740a75b0 2020-11-20T12:02:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1740a75b0 msgr2=0x7fa1740a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1740a75b0 0x7fa1740a6620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fa16400ee90 tx=0x7fa164009040).stop 2020-11-20T12:02:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 shutdown_connections 2020-11-20T12:02:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa17409ecc0 0x7fa1740a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1600038f0 0x7fa17413dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 --2- 172.21.15.44:0/3184116859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1740a75b0 0x7fa1740a6620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 >> 172.21.15.44:0/3184116859 conn(0x7fa174060170 msgr2=0x7fa1740a8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:41.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 shutdown_connections 2020-11-20T12:02:41.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:41.152+0000 7fa17be70700 1 -- 172.21.15.44:0/3184116859 wait complete. 2020-11-20T12:02:41.158 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2020-11-20T12:02:41.167 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-20T12:02:41.168 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:41.168 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-20T12:02:41.188 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-20T12:02:41.205 INFO:teuthology.orchestra.run.smithi044.stdout:2856c45e-54ed-4130-bd9f-898ffadae60a 2020-11-20T12:02:41.205 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:41.206 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:41.206 INFO:teuthology.orchestra.run.smithi044.stderr:37 bytes copied, 3.7525e-05 s, 986 kB/s 2020-11-20T12:02:41.207 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:41.207 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-20T12:02:41.226 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-20T12:02:41.242 INFO:teuthology.orchestra.run.smithi044.stdout:9af002f0-d9d2-4362-8855-73c63682dc0d 2020-11-20T12:02:41.243 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:41.243 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:41.243 INFO:teuthology.orchestra.run.smithi044.stderr:37 bytes copied, 3.8007e-05 s, 974 kB/s 2020-11-20T12:02:41.244 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:41.245 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-20T12:02:41.307 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-20T12:02:41.323 INFO:teuthology.orchestra.run.smithi044.stdout:475fc90c-4230-4093-9438-34ba65bdfdde 2020-11-20T12:02:41.323 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:41.324 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:41.324 INFO:teuthology.orchestra.run.smithi044.stderr:37 bytes copied, 3.5091e-05 s, 1.1 MB/s 2020-11-20T12:02:41.325 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:02:41.325 INFO:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-20T12:02:41.380 INFO:teuthology.orchestra.run.smithi044.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-20T12:02:41.397 INFO:teuthology.orchestra.run.smithi044.stdout:f951d29a-4ead-4d15-8e19-178a8eef2b91 2020-11-20T12:02:41.397 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2020-11-20T12:02:41.397 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2020-11-20T12:02:41.398 INFO:teuthology.orchestra.run.smithi044.stderr:37 bytes copied, 3.7005e-05 s, 1000 kB/s 2020-11-20T12:02:41.399 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:41.399 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-20T12:02:41.420 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-20T12:02:41.432 INFO:teuthology.orchestra.run.smithi079.stdout:74c35433-a1af-4165-ba72-6935b43df900 2020-11-20T12:02:41.432 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:41.433 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:41.433 INFO:teuthology.orchestra.run.smithi079.stderr:37 bytes copied, 3.1889e-05 s, 1.2 MB/s 2020-11-20T12:02:41.434 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:41.435 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-20T12:02:41.494 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-20T12:02:41.507 INFO:teuthology.orchestra.run.smithi079.stdout:9495ddb8-00b5-4d73-acb2-86bf2949700f 2020-11-20T12:02:41.508 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:41.508 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:41.508 INFO:teuthology.orchestra.run.smithi079.stderr:37 bytes copied, 4.4198e-05 s, 837 kB/s 2020-11-20T12:02:41.510 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:41.510 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-20T12:02:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-20T12:02:41.581 INFO:teuthology.orchestra.run.smithi079.stdout:6502bfc3-d91f-407c-a2e7-83bf8472704a 2020-11-20T12:02:41.581 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:41.582 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:41.582 INFO:teuthology.orchestra.run.smithi079.stderr:37 bytes copied, 3.6785e-05 s, 1.0 MB/s 2020-11-20T12:02:41.583 INFO:teuthology.orchestra.run.smithi079:> set -ex 2020-11-20T12:02:41.583 INFO:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-20T12:02:41.644 INFO:teuthology.orchestra.run.smithi079.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-20T12:02:41.657 INFO:teuthology.orchestra.run.smithi079.stdout:7d22fe22-d60a-468f-b9c6-bfe4d9f50c37 2020-11-20T12:02:41.657 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2020-11-20T12:02:41.658 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2020-11-20T12:02:41.658 INFO:teuthology.orchestra.run.smithi079.stderr:37 bytes copied, 3.1935e-05 s, 1.2 MB/s 2020-11-20T12:02:41.659 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 2856c45e-54ed-4130-bd9f-898ffadae60a 0 2020-11-20T12:02:41.824 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.820+0000 7f3533f7a700 1 Processor -- start 2020-11-20T12:02:41.825 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.820+0000 7f3533f7a700 1 -- start start 2020-11-20T12:02:41.827 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3533f7a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c0ad890 0x7f352c0a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.828 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3533f7a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f352c13a550 0x7f352c0a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.828 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3533f7a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c0a39f0 con 0x7f352c0a4750 2020-11-20T12:02:41.828 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3533f7a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c0a50a0 con 0x7f352c13a550 2020-11-20T12:02:41.828 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3533f7a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c13aa00 con 0x7f352c0ad890 2020-11-20T12:02:41.828 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c0ad890 0x7f352c0a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3532517700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f352c13a550 0x7f352c0a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c0ad890 0x7f352c0a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41296/0 (socket says 172.21.15.79:41296) 2020-11-20T12:02:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 -- 172.21.15.79:0/2580394657 learned_addr learned my addr 172.21.15.79:0/2580394657 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3532517700 1 --2- 172.21.15.79:0/2580394657 >> v2:172.21.15.79:3300/0 conn(0x7f352c13a550 0x7f352c0a9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:42984/0 (socket says 172.21.15.79:42984) 2020-11-20T12:02:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 -- 172.21.15.79:0/2580394657 >> v2:172.21.15.79:3300/0 conn(0x7f352c13a550 msgr2=0x7f352c0a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 --2- 172.21.15.79:0/2580394657 >> v2:172.21.15.79:3300/0 conn(0x7f352c13a550 0x7f352c0a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 -- 172.21.15.79:0/2580394657 >> v1:172.21.15.44:6789/0 conn(0x7f352c0a4750 legacy=0x7f352c0a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 -- 172.21.15.79:0/2580394657 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0a9300 con 0x7f352c0ad890 2020-11-20T12:02:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3532517700 1 --2- 172.21.15.79:0/2580394657 >> v2:172.21.15.79:3300/0 conn(0x7f352c13a550 0x7f352c0a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.821+0000 7f3531d16700 1 --2- 172.21.15.79:0/2580394657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c0ad890 0x7f352c0a8c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3524009d50 tx=0x7f3524009290).ready entity=mon.2 client_cookie=f2fe48075c60740e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:41.831 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3530d14700 1 -- 172.21.15.79:0/2580394657 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352400bbc0 con 0x7f352c0ad890 2020-11-20T12:02:41.831 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3530d14700 1 -- 172.21.15.79:0/2580394657 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352400bd20 con 0x7f352c0ad890 2020-11-20T12:02:41.831 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3530d14700 1 -- 172.21.15.79:0/2580394657 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524010620 con 0x7f352c0ad890 2020-11-20T12:02:41.831 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 -- 172.21.15.79:0/2580394657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c0ad890 msgr2=0x7f352c0a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.832 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 --2- 172.21.15.79:0/2580394657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c0ad890 0x7f352c0a8c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3524009d50 tx=0x7f3524009290).stop 2020-11-20T12:02:41.832 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 -- 172.21.15.79:0/2580394657 shutdown_connections 2020-11-20T12:02:41.832 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 --2- 172.21.15.79:0/2580394657 >> v2:172.21.15.79:3300/0 conn(0x7f352c13a550 0x7f352c0a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.832 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 --2- 172.21.15.79:0/2580394657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c0ad890 0x7f352c0a8c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.832 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 -- 172.21.15.79:0/2580394657 >> 172.21.15.79:0/2580394657 conn(0x7f352c0619a0 msgr2=0x7f352c061db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:41.833 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 -- 172.21.15.79:0/2580394657 shutdown_connections 2020-11-20T12:02:41.833 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.822+0000 7f3533f7a700 1 -- 172.21.15.79:0/2580394657 wait complete. 2020-11-20T12:02:41.833 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.823+0000 7f3533f7a700 1 Processor -- start 2020-11-20T12:02:41.833 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.823+0000 7f3533f7a700 1 -- start start 2020-11-20T12:02:41.835 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3533f7a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352c0a4750 0x7f352c067210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.836 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3533f7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f352c0ad890 0x7f352c147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.836 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3533f7a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c13a550 0x7f352c1484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:41.836 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3533f7a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1489a0 con 0x7f352c0a4750 2020-11-20T12:02:41.836 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3533f7a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1373a0 con 0x7f352c0ad890 2020-11-20T12:02:41.836 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3533f7a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c09f630 con 0x7f352c13a550 2020-11-20T12:02:41.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3532517700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c13a550 0x7f352c1484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531515700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f352c0ad890 0x7f352c147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3532517700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c13a550 0x7f352c1484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41306/0 (socket says 172.21.15.79:41306) 2020-11-20T12:02:41.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3532517700 1 -- 172.21.15.79:0/3611244172 learned_addr learned my addr 172.21.15.79:0/3611244172 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:41.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531d16700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352c0a4750 0x7f352c067210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:41.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531515700 1 -- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c13a550 msgr2=0x7f352c1484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531515700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c13a550 0x7f352c1484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531515700 1 -- 172.21.15.79:0/3611244172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352c0a4750 msgr2=0x7f352c067210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531515700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352c0a4750 0x7f352c067210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531515700 1 -- 172.21.15.79:0/3611244172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c09f8d0 con 0x7f352c0ad890 2020-11-20T12:02:41.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531d16700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352c0a4750 0x7f352c067210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:41.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3532517700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c13a550 0x7f352c1484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:41.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.824+0000 7f3531515700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f352c0ad890 0x7f352c147fc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f351c00c710 tx=0x7f351c007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:41.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.825+0000 7f3519ffb700 1 -- 172.21.15.79:0/3611244172 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c007020 con 0x7f352c0ad890 2020-11-20T12:02:41.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.825+0000 7f3519ffb700 1 -- 172.21.15.79:0/3611244172 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c007180 con 0x7f352c0ad890 2020-11-20T12:02:41.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.825+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c09fb70 con 0x7f352c0ad890 2020-11-20T12:02:41.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.825+0000 7f3519ffb700 1 -- 172.21.15.79:0/3611244172 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c0064f0 con 0x7f352c0ad890 2020-11-20T12:02:41.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.825+0000 7f3519ffb700 1 -- 172.21.15.79:0/3611244172 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f351c010030 con 0x7f352c0ad890 2020-11-20T12:02:41.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.825+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c0a17b0 con 0x7f352c0ad890 2020-11-20T12:02:41.841 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.825+0000 7f3519ffb700 1 -- 172.21.15.79:0/3611244172 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f351c00e940 con 0x7f352c0ad890 2020-11-20T12:02:41.841 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.826+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c068740 con 0x7f352c0ad890 2020-11-20T12:02:41.841 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.828+0000 7f3519ffb700 1 -- 172.21.15.79:0/3611244172 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f351c006cf0 con 0x7f352c0ad890 2020-11-20T12:02:41.924 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.919+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2856c45e-54ed-4130-bd9f-898ffadae60a", "id": 0} v 0) v1 -- 0x7f352c09fd00 con 0x7f352c0ad890 2020-11-20T12:02:41.927 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.923+0000 7f3519ffb700 1 -- 172.21.15.79:0/3611244172 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2856c45e-54ed-4130-bd9f-898ffadae60a", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f351c00eb80 con 0x7f352c0ad890 2020-11-20T12:02:41.927 INFO:teuthology.orchestra.run.smithi079.stdout:0 2020-11-20T12:02:41.928 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f352c0ad890 msgr2=0x7f352c147fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:41.928 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f352c0ad890 0x7f352c147fc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f351c00c710 tx=0x7f351c007b90).stop 2020-11-20T12:02:41.929 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 shutdown_connections 2020-11-20T12:02:41.929 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f352c13a550 0x7f352c1484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.929 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f352c0a4750 0x7f352c067210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.929 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 --2- 172.21.15.79:0/3611244172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f352c0ad890 0x7f352c147fc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:41.930 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 >> 172.21.15.79:0/3611244172 conn(0x7f352c0619a0 msgr2=0x7f352c139e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:41.930 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 shutdown_connections 2020-11-20T12:02:41.930 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:41.924+0000 7f3533f7a700 1 -- 172.21.15.79:0/3611244172 wait complete. 2020-11-20T12:02:41.939 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 9af002f0-d9d2-4362-8855-73c63682dc0d 1 2020-11-20T12:02:42.067 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.062+0000 7fe2250d8700 1 Processor -- start 2020-11-20T12:02:42.067 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.062+0000 7fe2250d8700 1 -- start start 2020-11-20T12:02:42.069 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe2250d8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe220067f70 0x7fe220138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.069 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe2250d8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2200a75b0 0x7fe2201393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.070 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe2250d8700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2200aa5f0 con 0x7fe220060a40 2020-11-20T12:02:42.070 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe2250d8700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2200647f0 con 0x7fe220067f70 2020-11-20T12:02:42.070 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe2250d8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2200aa820 con 0x7fe2200a75b0 2020-11-20T12:02:42.070 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2200a75b0 0x7fe2201393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.071 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2200a75b0 0x7fe2201393e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41312/0 (socket says 172.21.15.79:41312) 2020-11-20T12:02:42.071 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 -- 172.21.15.79:0/4269198591 learned_addr learned my addr 172.21.15.79:0/4269198591 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:42.071 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21e59c700 1 --2- 172.21.15.79:0/4269198591 >> v2:172.21.15.44:3300/0 conn(0x7fe220067f70 0x7fe220138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.071 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 -- 172.21.15.79:0/4269198591 >> v2:172.21.15.44:3300/0 conn(0x7fe220067f70 msgr2=0x7fe220138ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:42.071 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 --2- 172.21.15.79:0/4269198591 >> v2:172.21.15.44:3300/0 conn(0x7fe220067f70 0x7fe220138ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.072 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 -- 172.21.15.79:0/4269198591 >> v1:172.21.15.44:6789/0 conn(0x7fe220060a40 legacy=0x7fe2200a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.072 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 -- 172.21.15.79:0/4269198591 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200aad60 con 0x7fe2200a75b0 2020-11-20T12:02:42.072 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.063+0000 7fe21f59e700 1 --2- 172.21.15.79:0/4269198591 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2200a75b0 0x7fe2201393e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe210006fd0 tx=0x7fe210006eb0).ready entity=mon.2 client_cookie=ee125a0db566b6bd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.072 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe21dd9b700 1 -- 172.21.15.79:0/4269198591 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210009810 con 0x7fe2200a75b0 2020-11-20T12:02:42.072 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe21dd9b700 1 -- 172.21.15.79:0/4269198591 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe210009970 con 0x7fe2200a75b0 2020-11-20T12:02:42.073 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe21dd9b700 1 -- 172.21.15.79:0/4269198591 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210009cf0 con 0x7fe2200a75b0 2020-11-20T12:02:42.073 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 -- 172.21.15.79:0/4269198591 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2200a75b0 msgr2=0x7fe2201393e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.073 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 --2- 172.21.15.79:0/4269198591 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2200a75b0 0x7fe2201393e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe210006fd0 tx=0x7fe210006eb0).stop 2020-11-20T12:02:42.073 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 -- 172.21.15.79:0/4269198591 shutdown_connections 2020-11-20T12:02:42.073 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 --2- 172.21.15.79:0/4269198591 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2200a75b0 0x7fe2201393e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 --2- 172.21.15.79:0/4269198591 >> v2:172.21.15.44:3300/0 conn(0x7fe220067f70 0x7fe220138ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 -- 172.21.15.79:0/4269198591 >> 172.21.15.79:0/4269198591 conn(0x7fe220069830 msgr2=0x7fe220060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 -- 172.21.15.79:0/4269198591 shutdown_connections 2020-11-20T12:02:42.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.064+0000 7fe2250d8700 1 -- 172.21.15.79:0/4269198591 wait complete. 2020-11-20T12:02:42.075 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.065+0000 7fe2250d8700 1 Processor -- start 2020-11-20T12:02:42.075 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.065+0000 7fe2250d8700 1 -- start start 2020-11-20T12:02:42.077 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe2250d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe220060a40 0x7fe2200a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.077 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe2250d8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe220067f70 0x7fe22013fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.077 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe2250d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200a75b0 0x7fe220149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.078 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe2250d8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220062bf0 con 0x7fe2200a75b0 2020-11-20T12:02:42.078 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe2250d8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220062e90 con 0x7fe220060a40 2020-11-20T12:02:42.078 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe2250d8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe220063130 con 0x7fe220067f70 2020-11-20T12:02:42.078 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe220067f70 0x7fe22013fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.079 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe220067f70 0x7fe22013fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41316/0 (socket says 172.21.15.79:41316) 2020-11-20T12:02:42.079 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 -- 172.21.15.79:0/1765863582 learned_addr learned my addr 172.21.15.79:0/1765863582 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:42.079 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21ed9d700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe220060a40 0x7fe2200a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.079 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21f59e700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200a75b0 0x7fe220149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.079 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 -- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe220060a40 msgr2=0x7fe2200a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.080 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe220060a40 0x7fe2200a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.080 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 -- 172.21.15.79:0/1765863582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200a75b0 msgr2=0x7fe220149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.080 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200a75b0 0x7fe220149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.080 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 -- 172.21.15.79:0/1765863582 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200633d0 con 0x7fe220067f70 2020-11-20T12:02:42.081 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21f59e700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200a75b0 0x7fe220149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:42.081 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe21e59c700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe220067f70 0x7fe22013fbf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe20800eba0 tx=0x7fe208006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.081 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe216ffd700 1 -- 172.21.15.79:0/1765863582 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800c8e0 con 0x7fe220067f70 2020-11-20T12:02:42.081 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.066+0000 7fe216ffd700 1 -- 172.21.15.79:0/1765863582 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20800ca40 con 0x7fe220067f70 2020-11-20T12:02:42.081 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.067+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe220063870 con 0x7fe220067f70 2020-11-20T12:02:42.082 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.067+0000 7fe216ffd700 1 -- 172.21.15.79:0/1765863582 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800cdc0 con 0x7fe220067f70 2020-11-20T12:02:42.082 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.067+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22014a350 con 0x7fe220067f70 2020-11-20T12:02:42.082 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.067+0000 7fe216ffd700 1 -- 172.21.15.79:0/1765863582 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fe208012860 con 0x7fe220067f70 2020-11-20T12:02:42.082 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.067+0000 7fe216ffd700 1 -- 172.21.15.79:0/1765863582 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fe20801c880 con 0x7fe220067f70 2020-11-20T12:02:42.082 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.067+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe220063560 con 0x7fe220067f70 2020-11-20T12:02:42.083 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.070+0000 7fe216ffd700 1 -- 172.21.15.79:0/1765863582 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fe208012cd0 con 0x7fe220067f70 2020-11-20T12:02:42.196 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.192+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9af002f0-d9d2-4362-8855-73c63682dc0d", "id": 1} v 0) v1 -- 0x7fe2200446d0 con 0x7fe220067f70 2020-11-20T12:02:42.203 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.199+0000 7fe216ffd700 1 -- 172.21.15.79:0/1765863582 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9af002f0-d9d2-4362-8855-73c63682dc0d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe20801cac0 con 0x7fe220067f70 2020-11-20T12:02:42.203 INFO:teuthology.orchestra.run.smithi079.stdout:1 2020-11-20T12:02:42.204 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe220067f70 msgr2=0x7fe22013fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.205 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe220067f70 0x7fe22013fbf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe20800eba0 tx=0x7fe208006df0).stop 2020-11-20T12:02:42.205 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 shutdown_connections 2020-11-20T12:02:42.205 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200a75b0 0x7fe220149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.205 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe220060a40 0x7fe2200a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.206 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 --2- 172.21.15.79:0/1765863582 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe220067f70 0x7fe22013fbf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.206 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 >> 172.21.15.79:0/1765863582 conn(0x7fe220069830 msgr2=0x7fe2200a6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.206 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 shutdown_connections 2020-11-20T12:02:42.206 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.200+0000 7fe2250d8700 1 -- 172.21.15.79:0/1765863582 wait complete. 2020-11-20T12:02:42.215 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 475fc90c-4230-4093-9438-34ba65bdfdde 2 2020-11-20T12:02:42.341 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.337+0000 7faf9502c700 1 Processor -- start 2020-11-20T12:02:42.342 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.337+0000 7faf9502c700 1 -- start start 2020-11-20T12:02:42.344 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf9502c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faf900ab050 0x7faf900abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf9502c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faf9013b590 0x7faf900a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf9502c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ac2e0 0x7faf900ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf9502c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf900a8770 con 0x7faf9013b590 2020-11-20T12:02:42.345 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf9502c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf90138480 con 0x7faf900ab050 2020-11-20T12:02:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf9502c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9013abb0 con 0x7faf900ac2e0 2020-11-20T12:02:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8ed9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faf900ab050 0x7faf900abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8f59e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ac2e0 0x7faf900ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8ed9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faf900ab050 0x7faf900abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43004/0 (socket says 172.21.15.79:43004) 2020-11-20T12:02:42.346 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8ed9d700 1 -- 172.21.15.79:0/2276427218 learned_addr learned my addr 172.21.15.79:0/2276427218 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8e59c700 1 --2- 172.21.15.79:0/2276427218 >> v2:172.21.15.44:3300/0 conn(0x7faf9013b590 0x7faf900a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8f59e700 1 -- 172.21.15.79:0/2276427218 >> v2:172.21.15.79:3300/0 conn(0x7faf900ab050 msgr2=0x7faf900abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8f59e700 1 --2- 172.21.15.79:0/2276427218 >> v2:172.21.15.79:3300/0 conn(0x7faf900ab050 0x7faf900abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8f59e700 1 -- 172.21.15.79:0/2276427218 >> v2:172.21.15.44:3300/0 conn(0x7faf9013b590 msgr2=0x7faf900a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:42.347 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8f59e700 1 --2- 172.21.15.79:0/2276427218 >> v2:172.21.15.44:3300/0 conn(0x7faf9013b590 0x7faf900a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.348 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8f59e700 1 -- 172.21.15.79:0/2276427218 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9013ae50 con 0x7faf900ac2e0 2020-11-20T12:02:42.348 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8ed9d700 1 --2- 172.21.15.79:0/2276427218 >> v2:172.21.15.79:3300/0 conn(0x7faf900ab050 0x7faf900abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:42.348 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8f59e700 1 --2- 172.21.15.79:0/2276427218 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ac2e0 0x7faf900ac740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7faf900b1190 tx=0x7faf80006df0).ready entity=mon.2 client_cookie=926f04bda386b906 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.348 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8dd9b700 1 -- 172.21.15.79:0/2276427218 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80009640 con 0x7faf900ac2e0 2020-11-20T12:02:42.349 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8dd9b700 1 -- 172.21.15.79:0/2276427218 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf800097a0 con 0x7faf900ac2e0 2020-11-20T12:02:42.349 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.338+0000 7faf8dd9b700 1 -- 172.21.15.79:0/2276427218 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80009b20 con 0x7faf900ac2e0 2020-11-20T12:02:42.349 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 -- 172.21.15.79:0/2276427218 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ac2e0 msgr2=0x7faf900ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.349 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 --2- 172.21.15.79:0/2276427218 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ac2e0 0x7faf900ac740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7faf900b1190 tx=0x7faf80006df0).stop 2020-11-20T12:02:42.349 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 -- 172.21.15.79:0/2276427218 shutdown_connections 2020-11-20T12:02:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 --2- 172.21.15.79:0/2276427218 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ac2e0 0x7faf900ac740 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 --2- 172.21.15.79:0/2276427218 >> v2:172.21.15.79:3300/0 conn(0x7faf900ab050 0x7faf900abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 --2- 172.21.15.79:0/2276427218 >> v2:172.21.15.44:3300/0 conn(0x7faf9013b590 0x7faf900a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 -- 172.21.15.79:0/2276427218 >> 172.21.15.79:0/2276427218 conn(0x7faf90060170 msgr2=0x7faf90060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.350 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 -- 172.21.15.79:0/2276427218 shutdown_connections 2020-11-20T12:02:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.339+0000 7faf9502c700 1 -- 172.21.15.79:0/2276427218 wait complete. 2020-11-20T12:02:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.340+0000 7faf9502c700 1 Processor -- start 2020-11-20T12:02:42.351 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.340+0000 7faf9502c700 1 -- start start 2020-11-20T12:02:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ab050 0x7faf900b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.353 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ac2e0 0x7faf9013e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.354 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9013b590 0x7faf90137400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.354 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9013e150 con 0x7faf900ac2e0 2020-11-20T12:02:42.354 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900655f0 con 0x7faf9013b590 2020-11-20T12:02:42.354 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf90065890 con 0x7faf900ab050 2020-11-20T12:02:42.355 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8ed9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ab050 0x7faf900b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.355 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8f59e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9013b590 0x7faf90137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.355 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8ed9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ab050 0x7faf900b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41326/0 (socket says 172.21.15.79:41326) 2020-11-20T12:02:42.355 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8ed9d700 1 -- 172.21.15.79:0/2145813989 learned_addr learned my addr 172.21.15.79:0/2145813989 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:42.355 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8e59c700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ac2e0 0x7faf9013e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.356 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8f59e700 1 -- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ab050 msgr2=0x7faf900b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.356 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8f59e700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ab050 0x7faf900b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.356 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8f59e700 1 -- 172.21.15.79:0/2145813989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ac2e0 msgr2=0x7faf9013e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.356 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8f59e700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ac2e0 0x7faf9013e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.356 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8f59e700 1 -- 172.21.15.79:0/2145813989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90065c20 con 0x7faf9013b590 2020-11-20T12:02:42.357 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8ed9d700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ab050 0x7faf900b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:42.357 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf8f59e700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9013b590 0x7faf90137400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7faf80012000 tx=0x7faf80015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.357 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf86ffd700 1 -- 172.21.15.79:0/2145813989 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80009b60 con 0x7faf9013b590 2020-11-20T12:02:42.357 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf86ffd700 1 -- 172.21.15.79:0/2145813989 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8001e460 con 0x7faf9013b590 2020-11-20T12:02:42.357 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf900a7540 con 0x7faf9013b590 2020-11-20T12:02:42.358 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf86ffd700 1 -- 172.21.15.79:0/2145813989 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8001e800 con 0x7faf9013b590 2020-11-20T12:02:42.358 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.341+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf900a4940 con 0x7faf9013b590 2020-11-20T12:02:42.358 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.342+0000 7faf86ffd700 1 -- 172.21.15.79:0/2145813989 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7faf80025030 con 0x7faf9013b590 2020-11-20T12:02:42.358 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.342+0000 7faf86ffd700 1 -- 172.21.15.79:0/2145813989 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7faf8001d810 con 0x7faf9013b590 2020-11-20T12:02:42.358 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.342+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9013e2c0 con 0x7faf9013b590 2020-11-20T12:02:42.359 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.344+0000 7faf86ffd700 1 -- 172.21.15.79:0/2145813989 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7faf9013e2c0 con 0x7faf9013b590 2020-11-20T12:02:42.438 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.434+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd new", "uuid": "475fc90c-4230-4093-9438-34ba65bdfdde", "id": 2} v 0) v1 -- 0x7faf900a4b30 con 0x7faf9013b590 2020-11-20T12:02:42.449 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.444+0000 7faf86ffd700 1 -- 172.21.15.79:0/2145813989 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "475fc90c-4230-4093-9438-34ba65bdfdde", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7faf80023070 con 0x7faf9013b590 2020-11-20T12:02:42.449 INFO:teuthology.orchestra.run.smithi079.stdout:2 2020-11-20T12:02:42.450 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9013b590 msgr2=0x7faf90137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.450 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9013b590 0x7faf90137400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7faf80012000 tx=0x7faf80015000).stop 2020-11-20T12:02:42.451 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 shutdown_connections 2020-11-20T12:02:42.451 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf9013b590 0x7faf90137400 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.451 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf900ab050 0x7faf900b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.451 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 --2- 172.21.15.79:0/2145813989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf900ac2e0 0x7faf9013e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.451 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 >> 172.21.15.79:0/2145813989 conn(0x7faf90060170 msgr2=0x7faf900ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.452 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 shutdown_connections 2020-11-20T12:02:42.452 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.445+0000 7faf9502c700 1 -- 172.21.15.79:0/2145813989 wait complete. 2020-11-20T12:02:42.460 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new f951d29a-4ead-4d15-8e19-178a8eef2b91 3 2020-11-20T12:02:42.585 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.580+0000 7fbf81108700 1 Processor -- start 2020-11-20T12:02:42.585 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.580+0000 7fbf81108700 1 -- start start 2020-11-20T12:02:42.586 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf81108700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbf7c0a4280 0x7fbf7c0a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.586 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf81108700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c0aa700 con 0x7fbf7c0a8cd0 2020-11-20T12:02:42.586 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf81108700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c0a51e0 con 0x7fbf7c0a88f0 2020-11-20T12:02:42.586 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf81108700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c1386c0 con 0x7fbf7c0a4280 2020-11-20T12:02:42.586 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf7ad9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbf7c0a88f0 0x7fbf7c137fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53368/0 (socket says 172.21.15.79:53368) 2020-11-20T12:02:42.587 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf7ad9d700 1 -- 172.21.15.79:0/731043505 learned_addr learned my addr 172.21.15.79:0/731043505 (peer_addr_for_me v1:172.21.15.79:0/0) 2020-11-20T12:02:42.591 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf7b59e700 1 --2- 172.21.15.79:0/731043505 >> v2:172.21.15.44:3300/0 conn(0x7fbf7c0a4280 0x7fbf7c0a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.592 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316675957 0 0) 0x7fbf7c0a51e0 con 0x7fbf7c0a88f0 2020-11-20T12:02:42.592 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf68003240 con 0x7fbf7c0a88f0 2020-11-20T12:02:42.592 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140854706 0 0) 0x7fbf700010d0 con 0x7fbf7c0a8cd0 2020-11-20T12:02:42.592 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf7c0a51e0 con 0x7fbf7c0a8cd0 2020-11-20T12:02:42.592 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 848833807 0 0) 0x7fbf68003240 con 0x7fbf7c0a88f0 2020-11-20T12:02:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 >> v2:172.21.15.44:3300/0 conn(0x7fbf7c0a4280 msgr2=0x7fbf7c0a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 --2- 172.21.15.79:0/731043505 >> v2:172.21.15.44:3300/0 conn(0x7fbf7c0a4280 0x7fbf7c0a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 >> v1:172.21.15.44:6789/0 conn(0x7fbf7c0a8cd0 legacy=0x7fbf7c0a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0a68f0 con 0x7fbf7c0a88f0 2020-11-20T12:02:42.593 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fbf740018a0 con 0x7fbf7c0a88f0 2020-11-20T12:02:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 >> v1:172.21.15.79:6789/0 conn(0x7fbf7c0a88f0 legacy=0x7fbf7c137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf680038c0 0x7fbf68003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf68004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.594 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf68004e50 0x7fbf6800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c1386c0 con 0x7fbf680038c0 2020-11-20T12:02:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680053b0 con 0x7fbf680044e0 2020-11-20T12:02:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6800bca0 con 0x7fbf68004e50 2020-11-20T12:02:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf7a59c700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf68004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.595 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf7ad9d700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf680038c0 0x7fbf68003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.581+0000 7fbf7b59e700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf68004e50 0x7fbf6800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf7a59c700 1 -- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf68004e50 msgr2=0x7fbf6800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf7a59c700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf68004e50 0x7fbf6800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf7a59c700 1 -- 172.21.15.79:0/731043505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf680038c0 msgr2=0x7fbf68003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:42.596 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf7a59c700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf680038c0 0x7fbf68003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf7a59c700 1 -- 172.21.15.79:0/731043505 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf68003240 con 0x7fbf680044e0 2020-11-20T12:02:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf7b59e700 1 -- 172.21.15.79:0/731043505 reap_dead start 2020-11-20T12:02:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf7a59c700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf68004960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fbf70007c60 tx=0x7fbf70001b40).ready entity=mon.1 client_cookie=84e6c5aa01dd7502 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000c900 con 0x7fbf680044e0 2020-11-20T12:02:42.597 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7000ca60 con 0x7fbf680044e0 2020-11-20T12:02:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf79d9b700 1 -- 172.21.15.79:0/731043505 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000ce20 con 0x7fbf680044e0 2020-11-20T12:02:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf81108700 1 -- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 msgr2=0x7fbf68004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf81108700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf68004960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fbf70007c60 tx=0x7fbf70001b40).stop 2020-11-20T12:02:42.598 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf81108700 1 -- 172.21.15.79:0/731043505 shutdown_connections 2020-11-20T12:02:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf81108700 1 --2- 172.21.15.79:0/731043505 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf68004960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf81108700 1 -- 172.21.15.79:0/731043505 >> 172.21.15.79:0/731043505 conn(0x7fbf7c0619a0 msgr2=0x7fbf7c061db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.582+0000 7fbf81108700 1 -- 172.21.15.79:0/731043505 shutdown_connections 2020-11-20T12:02:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.583+0000 7fbf81108700 1 -- 172.21.15.79:0/731043505 wait complete. 2020-11-20T12:02:42.599 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 Processor -- start 2020-11-20T12:02:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 -- start start 2020-11-20T12:02:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf7c0ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf7c0a3590 0x7fbf7c0ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf7c0ac770 0x7fbf7c0a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.600 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0a21b0 con 0x7fbf7c0a3590 2020-11-20T12:02:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0acbf0 con 0x7fbf680044e0 2020-11-20T12:02:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf81108700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0a1c70 con 0x7fbf7c0ac770 2020-11-20T12:02:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf7ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf7c0ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf7b59e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf7c0ac770 0x7fbf7c0a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.601 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf7ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf7c0ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43028/0 (socket says 172.21.15.79:43028) 2020-11-20T12:02:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf7ad9d700 1 -- 172.21.15.79:0/3437003450 learned_addr learned my addr 172.21.15.79:0/3437003450 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf7b59e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf7c0ac770 0x7fbf7c0a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41348/0 (socket says 172.21.15.79:41348) 2020-11-20T12:02:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.584+0000 7fbf7a59c700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf7c0a3590 0x7fbf7c0ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf7ad9d700 1 -- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf7c0ac770 msgr2=0x7fbf7c0a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.602 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf7ad9d700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf7c0ac770 0x7fbf7c0a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.603 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf7ad9d700 1 -- 172.21.15.79:0/3437003450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf7c0a3590 msgr2=0x7fbf7c0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.603 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf7ad9d700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf7c0a3590 0x7fbf7c0ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.603 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf7ad9d700 1 -- 172.21.15.79:0/3437003450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0a1f10 con 0x7fbf680044e0 2020-11-20T12:02:42.603 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf7b59e700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf7c0ac770 0x7fbf7c0a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:42.604 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf7ad9d700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf7c0ad530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fbf74007710 tx=0x7fbf7c0a6a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.604 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf62ffd700 1 -- 172.21.15.79:0/3437003450 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400fbb0 con 0x7fbf680044e0 2020-11-20T12:02:42.604 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf62ffd700 1 -- 172.21.15.79:0/3437003450 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400fd10 con 0x7fbf680044e0 2020-11-20T12:02:42.604 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c1388d0 con 0x7fbf680044e0 2020-11-20T12:02:42.604 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf62ffd700 1 -- 172.21.15.79:0/3437003450 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74012560 con 0x7fbf680044e0 2020-11-20T12:02:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c0ab570 con 0x7fbf680044e0 2020-11-20T12:02:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf62ffd700 1 -- 172.21.15.79:0/3437003450 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fbf7401b030 con 0x7fbf680044e0 2020-11-20T12:02:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.585+0000 7fbf62ffd700 1 -- 172.21.15.79:0/3437003450 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7fbf740015b0 con 0x7fbf680044e0 2020-11-20T12:02:42.605 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.586+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c068740 con 0x7fbf680044e0 2020-11-20T12:02:42.606 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.588+0000 7fbf62ffd700 1 -- 172.21.15.79:0/3437003450 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fbf74019070 con 0x7fbf680044e0 2020-11-20T12:02:42.695 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.691+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f951d29a-4ead-4d15-8e19-178a8eef2b91", "id": 3} v 0) v1 -- 0x7fbf7c0446d0 con 0x7fbf680044e0 2020-11-20T12:02:42.703 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.699+0000 7fbf62ffd700 1 -- 172.21.15.79:0/3437003450 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f951d29a-4ead-4d15-8e19-178a8eef2b91", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf74012da0 con 0x7fbf680044e0 2020-11-20T12:02:42.703 INFO:teuthology.orchestra.run.smithi079.stdout:3 2020-11-20T12:02:42.704 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 msgr2=0x7fbf7c0ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.704 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf7c0ad530 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fbf74007710 tx=0x7fbf7c0a6a80).stop 2020-11-20T12:02:42.705 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 shutdown_connections 2020-11-20T12:02:42.705 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf7c0ac770 0x7fbf7c0a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.705 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf680044e0 0x7fbf7c0ad530 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.705 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 --2- 172.21.15.79:0/3437003450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf7c0a3590 0x7fbf7c0ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.706 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 >> 172.21.15.79:0/3437003450 conn(0x7fbf7c0619a0 msgr2=0x7fbf7c05ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.706 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 shutdown_connections 2020-11-20T12:02:42.706 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.700+0000 7fbf81108700 1 -- 172.21.15.79:0/3437003450 wait complete. 2020-11-20T12:02:42.715 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 74c35433-a1af-4165-ba72-6935b43df900 4 2020-11-20T12:02:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.829+0000 7f20025d3700 1 Processor -- start 2020-11-20T12:02:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f20025d3700 1 -- start start 2020-11-20T12:02:42.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f20025d3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1ffc0ab050 0x7f1ffc0abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f20025d3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1ffc0ac2e0 0x7f1ffc0ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f20025d3700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc13abb0 con 0x7f1ffc13b590 2020-11-20T12:02:42.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f20025d3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc0a8770 con 0x7f1ffc0ab050 2020-11-20T12:02:42.837 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f20025d3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc138480 con 0x7f1ffc0ac2e0 2020-11-20T12:02:42.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f2000b70700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1ffc0ac2e0 0x7f1ffc0ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f2000b70700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1ffc0ac2e0 0x7f1ffc0ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43036/0 (socket says 172.21.15.79:43036) 2020-11-20T12:02:42.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f2000b70700 1 -- 172.21.15.79:0/2116936713 learned_addr learned my addr 172.21.15.79:0/2116936713 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:42.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.830+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/2116936713 >> v2:172.21.15.44:3300/0 conn(0x7f1ffc0ab050 0x7f1ffc0abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.838 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f2000b70700 1 -- 172.21.15.79:0/2116936713 >> v2:172.21.15.44:3300/0 conn(0x7f1ffc0ab050 msgr2=0x7f1ffc0abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:42.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f2000b70700 1 --2- 172.21.15.79:0/2116936713 >> v2:172.21.15.44:3300/0 conn(0x7f1ffc0ab050 0x7f1ffc0abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f2000b70700 1 -- 172.21.15.79:0/2116936713 >> v1:172.21.15.44:6789/0 conn(0x7f1ffc13b590 legacy=0x7f1ffc0a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f2000b70700 1 -- 172.21.15.79:0/2116936713 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc13ae50 con 0x7f1ffc0ac2e0 2020-11-20T12:02:42.839 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f2000b70700 1 --2- 172.21.15.79:0/2116936713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ac2e0 0x7f1ffc0ac740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1ffc0640d0 tx=0x7f1ff4009290).ready entity=mon.1 client_cookie=81e27f23f28a3b88 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f1ffaffd700 1 -- 172.21.15.79:0/2116936713 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400bbc0 con 0x7f1ffc0ac2e0 2020-11-20T12:02:42.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f1ffaffd700 1 -- 172.21.15.79:0/2116936713 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400bd20 con 0x7f1ffc0ac2e0 2020-11-20T12:02:42.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f1ffaffd700 1 -- 172.21.15.79:0/2116936713 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4011610 con 0x7f1ffc0ac2e0 2020-11-20T12:02:42.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 -- 172.21.15.79:0/2116936713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ac2e0 msgr2=0x7f1ffc0ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.840 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 --2- 172.21.15.79:0/2116936713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ac2e0 0x7f1ffc0ac740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f1ffc0640d0 tx=0x7f1ff4009290).stop 2020-11-20T12:02:42.841 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 -- 172.21.15.79:0/2116936713 shutdown_connections 2020-11-20T12:02:42.841 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 --2- 172.21.15.79:0/2116936713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ac2e0 0x7f1ffc0ac740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.841 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 --2- 172.21.15.79:0/2116936713 >> v2:172.21.15.44:3300/0 conn(0x7f1ffc0ab050 0x7f1ffc0abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.841 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 -- 172.21.15.79:0/2116936713 >> 172.21.15.79:0/2116936713 conn(0x7f1ffc060170 msgr2=0x7f1ffc060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.842 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 -- 172.21.15.79:0/2116936713 shutdown_connections 2020-11-20T12:02:42.842 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.831+0000 7f20025d3700 1 -- 172.21.15.79:0/2116936713 wait complete. 2020-11-20T12:02:42.844 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.833+0000 7f20025d3700 1 Processor -- start 2020-11-20T12:02:42.844 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.833+0000 7f20025d3700 1 -- start start 2020-11-20T12:02:42.845 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f20025d3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ab050 0x7f1ffc1391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.845 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f20025d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ffc0ac2e0 0x7f1ffc13dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.845 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f20025d3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ffc13b590 0x7f1ffc13e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:42.845 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f20025d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc13e7c0 con 0x7f1ffc0ac2e0 2020-11-20T12:02:42.846 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f20025d3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc13f490 con 0x7f1ffc0ab050 2020-11-20T12:02:42.846 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f20025d3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc13f730 con 0x7f1ffc13b590 2020-11-20T12:02:42.846 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ab050 0x7f1ffc1391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.846 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ab050 0x7f1ffc1391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43040/0 (socket says 172.21.15.79:43040) 2020-11-20T12:02:42.846 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f2000b70700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ffc13b590 0x7f1ffc13e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.847 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f1ffbfff700 1 -- 172.21.15.79:0/1725765817 learned_addr learned my addr 172.21.15.79:0/1725765817 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:42.847 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f1ffb7fe700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ffc0ac2e0 0x7f1ffc13dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:42.847 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.834+0000 7f1ffbfff700 1 -- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ffc13b590 msgr2=0x7f1ffc13e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.847 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ffc13b590 0x7f1ffc13e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.848 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ffbfff700 1 -- 172.21.15.79:0/1725765817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ffc0ac2e0 msgr2=0x7f1ffc13dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.848 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ffc0ac2e0 0x7f1ffc13dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.848 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ffbfff700 1 -- 172.21.15.79:0/1725765817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc13f1a0 con 0x7f1ffc0ab050 2020-11-20T12:02:42.848 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f2000b70700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ffc13b590 0x7f1ffc13e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:42.848 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ffbfff700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ab050 0x7f1ffc1391d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f1fe400ebd0 tx=0x7f1fe4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:42.849 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ff3fff700 1 -- 172.21.15.79:0/1725765817 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400c820 con 0x7f1ffc0ab050 2020-11-20T12:02:42.849 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ff3fff700 1 -- 172.21.15.79:0/1725765817 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe400c980 con 0x7f1ffc0ab050 2020-11-20T12:02:42.849 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc13d220 con 0x7f1ffc0ab050 2020-11-20T12:02:42.849 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ff3fff700 1 -- 172.21.15.79:0/1725765817 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400cd00 con 0x7f1ffc0ab050 2020-11-20T12:02:42.849 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc0a4650 con 0x7f1ffc0ab050 2020-11-20T12:02:42.850 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ff3fff700 1 -- 172.21.15.79:0/1725765817 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f1fe4022070 con 0x7f1ffc0ab050 2020-11-20T12:02:42.850 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.835+0000 7f1ff3fff700 1 -- 172.21.15.79:0/1725765817 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f1fe400ce60 con 0x7f1ffc0ab050 2020-11-20T12:02:42.850 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.836+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc13f8f0 con 0x7f1ffc0ab050 2020-11-20T12:02:42.850 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.838+0000 7f1ff3fff700 1 -- 172.21.15.79:0/1725765817 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f1fe403a880 con 0x7f1ffc0ab050 2020-11-20T12:02:42.931 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.927+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd new", "uuid": "74c35433-a1af-4165-ba72-6935b43df900", "id": 4} v 0) v1 -- 0x7f1ffc0696f0 con 0x7f1ffc0ab050 2020-11-20T12:02:42.939 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.935+0000 7f1ff3fff700 1 -- 172.21.15.79:0/1725765817 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74c35433-a1af-4165-ba72-6935b43df900", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1fe4029030 con 0x7f1ffc0ab050 2020-11-20T12:02:42.939 INFO:teuthology.orchestra.run.smithi079.stdout:4 2020-11-20T12:02:42.940 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ab050 msgr2=0x7f1ffc1391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:42.940 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ab050 0x7f1ffc1391d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f1fe400ebd0 tx=0x7f1fe4008040).stop 2020-11-20T12:02:42.941 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 shutdown_connections 2020-11-20T12:02:42.941 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ffc13b590 0x7f1ffc13e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.941 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ffc0ab050 0x7f1ffc1391d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.941 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 --2- 172.21.15.79:0/1725765817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ffc0ac2e0 0x7f1ffc13dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:42.942 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 >> 172.21.15.79:0/1725765817 conn(0x7f1ffc060170 msgr2=0x7f1ffc0ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:42.942 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 shutdown_connections 2020-11-20T12:02:42.942 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:42.936+0000 7f20025d3700 1 -- 172.21.15.79:0/1725765817 wait complete. 2020-11-20T12:02:42.951 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 9495ddb8-00b5-4d73-acb2-86bf2949700f 5 2020-11-20T12:02:43.069 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.064+0000 7f07d520f700 1 Processor -- start 2020-11-20T12:02:43.069 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.064+0000 7f07d520f700 1 -- start start 2020-11-20T12:02:43.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07d520f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07d00ad890 0x7f07d00a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07d520f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d00a39f0 con 0x7f07d013a550 2020-11-20T12:02:43.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07d520f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d013a9c0 con 0x7f07d00a4750 2020-11-20T12:02:43.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07d520f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d00a50a0 con 0x7f07d00ad890 2020-11-20T12:02:43.074 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07ced9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07d00ad890 0x7f07d00a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.075 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07ce59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f07d00a4750 0x7f07d00a9870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53398/0 (socket says 172.21.15.79:53398) 2020-11-20T12:02:43.075 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07ce59c700 1 -- 172.21.15.79:0/1077031840 learned_addr learned my addr 172.21.15.79:0/1077031840 (peer_addr_for_me v1:172.21.15.79:0/0) 2020-11-20T12:02:43.075 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07ced9d700 1 -- 172.21.15.79:0/1077031840 >> v1:172.21.15.79:6789/0 conn(0x7f07d00a4750 legacy=0x7f07d00a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.075 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07ced9d700 1 -- 172.21.15.79:0/1077031840 >> v1:172.21.15.44:6789/0 conn(0x7f07d013a550 legacy=0x7f07d00a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.076 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07cdd9b700 1 -- 172.21.15.79:0/1077031840 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962483447 0 0) 0x7f07d013a9c0 con 0x7f07d00a4750 2020-11-20T12:02:43.076 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.065+0000 7f07ced9d700 1 -- 172.21.15.79:0/1077031840 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d00a9300 con 0x7f07d00ad890 2020-11-20T12:02:43.076 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07ced9d700 1 --2- 172.21.15.79:0/1077031840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00ad890 0x7f07d00a8c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f07c8009c90 tx=0x7f07c80091d0).ready entity=mon.1 client_cookie=6be21e0ae4be0716 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:43.076 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07cdd9b700 1 -- 172.21.15.79:0/1077031840 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c800bbc0 con 0x7f07d00ad890 2020-11-20T12:02:43.076 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07cdd9b700 1 -- 172.21.15.79:0/1077031840 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c800bd20 con 0x7f07d00ad890 2020-11-20T12:02:43.077 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07cdd9b700 1 -- 172.21.15.79:0/1077031840 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8011610 con 0x7f07d00ad890 2020-11-20T12:02:43.077 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07d520f700 1 -- 172.21.15.79:0/1077031840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00ad890 msgr2=0x7f07d00a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.077 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07d520f700 1 --2- 172.21.15.79:0/1077031840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00ad890 0x7f07d00a8c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f07c8009c90 tx=0x7f07c80091d0).stop 2020-11-20T12:02:43.077 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07d520f700 1 -- 172.21.15.79:0/1077031840 shutdown_connections 2020-11-20T12:02:43.078 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07d520f700 1 --2- 172.21.15.79:0/1077031840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00ad890 0x7f07d00a8c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.078 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07d520f700 1 -- 172.21.15.79:0/1077031840 >> 172.21.15.79:0/1077031840 conn(0x7f07d00619a0 msgr2=0x7f07d0061db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:43.078 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07d520f700 1 -- 172.21.15.79:0/1077031840 shutdown_connections 2020-11-20T12:02:43.079 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.066+0000 7f07d520f700 1 -- 172.21.15.79:0/1077031840 wait complete. 2020-11-20T12:02:43.079 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.067+0000 7f07d520f700 1 Processor -- start 2020-11-20T12:02:43.080 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.067+0000 7f07d520f700 1 -- start start 2020-11-20T12:02:43.080 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07d520f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00a4750 0x7f07d00a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.081 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07d520f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07d00ad890 0x7f07d009e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.081 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07d520f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d013a550 0x7f07d009ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.082 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07d520f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0137c80 con 0x7f07d013a550 2020-11-20T12:02:43.082 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07d520f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d00a03b0 con 0x7f07d00a4750 2020-11-20T12:02:43.083 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07d520f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d00a0650 con 0x7f07d00ad890 2020-11-20T12:02:43.083 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ce59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07d00ad890 0x7f07d009e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.083 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00a4750 0x7f07d00a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.083 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00a4750 0x7f07d00a32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43056/0 (socket says 172.21.15.79:43056) 2020-11-20T12:02:43.083 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ce59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07d00ad890 0x7f07d009e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41368/0 (socket says 172.21.15.79:41368) 2020-11-20T12:02:43.084 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 -- 172.21.15.79:0/2551795176 learned_addr learned my addr 172.21.15.79:0/2551795176 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:43.084 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07cf59e700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d013a550 0x7f07d009ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.084 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 -- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07d00ad890 msgr2=0x7f07d009e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.084 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07d00ad890 0x7f07d009e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.084 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 -- 172.21.15.79:0/2551795176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d013a550 msgr2=0x7f07d009ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.085 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d013a550 0x7f07d009ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.085 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 -- 172.21.15.79:0/2551795176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d00a08f0 con 0x7f07d00a4750 2020-11-20T12:02:43.085 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ce59c700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07d00ad890 0x7f07d009e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:43.085 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.068+0000 7f07ced9d700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00a4750 0x7f07d00a32b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f07c8011730 tx=0x7f07c8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:43.086 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.069+0000 7f07c6ffd700 1 -- 172.21.15.79:0/2551795176 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c801c6c0 con 0x7f07d00a4750 2020-11-20T12:02:43.086 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.069+0000 7f07c6ffd700 1 -- 172.21.15.79:0/2551795176 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c801c840 con 0x7f07d00a4750 2020-11-20T12:02:43.086 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.069+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d009e470 con 0x7f07d00a4750 2020-11-20T12:02:43.086 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.069+0000 7f07c6ffd700 1 -- 172.21.15.79:0/2551795176 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c801cc00 con 0x7f07d00a4750 2020-11-20T12:02:43.086 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.069+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d01457d0 con 0x7f07d00a4750 2020-11-20T12:02:43.087 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.069+0000 7f07c6ffd700 1 -- 172.21.15.79:0/2551795176 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f07c8017070 con 0x7f07d00a4750 2020-11-20T12:02:43.087 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.069+0000 7f07c6ffd700 1 -- 172.21.15.79:0/2551795176 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f07c801de10 con 0x7f07d00a4750 2020-11-20T12:02:43.087 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.070+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d0068740 con 0x7f07d00a4750 2020-11-20T12:02:43.088 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.072+0000 7f07c6ffd700 1 -- 172.21.15.79:0/2551795176 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f07c8015020 con 0x7f07d00a4750 2020-11-20T12:02:43.168 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.164+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9495ddb8-00b5-4d73-acb2-86bf2949700f", "id": 5} v 0) v1 -- 0x7f07d00a0a80 con 0x7f07d00a4750 2020-11-20T12:02:43.171 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.167+0000 7f07c6ffd700 1 -- 172.21.15.79:0/2551795176 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9495ddb8-00b5-4d73-acb2-86bf2949700f", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f07c801d9e0 con 0x7f07d00a4750 2020-11-20T12:02:43.172 INFO:teuthology.orchestra.run.smithi079.stdout:5 2020-11-20T12:02:43.173 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00a4750 msgr2=0x7f07d00a32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.173 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00a4750 0x7f07d00a32b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f07c8011730 tx=0x7f07c8016000).stop 2020-11-20T12:02:43.173 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 shutdown_connections 2020-11-20T12:02:43.174 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07d013a550 0x7f07d009ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.174 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07d00a4750 0x7f07d00a32b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.174 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 --2- 172.21.15.79:0/2551795176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07d00ad890 0x7f07d009e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.174 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 >> 172.21.15.79:0/2551795176 conn(0x7f07d00619a0 msgr2=0x7f07d00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:43.174 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 shutdown_connections 2020-11-20T12:02:43.175 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.168+0000 7f07d520f700 1 -- 172.21.15.79:0/2551795176 wait complete. 2020-11-20T12:02:43.183 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 6502bfc3-d91f-407c-a2e7-83bf8472704a 6 2020-11-20T12:02:43.302 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.298+0000 7f640fe15700 1 Processor -- start 2020-11-20T12:02:43.303 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.298+0000 7f640fe15700 1 -- start start 2020-11-20T12:02:43.305 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640fe15700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ab050 0x7f64080abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.305 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640fe15700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f640813b590 0x7f64080a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.305 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640fe15700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f64080ac2e0 0x7f64080ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.306 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640fe15700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64080a8770 con 0x7f64080ac2e0 2020-11-20T12:02:43.306 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640fe15700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640813b9f0 con 0x7f640813b590 2020-11-20T12:02:43.306 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640fe15700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6408138480 con 0x7f64080ab050 2020-11-20T12:02:43.306 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f640813b590 0x7f64080a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.306 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f640813b590 0x7f64080a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43060/0 (socket says 172.21.15.79:43060) 2020-11-20T12:02:43.307 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 -- 172.21.15.79:0/2156522570 learned_addr learned my addr 172.21.15.79:0/2156522570 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:43.307 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640dbb1700 1 --2- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ab050 0x7f64080abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.307 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640e3b2700 1 --2- 172.21.15.79:0/2156522570 >> v2:172.21.15.44:3300/0 conn(0x7f64080ac2e0 0x7f64080ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.307 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 -- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ab050 msgr2=0x7f64080abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:43.307 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 --2- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ab050 0x7f64080abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.308 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 -- 172.21.15.79:0/2156522570 >> v2:172.21.15.44:3300/0 conn(0x7f64080ac2e0 msgr2=0x7f64080ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:43.308 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 --2- 172.21.15.79:0/2156522570 >> v2:172.21.15.44:3300/0 conn(0x7f64080ac2e0 0x7f64080ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.308 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640d3b0700 1 -- 172.21.15.79:0/2156522570 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640813abb0 con 0x7f640813b590 2020-11-20T12:02:43.308 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.299+0000 7f640e3b2700 1 --2- 172.21.15.79:0/2156522570 >> v2:172.21.15.44:3300/0 conn(0x7f64080ac2e0 0x7f64080ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:02:43.308 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640d3b0700 1 --2- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f64080a9410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f6408060ef0 tx=0x7f63f8006eb0).ready entity=mon.1 client_cookie=a02b4ddf756050f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:43.309 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640cbaf700 1 -- 172.21.15.79:0/2156522570 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80097d0 con 0x7f640813b590 2020-11-20T12:02:43.309 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640cbaf700 1 -- 172.21.15.79:0/2156522570 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f8009930 con 0x7f640813b590 2020-11-20T12:02:43.309 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640cbaf700 1 -- 172.21.15.79:0/2156522570 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8009cb0 con 0x7f640813b590 2020-11-20T12:02:43.309 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 -- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 msgr2=0x7f64080a9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 --2- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f64080a9410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f6408060ef0 tx=0x7f63f8006eb0).stop 2020-11-20T12:02:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 -- 172.21.15.79:0/2156522570 shutdown_connections 2020-11-20T12:02:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 --2- 172.21.15.79:0/2156522570 >> v2:172.21.15.44:3300/0 conn(0x7f64080ac2e0 0x7f64080ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 --2- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ab050 0x7f64080abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.310 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 --2- 172.21.15.79:0/2156522570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f64080a9410 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.311 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 -- 172.21.15.79:0/2156522570 >> 172.21.15.79:0/2156522570 conn(0x7f6408060170 msgr2=0x7f6408060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:43.311 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 -- 172.21.15.79:0/2156522570 shutdown_connections 2020-11-20T12:02:43.311 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.300+0000 7f640fe15700 1 -- 172.21.15.79:0/2156522570 wait complete. 2020-11-20T12:02:43.313 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.301+0000 7f640fe15700 1 Processor -- start 2020-11-20T12:02:43.313 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.301+0000 7f640fe15700 1 -- start start 2020-11-20T12:02:43.314 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640fe15700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64080ab050 0x7f64081402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.314 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640fe15700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ac2e0 0x7f640813f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.314 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640fe15700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f6408139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.314 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640fe15700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640813eda0 con 0x7f64080ab050 2020-11-20T12:02:43.315 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640fe15700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640813f990 con 0x7f640813b590 2020-11-20T12:02:43.315 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640fe15700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6408137630 con 0x7f64080ac2e0 2020-11-20T12:02:43.315 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640e3b2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f6408139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.315 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ac2e0 0x7f640813f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.315 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640e3b2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f6408139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43064/0 (socket says 172.21.15.79:43064) 2020-11-20T12:02:43.316 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ac2e0 0x7f640813f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.79:41384/0 (socket says 172.21.15.79:41384) 2020-11-20T12:02:43.316 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640e3b2700 1 -- 172.21.15.79:0/4276995584 learned_addr learned my addr 172.21.15.79:0/4276995584 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:43.316 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640dbb1700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64080ab050 0x7f64081402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.316 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 -- 172.21.15.79:0/4276995584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 msgr2=0x7f6408139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.316 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f6408139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.317 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 -- 172.21.15.79:0/4276995584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64080ab050 msgr2=0x7f64081402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.317 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64080ab050 0x7f64081402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.317 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 -- 172.21.15.79:0/4276995584 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640813f000 con 0x7f64080ac2e0 2020-11-20T12:02:43.317 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.302+0000 7f640d3b0700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ac2e0 0x7f640813f4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f64080aa920 tx=0x7f63f8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:43.317 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.303+0000 7f63f5ffb700 1 -- 172.21.15.79:0/4276995584 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8011c80 con 0x7f64080ac2e0 2020-11-20T12:02:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.303+0000 7f63f5ffb700 1 -- 172.21.15.79:0/4276995584 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f8011de0 con 0x7f64080ac2e0 2020-11-20T12:02:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.303+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640813e1b0 con 0x7f64080ac2e0 2020-11-20T12:02:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.303+0000 7f63f5ffb700 1 -- 172.21.15.79:0/4276995584 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f801d680 con 0x7f64080ac2e0 2020-11-20T12:02:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.303+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640813a0a0 con 0x7f64080ac2e0 2020-11-20T12:02:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.303+0000 7f63f5ffb700 1 -- 172.21.15.79:0/4276995584 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f63f800f070 con 0x7f64080ac2e0 2020-11-20T12:02:43.319 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.303+0000 7f63f5ffb700 1 -- 172.21.15.79:0/4276995584 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f63f801dd10 con 0x7f64080ac2e0 2020-11-20T12:02:43.319 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.304+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64080696f0 con 0x7f64080ac2e0 2020-11-20T12:02:43.319 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.308+0000 7f63f5ffb700 1 -- 172.21.15.79:0/4276995584 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f63f8027b50 con 0x7f64080ac2e0 2020-11-20T12:02:43.401 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.397+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6502bfc3-d91f-407c-a2e7-83bf8472704a", "id": 6} v 0) v1 -- 0x7f640813a630 con 0x7f64080ac2e0 2020-11-20T12:02:43.404 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.400+0000 7f63f5ffb700 1 -- 172.21.15.79:0/4276995584 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6502bfc3-d91f-407c-a2e7-83bf8472704a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f63f8027cf0 con 0x7f64080ac2e0 2020-11-20T12:02:43.405 INFO:teuthology.orchestra.run.smithi079.stdout:6 2020-11-20T12:02:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ac2e0 msgr2=0x7f640813f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ac2e0 0x7f640813f4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f64080aa920 tx=0x7f63f8015000).stop 2020-11-20T12:02:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 shutdown_connections 2020-11-20T12:02:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f640813b590 0x7f6408139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64080ab050 0x7f64081402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 --2- 172.21.15.79:0/4276995584 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64080ac2e0 0x7f640813f4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 >> 172.21.15.79:0/4276995584 conn(0x7f6408060170 msgr2=0x7f64080ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 shutdown_connections 2020-11-20T12:02:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.401+0000 7f640fe15700 1 -- 172.21.15.79:0/4276995584 wait complete. 2020-11-20T12:02:43.417 INFO:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 7d22fe22-d60a-468f-b9c6-bfe4d9f50c37 7 2020-11-20T12:02:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.532+0000 7ffa45865700 1 Processor -- start 2020-11-20T12:02:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.532+0000 7ffa45865700 1 -- start start 2020-11-20T12:02:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa45865700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ffa400a4750 0x7ffa4013a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa45865700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400a39f0 con 0x7ffa400ad890 2020-11-20T12:02:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa45865700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa400a9300 con 0x7ffa400a8c40 2020-11-20T12:02:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa45865700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa400a50a0 con 0x7ffa400a4750 2020-11-20T12:02:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3e7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ffa400a8c40 0x7ffa400a9f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53422/0 (socket says 172.21.15.79:53422) 2020-11-20T12:02:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3e7fc700 1 -- 172.21.15.79:0/855609456 learned_addr learned my addr 172.21.15.79:0/855609456 (peer_addr_for_me v1:172.21.15.79:0/0) 2020-11-20T12:02:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3f7fe700 1 --2- 172.21.15.79:0/855609456 >> v2:172.21.15.44:3300/0 conn(0x7ffa400a4750 0x7ffa4013a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239966743 0 0) 0x7ffa400a9300 con 0x7ffa400a8c40 2020-11-20T12:02:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa20003240 con 0x7ffa400a8c40 2020-11-20T12:02:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950837005 0 0) 0x7ffa400a39f0 con 0x7ffa400ad890 2020-11-20T12:02:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa400a9300 con 0x7ffa400ad890 2020-11-20T12:02:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1699675291 0 0) 0x7ffa20003240 con 0x7ffa400a8c40 2020-11-20T12:02:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 >> v2:172.21.15.44:3300/0 conn(0x7ffa400a4750 msgr2=0x7ffa4013a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 --2- 172.21.15.79:0/855609456 >> v2:172.21.15.44:3300/0 conn(0x7ffa400a4750 0x7ffa4013a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 >> v1:172.21.15.44:6789/0 conn(0x7ffa400ad890 legacy=0x7ffa400a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400aa690 con 0x7ffa400a8c40 2020-11-20T12:02:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7ffa280014e0 con 0x7ffa400a8c40 2020-11-20T12:02:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 >> v1:172.21.15.79:6789/0 conn(0x7ffa400a8c40 legacy=0x7ffa400a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.533+0000 7ffa3dffb700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa200038c0 0x7ffa20003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa200044e0 0x7ffa20004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa20004e50 0x7ffa2000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400a50a0 con 0x7ffa200038c0 2020-11-20T12:02:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2000bc20 con 0x7ffa20004e50 2020-11-20T12:02:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2000dff0 con 0x7ffa200044e0 2020-11-20T12:02:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3e7fc700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa200044e0 0x7ffa20004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3f7fe700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa20004e50 0x7ffa2000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3effd700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa200038c0 0x7ffa20003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3e7fc700 1 -- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa20004e50 msgr2=0x7ffa2000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3e7fc700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa20004e50 0x7ffa2000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3e7fc700 1 -- 172.21.15.79:0/855609456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa200038c0 msgr2=0x7ffa20003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3e7fc700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa200038c0 0x7ffa20003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3e7fc700 1 -- 172.21.15.79:0/855609456 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa20003240 con 0x7ffa200044e0 2020-11-20T12:02:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3f7fe700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa20004e50 0x7ffa2000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3f7fe700 1 -- 172.21.15.79:0/855609456 reap_dead start 2020-11-20T12:02:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3e7fc700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa200044e0 0x7ffa20004960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7ffa2800c440 tx=0x7ffa40137000).ready entity=mon.2 client_cookie=c8eac9fdb93e7a5b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800cea0 con 0x7ffa200044e0 2020-11-20T12:02:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa280070d0 con 0x7ffa200044e0 2020-11-20T12:02:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.534+0000 7ffa3dffb700 1 -- 172.21.15.79:0/855609456 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28007490 con 0x7ffa200044e0 2020-11-20T12:02:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.535+0000 7ffa45865700 1 -- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa200044e0 msgr2=0x7ffa20004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.535+0000 7ffa45865700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa200044e0 0x7ffa20004960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7ffa2800c440 tx=0x7ffa40137000).stop 2020-11-20T12:02:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.535+0000 7ffa45865700 1 -- 172.21.15.79:0/855609456 shutdown_connections 2020-11-20T12:02:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.535+0000 7ffa45865700 1 --2- 172.21.15.79:0/855609456 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa200044e0 0x7ffa20004960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.535+0000 7ffa45865700 1 -- 172.21.15.79:0/855609456 >> 172.21.15.79:0/855609456 conn(0x7ffa400619a0 msgr2=0x7ffa40061db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.535+0000 7ffa45865700 1 -- 172.21.15.79:0/855609456 shutdown_connections 2020-11-20T12:02:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.535+0000 7ffa45865700 1 -- 172.21.15.79:0/855609456 wait complete. 2020-11-20T12:02:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.537+0000 7ffa45865700 1 Processor -- start 2020-11-20T12:02:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.537+0000 7ffa45865700 1 -- start start 2020-11-20T12:02:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa45865700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa200044e0 0x7ffa4009f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa45865700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa400a3500 0x7ffa40067640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa45865700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa40067b30 0x7ffa40146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa45865700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40145fb0 con 0x7ffa40067b30 2020-11-20T12:02:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa45865700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4009de60 con 0x7ffa200044e0 2020-11-20T12:02:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa45865700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4009e100 con 0x7ffa400a3500 2020-11-20T12:02:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa200044e0 0x7ffa4009f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3e7fc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa400a3500 0x7ffa40067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa200044e0 0x7ffa4009f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:43082/0 (socket says 172.21.15.79:43082) 2020-11-20T12:02:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 -- 172.21.15.79:0/301431914 learned_addr learned my addr 172.21.15.79:0/301431914 (peer_addr_for_me v2:172.21.15.79:0/0) 2020-11-20T12:02:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3f7fe700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa40067b30 0x7ffa40146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 -- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa400a3500 msgr2=0x7ffa40067640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa400a3500 0x7ffa40067640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 -- 172.21.15.79:0/301431914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa40067b30 msgr2=0x7ffa40146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa40067b30 0x7ffa40146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 -- 172.21.15.79:0/301431914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa401381e0 con 0x7ffa200044e0 2020-11-20T12:02:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3e7fc700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa400a3500 0x7ffa40067640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa3effd700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa200044e0 0x7ffa4009f920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ffa3800ec40 tx=0x7ffa38007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa36ffd700 1 -- 172.21.15.79:0/301431914 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800c9d0 con 0x7ffa200044e0 2020-11-20T12:02:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa36ffd700 1 -- 172.21.15.79:0/301431914 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3800cb30 con 0x7ffa200044e0 2020-11-20T12:02:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.538+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa40066740 con 0x7ffa200044e0 2020-11-20T12:02:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.539+0000 7ffa36ffd700 1 -- 172.21.15.79:0/301431914 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38002030 con 0x7ffa200044e0 2020-11-20T12:02:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.539+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa401469e0 con 0x7ffa200044e0 2020-11-20T12:02:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.539+0000 7ffa36ffd700 1 -- 172.21.15.79:0/301431914 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7ffa38001750 con 0x7ffa200044e0 2020-11-20T12:02:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.539+0000 7ffa36ffd700 1 -- 172.21.15.79:0/301431914 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7ffa38013ab0 con 0x7ffa200044e0 2020-11-20T12:02:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.539+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa40067f10 con 0x7ffa200044e0 2020-11-20T12:02:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.541+0000 7ffa36ffd700 1 -- 172.21.15.79:0/301431914 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7ffa38002640 con 0x7ffa200044e0 2020-11-20T12:02:43.636 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.632+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7d22fe22-d60a-468f-b9c6-bfe4d9f50c37", "id": 7} v 0) v1 -- 0x7ffa4009dca0 con 0x7ffa200044e0 2020-11-20T12:02:43.643 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.639+0000 7ffa36ffd700 1 -- 172.21.15.79:0/301431914 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d22fe22-d60a-468f-b9c6-bfe4d9f50c37", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffa38013d40 con 0x7ffa200044e0 2020-11-20T12:02:43.643 INFO:teuthology.orchestra.run.smithi079.stdout:7 2020-11-20T12:02:43.646 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.639+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa200044e0 msgr2=0x7ffa4009f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:43.646 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.639+0000 7ffa45865700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa200044e0 0x7ffa4009f920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ffa3800ec40 tx=0x7ffa38007f10).stop 2020-11-20T12:02:43.647 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.640+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 shutdown_connections 2020-11-20T12:02:43.647 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.640+0000 7ffa45865700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa40067b30 0x7ffa40146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.647 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.640+0000 7ffa45865700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa200044e0 0x7ffa4009f920 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.648 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.640+0000 7ffa45865700 1 --2- 172.21.15.79:0/301431914 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa400a3500 0x7ffa40067640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:43.648 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.640+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 >> 172.21.15.79:0/301431914 conn(0x7ffa400619a0 msgr2=0x7ffa40133010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:43.649 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.640+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 shutdown_connections 2020-11-20T12:02:43.649 INFO:teuthology.orchestra.run.smithi079.stderr:2020-11-20T12:02:43.640+0000 7ffa45865700 1 -- 172.21.15.79:0/301431914 wait complete. 2020-11-20T12:02:43.655 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-20T12:02:43.655 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-20T12:02:43.658 INFO:tasks.ceph.osd.0:Started 2020-11-20T12:02:43.659 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-20T12:02:43.659 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-20T12:02:43.662 INFO:tasks.ceph.osd.1:Started 2020-11-20T12:02:43.662 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-20T12:02:43.663 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-20T12:02:43.665 INFO:tasks.ceph.osd.2:Started 2020-11-20T12:02:43.666 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-20T12:02:43.666 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-20T12:02:43.669 INFO:tasks.ceph.osd.3:Started 2020-11-20T12:02:43.669 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-20T12:02:43.670 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-20T12:02:43.672 INFO:tasks.ceph.osd.4:Started 2020-11-20T12:02:43.673 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-20T12:02:43.673 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-20T12:02:43.675 INFO:tasks.ceph.osd.5:Started 2020-11-20T12:02:43.676 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-20T12:02:43.676 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-20T12:02:43.679 INFO:tasks.ceph.osd.6:Started 2020-11-20T12:02:43.679 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-20T12:02:43.680 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-20T12:02:43.682 INFO:tasks.ceph.osd.7:Started 2020-11-20T12:02:43.683 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-20T12:02:44.028 INFO:tasks.ceph.osd.2.smithi044.stderr:2020-11-20T12:02:44.024+0000 7f62cfd07ec0 -1 Falling back to public interface 2020-11-20T12:02:44.273 INFO:tasks.ceph.osd.5.smithi079.stderr:2020-11-20T12:02:44.269+0000 7feeef6f2ec0 -1 Falling back to public interface 2020-11-20T12:02:44.285 INFO:tasks.ceph.osd.1.smithi044.stderr:2020-11-20T12:02:44.281+0000 7f886d842ec0 -1 Falling back to public interface 2020-11-20T12:02:44.286 INFO:tasks.ceph.osd.0.smithi044.stderr:2020-11-20T12:02:44.281+0000 7f250e3a0ec0 -1 Falling back to public interface 2020-11-20T12:02:44.287 INFO:tasks.ceph.osd.6.smithi079.stderr:2020-11-20T12:02:44.281+0000 7f7605dd2ec0 -1 Falling back to public interface 2020-11-20T12:02:44.288 INFO:tasks.ceph.osd.0.smithi044.stderr:2020-11-20T12:02:44.284+0000 7f25083bd700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6815/0: (98) Address already in use 2020-11-20T12:02:44.293 INFO:tasks.ceph.osd.4.smithi079.stderr:2020-11-20T12:02:44.289+0000 7fa88df61ec0 -1 Falling back to public interface 2020-11-20T12:02:44.307 INFO:tasks.ceph.osd.7.smithi079.stderr:2020-11-20T12:02:44.303+0000 7f5df8477ec0 -1 Falling back to public interface 2020-11-20T12:02:44.308 INFO:tasks.ceph.osd.3.smithi044.stderr:2020-11-20T12:02:44.304+0000 7f5ce30e4ec0 -1 Falling back to public interface 2020-11-20T12:02:44.491 INFO:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T12:02:44.613 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 Processor -- start 2020-11-20T12:02:44.614 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 -- start start 2020-11-20T12:02:44.614 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f00dc0b12c0 0x7f00dc0ac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:44.614 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0a8630 0x7f00dc0aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:44.614 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f00dc0aada0 0x7f00dc13e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:44.615 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00dc0a9e70 con 0x7f00dc0aada0 2020-11-20T12:02:44.615 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00dc0aa110 con 0x7f00dc0b12c0 2020-11-20T12:02:44.615 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.608+0000 7f00e2e91700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0aa3b0 con 0x7f00dc0a8630 2020-11-20T12:02:44.615 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00e0c2d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f00dc0b12c0 0x7f00dc0ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:44.616 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00e142e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f00dc0aada0 0x7f00dc13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:44.616 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00dbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0a8630 0x7f00dc0aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:44.616 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00e0c2d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f00dc0b12c0 0x7f00dc0ac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:55828/0 (socket says 172.21.15.44:55828) 2020-11-20T12:02:44.616 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00e0c2d700 1 -- 172.21.15.44:0/2099304911 learned_addr learned my addr 172.21.15.44:0/2099304911 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:44.616 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00e142e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f00dc0aada0 0x7f00dc13e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35550/0 (socket says 172.21.15.44:35550) 2020-11-20T12:02:44.617 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00dbfff700 1 -- 172.21.15.44:0/2099304911 >> v2:172.21.15.79:3300/0 conn(0x7f00dc0b12c0 msgr2=0x7f00dc0ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:44.617 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00dbfff700 1 --2- 172.21.15.44:0/2099304911 >> v2:172.21.15.79:3300/0 conn(0x7f00dc0b12c0 0x7f00dc0ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.617 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00dbfff700 1 -- 172.21.15.44:0/2099304911 >> v2:172.21.15.44:3300/0 conn(0x7f00dc0aada0 msgr2=0x7f00dc13e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:44.617 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00dbfff700 1 --2- 172.21.15.44:0/2099304911 >> v2:172.21.15.44:3300/0 conn(0x7f00dc0aada0 0x7f00dc13e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.617 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00dbfff700 1 -- 172.21.15.44:0/2099304911 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc0ab230 con 0x7f00dc0a8630 2020-11-20T12:02:44.618 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00e0c2d700 1 --2- 172.21.15.44:0/2099304911 >> v2:172.21.15.79:3300/0 conn(0x7f00dc0b12c0 0x7f00dc0ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:44.618 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00dbfff700 1 --2- 172.21.15.44:0/2099304911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0a8630 0x7f00dc0aa8b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f00dc062370 tx=0x7f00c8006df0).ready entity=mon.2 client_cookie=98c44a0f1aa57d94 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:44.618 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00db7fe700 1 -- 172.21.15.44:0/2099304911 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8009640 con 0x7f00dc0a8630 2020-11-20T12:02:44.618 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00db7fe700 1 -- 172.21.15.44:0/2099304911 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c80097a0 con 0x7f00dc0a8630 2020-11-20T12:02:44.619 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.609+0000 7f00db7fe700 1 -- 172.21.15.44:0/2099304911 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8009b20 con 0x7f00dc0a8630 2020-11-20T12:02:44.622 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.610+0000 7f00e2e91700 1 -- 172.21.15.44:0/2099304911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0a8630 msgr2=0x7f00dc0aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:44.622 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.610+0000 7f00e2e91700 1 --2- 172.21.15.44:0/2099304911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0a8630 0x7f00dc0aa8b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f00dc062370 tx=0x7f00c8006df0).stop 2020-11-20T12:02:44.623 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.610+0000 7f00e2e91700 1 -- 172.21.15.44:0/2099304911 shutdown_connections 2020-11-20T12:02:44.623 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.610+0000 7f00e2e91700 1 --2- 172.21.15.44:0/2099304911 >> v2:172.21.15.44:3300/0 conn(0x7f00dc0aada0 0x7f00dc13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.624 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.610+0000 7f00e2e91700 1 --2- 172.21.15.44:0/2099304911 >> v2:172.21.15.79:3300/0 conn(0x7f00dc0b12c0 0x7f00dc0ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.624 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.610+0000 7f00e2e91700 1 --2- 172.21.15.44:0/2099304911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0a8630 0x7f00dc0aa8b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.625 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.610+0000 7f00e2e91700 1 -- 172.21.15.44:0/2099304911 >> 172.21.15.44:0/2099304911 conn(0x7f00dc061810 msgr2=0x7f00dc061c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:44.625 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.611+0000 7f00e2e91700 1 -- 172.21.15.44:0/2099304911 shutdown_connections 2020-11-20T12:02:44.626 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.611+0000 7f00e2e91700 1 -- 172.21.15.44:0/2099304911 wait complete. 2020-11-20T12:02:44.626 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 Processor -- start 2020-11-20T12:02:44.626 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 -- start start 2020-11-20T12:02:44.626 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00dc0a8630 0x7f00dc13fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:44.626 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0aada0 0x7f00dc149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:44.627 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00dc0b12c0 0x7f00dc149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:44.627 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc13d630 con 0x7f00dc0b12c0 2020-11-20T12:02:44.627 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc13cc50 con 0x7f00dc0a8630 2020-11-20T12:02:44.627 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.612+0000 7f00e2e91700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc13cef0 con 0x7f00dc0aada0 2020-11-20T12:02:44.628 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e142e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00dc0b12c0 0x7f00dc149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:44.628 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00dc0a8630 0x7f00dc13fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:44.628 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00dbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0aada0 0x7f00dc149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:44.628 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00dc0a8630 0x7f00dc13fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:55834/0 (socket says 172.21.15.44:55834) 2020-11-20T12:02:44.628 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e142e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00dc0b12c0 0x7f00dc149f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35556/0 (socket says 172.21.15.44:35556) 2020-11-20T12:02:44.629 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 -- 172.21.15.44:0/1551914175 learned_addr learned my addr 172.21.15.44:0/1551914175 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:44.629 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 -- 172.21.15.44:0/1551914175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0aada0 msgr2=0x7f00dc149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:44.629 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 --2- 172.21.15.44:0/1551914175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0aada0 0x7f00dc149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.629 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 -- 172.21.15.44:0/1551914175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00dc0b12c0 msgr2=0x7f00dc149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:44.629 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 --2- 172.21.15.44:0/1551914175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00dc0b12c0 0x7f00dc149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.630 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 -- 172.21.15.44:0/1551914175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc13d190 con 0x7f00dc0a8630 2020-11-20T12:02:44.630 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e0c2d700 1 --2- 172.21.15.44:0/1551914175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00dc0a8630 0x7f00dc13fb40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f00d000ed40 tx=0x7f00d0006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:44.630 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00d8ff9700 1 -- 172.21.15.44:0/1551914175 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d000cb90 con 0x7f00dc0a8630 2020-11-20T12:02:44.630 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00d8ff9700 1 -- 172.21.15.44:0/1551914175 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d000ccf0 con 0x7f00dc0a8630 2020-11-20T12:02:44.630 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc13c8d0 con 0x7f00dc0a8630 2020-11-20T12:02:44.631 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00d8ff9700 1 -- 172.21.15.44:0/1551914175 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0012600 con 0x7f00dc0a8630 2020-11-20T12:02:44.631 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.613+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc13e1f0 con 0x7f00dc0a8630 2020-11-20T12:02:44.631 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.614+0000 7f00d8ff9700 1 -- 172.21.15.44:0/1551914175 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 2) v1 ==== 33694+0+0 (secure 0 0 0) 0x7f00d0022070 con 0x7f00dc0a8630 2020-11-20T12:02:44.631 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.614+0000 7f00d8ff9700 1 -- 172.21.15.44:0/1551914175 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f00d003d030 con 0x7f00dc0a8630 2020-11-20T12:02:44.631 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.615+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc13cac0 con 0x7f00dc0a8630 2020-11-20T12:02:44.632 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.618+0000 7f00d8ff9700 1 -- 172.21.15.44:0/1551914175 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f00d0019900 con 0x7f00dc0a8630 2020-11-20T12:02:44.698 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.695+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00dc044f60 con 0x7f00dc0a8630 2020-11-20T12:02:44.699 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.695+0000 7f00d8ff9700 1 -- 172.21.15.44:0/1551914175 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7001 (secure 0 0 0) 0x7f00d002a090 con 0x7f00dc0a8630 2020-11-20T12:02:44.699 INFO:teuthology.misc.health.smithi044.stdout: 2020-11-20T12:02:44.700 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":10,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:02:43.634392+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:02:44.701 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00dc0a8630 msgr2=0x7f00dc13fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:44.701 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 --2- 172.21.15.44:0/1551914175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00dc0a8630 0x7f00dc13fb40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f00d000ed40 tx=0x7f00d0006eb0).stop 2020-11-20T12:02:44.701 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 shutdown_connections 2020-11-20T12:02:44.701 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 --2- 172.21.15.44:0/1551914175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00dc0b12c0 0x7f00dc149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.702 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 --2- 172.21.15.44:0/1551914175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00dc0a8630 0x7f00dc13fb40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.702 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 --2- 172.21.15.44:0/1551914175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f00dc0aada0 0x7f00dc149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:44.702 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 >> 172.21.15.44:0/1551914175 conn(0x7f00dc061810 msgr2=0x7f00dc0aa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:44.702 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 shutdown_connections 2020-11-20T12:02:44.702 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:44.696+0000 7f00e2e91700 1 -- 172.21.15.44:0/1551914175 wait complete. 2020-11-20T12:02:44.710 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-20T12:02:45.363 INFO:tasks.ceph.osd.2.smithi044.stderr:2020-11-20T12:02:45.359+0000 7f62cfd07ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-11-20T12:02:45.602 INFO:tasks.ceph.osd.5.smithi079.stderr:2020-11-20T12:02:45.598+0000 7feeef6f2ec0 -1 osd.5 0 log_to_monitors {default=true} 2020-11-20T12:02:45.614 INFO:tasks.ceph.osd.6.smithi079.stderr:2020-11-20T12:02:45.609+0000 7f7605dd2ec0 -1 osd.6 0 log_to_monitors {default=true} 2020-11-20T12:02:45.623 INFO:tasks.ceph.osd.0.smithi044.stderr:2020-11-20T12:02:45.619+0000 7f250e3a0ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-11-20T12:02:45.626 INFO:tasks.ceph.osd.1.smithi044.stderr:2020-11-20T12:02:45.623+0000 7f886d842ec0 -1 osd.1 0 log_to_monitors {default=true} 2020-11-20T12:02:45.627 INFO:tasks.ceph.osd.4.smithi079.stderr:2020-11-20T12:02:45.622+0000 7fa88df61ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-11-20T12:02:45.638 INFO:tasks.ceph.osd.7.smithi079.stderr:2020-11-20T12:02:45.634+0000 7f5df8477ec0 -1 osd.7 0 log_to_monitors {default=true} 2020-11-20T12:02:45.660 INFO:tasks.ceph.osd.3.smithi044.stderr:2020-11-20T12:02:45.657+0000 7f5ce30e4ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-11-20T12:02:46.586 INFO:tasks.ceph.osd.2.smithi044.stderr:2020-11-20T12:02:46.582+0000 7f62c651d700 -1 osd.2 0 waiting for initial osdmap 2020-11-20T12:02:47.588 INFO:tasks.ceph.osd.6.smithi079.stderr:2020-11-20T12:02:47.583+0000 7f75fc5e8700 -1 osd.6 0 waiting for initial osdmap 2020-11-20T12:02:47.588 INFO:tasks.ceph.osd.0.smithi044.stderr:2020-11-20T12:02:47.585+0000 7f2504bb6700 -1 osd.0 0 waiting for initial osdmap 2020-11-20T12:02:47.589 INFO:tasks.ceph.osd.1.smithi044.stderr:2020-11-20T12:02:47.585+0000 7f8864058700 -1 osd.1 0 waiting for initial osdmap 2020-11-20T12:02:47.592 INFO:tasks.ceph.osd.5.smithi079.stderr:2020-11-20T12:02:47.587+0000 7feee5f08700 -1 osd.5 0 waiting for initial osdmap 2020-11-20T12:02:47.592 INFO:tasks.ceph.osd.4.smithi079.stderr:2020-11-20T12:02:47.588+0000 7fa884777700 -1 osd.4 0 waiting for initial osdmap 2020-11-20T12:02:47.595 INFO:tasks.ceph.osd.3.smithi044.stderr:2020-11-20T12:02:47.591+0000 7f5cd98fa700 -1 osd.3 0 waiting for initial osdmap 2020-11-20T12:02:47.595 INFO:tasks.ceph.osd.7.smithi079.stderr:2020-11-20T12:02:47.591+0000 7f5deec8d700 -1 osd.7 0 waiting for initial osdmap 2020-11-20T12:02:51.517 INFO:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T12:02:51.648 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.644+0000 7f778d568700 1 Processor -- start 2020-11-20T12:02:51.648 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.644+0000 7f778d568700 1 -- start start 2020-11-20T12:02:51.650 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f778d568700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f77880aaac0 0x7f7788137d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.650 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f778d568700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7788064a20 con 0x7f778813b800 2020-11-20T12:02:51.650 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f778d568700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77880aa690 con 0x7f77880af0e0 2020-11-20T12:02:51.650 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f778d568700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77880aafd0 con 0x7f77880aaac0 2020-11-20T12:02:51.651 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f77877fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f77880aaac0 0x7f7788137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.651 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f77877fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f77880aaac0 0x7f7788137d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35782/0 (socket says 172.21.15.44:35782) 2020-11-20T12:02:51.651 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f77867fc700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f778813b800 0x7f77881375f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39526/0 (socket says 172.21.15.44:39526) 2020-11-20T12:02:51.651 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f77877fe700 1 -- 172.21.15.44:0/1259535719 learned_addr learned my addr 172.21.15.44:0/1259535719 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:51.651 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f77877fe700 1 -- 172.21.15.44:0/1259535719 >> v1:172.21.15.79:6789/0 conn(0x7f77880af0e0 legacy=0x7f778813b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.652 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f77877fe700 1 -- 172.21.15.44:0/1259535719 >> v1:172.21.15.44:6789/0 conn(0x7f778813b800 legacy=0x7f77881375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.652 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f77877fe700 1 -- 172.21.15.44:0/1259535719 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778813ac50 con 0x7f77880aaac0 2020-11-20T12:02:51.652 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.645+0000 7f7785ffb700 1 -- 172.21.15.44:0/1259535719 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644343634 0 0) 0x7f7788064a20 con 0x7f778813b800 2020-11-20T12:02:51.652 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f77877fe700 1 --2- 172.21.15.44:0/1259535719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880aaac0 0x7f7788137d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7780009d50 tx=0x7f7780009290).ready entity=mon.0 client_cookie=9ff9e0de1c4f4b75 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:51.652 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f7785ffb700 1 -- 172.21.15.44:0/1259535719 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000bc80 con 0x7f77880aaac0 2020-11-20T12:02:51.655 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f7785ffb700 1 -- 172.21.15.44:0/1259535719 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778000bde0 con 0x7f77880aaac0 2020-11-20T12:02:51.655 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f7785ffb700 1 -- 172.21.15.44:0/1259535719 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780011610 con 0x7f77880aaac0 2020-11-20T12:02:51.656 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f778d568700 1 -- 172.21.15.44:0/1259535719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880aaac0 msgr2=0x7f7788137d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.656 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f778d568700 1 --2- 172.21.15.44:0/1259535719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880aaac0 0x7f7788137d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7780009d50 tx=0x7f7780009290).stop 2020-11-20T12:02:51.656 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f778d568700 1 -- 172.21.15.44:0/1259535719 shutdown_connections 2020-11-20T12:02:51.656 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f778d568700 1 --2- 172.21.15.44:0/1259535719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880aaac0 0x7f7788137d00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.656 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f778d568700 1 -- 172.21.15.44:0/1259535719 >> 172.21.15.44:0/1259535719 conn(0x7f7788061a20 msgr2=0x7f7788061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:51.657 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f778d568700 1 -- 172.21.15.44:0/1259535719 shutdown_connections 2020-11-20T12:02:51.657 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.646+0000 7f778d568700 1 -- 172.21.15.44:0/1259535719 wait complete. 2020-11-20T12:02:51.657 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.647+0000 7f778d568700 1 Processor -- start 2020-11-20T12:02:51.657 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f778d568700 1 -- start start 2020-11-20T12:02:51.658 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f778d568700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f77880aaac0 0x7f7788063980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.658 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f778d568700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880af0e0 0x7f778813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.658 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f778d568700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778813b800 0x7f778813df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.658 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f778d568700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778814a940 con 0x7f77880af0e0 2020-11-20T12:02:51.658 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f778d568700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7788149f60 con 0x7f778813b800 2020-11-20T12:02:51.659 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f778d568700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778814a200 con 0x7f77880aaac0 2020-11-20T12:02:51.659 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f77867fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880af0e0 0x7f778813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.659 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f77867fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880af0e0 0x7f778813da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35786/0 (socket says 172.21.15.44:35786) 2020-11-20T12:02:51.659 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f77867fc700 1 -- 172.21.15.44:0/4081964882 learned_addr learned my addr 172.21.15.44:0/4081964882 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:51.659 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f77877fe700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778813b800 0x7f778813df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.660 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.648+0000 7f7786ffd700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f77880aaac0 0x7f7788063980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.660 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f77867fc700 1 -- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f77880aaac0 msgr2=0x7f7788063980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.660 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f77867fc700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f77880aaac0 0x7f7788063980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.660 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f77867fc700 1 -- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778813b800 msgr2=0x7f778813df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.660 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f77867fc700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778813b800 0x7f778813df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.661 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f77867fc700 1 -- 172.21.15.44:0/4081964882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778814a4a0 con 0x7f77880af0e0 2020-11-20T12:02:51.661 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f77867fc700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880af0e0 0x7f778813da80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f777800c730 tx=0x7f7778007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:51.661 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f776effd700 1 -- 172.21.15.44:0/4081964882 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780072e0 con 0x7f77880af0e0 2020-11-20T12:02:51.661 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f776effd700 1 -- 172.21.15.44:0/4081964882 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7778005530 con 0x7f77880af0e0 2020-11-20T12:02:51.661 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7788064080 con 0x7f77880af0e0 2020-11-20T12:02:51.662 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f776effd700 1 -- 172.21.15.44:0/4081964882 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780058f0 con 0x7f77880af0e0 2020-11-20T12:02:51.662 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.649+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778813d7a0 con 0x7f77880af0e0 2020-11-20T12:02:51.662 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.650+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778814a630 con 0x7f77880af0e0 2020-11-20T12:02:51.662 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.650+0000 7f776effd700 1 -- 172.21.15.44:0/4081964882 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7778010070 con 0x7f77880af0e0 2020-11-20T12:02:51.662 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.650+0000 7f776effd700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7764056d20 0x7f77640591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.663 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.650+0000 7f776effd700 1 -- 172.21.15.44:0/4081964882 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f777806f0a0 con 0x7f77880af0e0 2020-11-20T12:02:51.663 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.650+0000 7f7786ffd700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7764056d20 0x7f77640591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.663 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.651+0000 7f7786ffd700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7764056d20 0x7f77640591e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f778813c880 tx=0x7f777c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:51.664 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.654+0000 7f776effd700 1 -- 172.21.15.44:0/4081964882 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7778044980 con 0x7f77880af0e0 2020-11-20T12:02:51.836 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.833+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7788044f60 con 0x7f77880af0e0 2020-11-20T12:02:51.837 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.833+0000 7f776effd700 1 -- 172.21.15.44:0/4081964882 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12105 (secure 0 0 0) 0x7f7778007d70 con 0x7f77880af0e0 2020-11-20T12:02:51.838 INFO:teuthology.misc.health.smithi044.stdout: 2020-11-20T12:02:51.838 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":16,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:02:50.600073+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:02:51.839 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7764056d20 msgr2=0x7f77640591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.840 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7764056d20 0x7f77640591e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f778813c880 tx=0x7f777c009040).stop 2020-11-20T12:02:51.840 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880af0e0 msgr2=0x7f778813da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.840 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880af0e0 0x7f778813da80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f777800c730 tx=0x7f7778007c30).stop 2020-11-20T12:02:51.840 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 shutdown_connections 2020-11-20T12:02:51.840 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7764056d20 0x7f77640591e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.841 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f778813b800 0x7f778813df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.841 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f77880aaac0 0x7f7788063980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.841 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 --2- 172.21.15.44:0/4081964882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77880af0e0 0x7f778813da80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.841 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 >> 172.21.15.44:0/4081964882 conn(0x7f7788061a20 msgr2=0x7f778805bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:51.842 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 shutdown_connections 2020-11-20T12:02:51.842 INFO:teuthology.misc.health.smithi044.stderr:2020-11-20T12:02:51.835+0000 7f778d568700 1 -- 172.21.15.44:0/4081964882 wait complete. 2020-11-20T12:02:51.849 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-20T12:02:51.849 INFO:tasks.ceph:Creating RBD pool 2020-11-20T12:02:51.850 INFO:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-20T12:02:51.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7c11ed700 1 Processor -- start 2020-11-20T12:02:51.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7c11ed700 1 -- start start 2020-11-20T12:02:51.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7c11ed700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb7bc0a8b20 0x7fb7bc0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7c11ed700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc13af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7c11ed700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7bc0a4070 con 0x7fb7bc0a34d0 2020-11-20T12:02:51.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7c11ed700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc0a5c00 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7c11ed700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0a7120 con 0x7fb7bc0a6ba0 2020-11-20T12:02:51.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7bad9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb7bc0a8b20 0x7fb7bc0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7bad9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb7bc0a8b20 0x7fb7bc0a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35792/0 (socket says 172.21.15.44:35792) 2020-11-20T12:02:51.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.976+0000 7fb7bad9d700 1 -- 172.21.15.44:0/2578788401 learned_addr learned my addr 172.21.15.44:0/2578788401 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:51.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7ba59c700 1 --2- 172.21.15.44:0/2578788401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc13af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7bad9d700 1 -- 172.21.15.44:0/2578788401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 msgr2=0x7fb7bc13af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:51.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7bad9d700 1 --2- 172.21.15.44:0/2578788401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc13af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7bad9d700 1 -- 172.21.15.44:0/2578788401 >> v1:172.21.15.79:6789/0 conn(0x7fb7bc0a34d0 legacy=0x7fb7bc0a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7bad9d700 1 -- 172.21.15.44:0/2578788401 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc0a4fb0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7bad9d700 1 --2- 172.21.15.44:0/2578788401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a5450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fb7ac006fd0 tx=0x7fb7ac006f40).ready entity=mon.0 client_cookie=7073b018b48f2c2c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:51.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7b9d9b700 1 -- 172.21.15.44:0/2578788401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00ddd0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7b9d9b700 1 -- 172.21.15.44:0/2578788401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac010470 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7b9d9b700 1 -- 172.21.15.44:0/2578788401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0107f0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2578788401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 msgr2=0x7fb7bc0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2578788401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a5450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fb7ac006fd0 tx=0x7fb7ac006f40).stop 2020-11-20T12:02:51.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2578788401 shutdown_connections 2020-11-20T12:02:51.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2578788401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a5450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2578788401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc13af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.977+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2578788401 >> 172.21.15.44:0/2578788401 conn(0x7fb7bc0aceb0 msgr2=0x7fb7bc0ad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:51.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.978+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2578788401 shutdown_connections 2020-11-20T12:02:51.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.978+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2578788401 wait complete. 2020-11-20T12:02:51.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 Processor -- start 2020-11-20T12:02:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 -- start start 2020-11-20T12:02:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7bc0a34d0 0x7fb7bc13a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc146240 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc146460 con 0x7fb7bc0a34d0 2020-11-20T12:02:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7c11ed700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc146700 con 0x7fb7bc0a6ba0 2020-11-20T12:02:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7bb59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7bad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7bc0a34d0 0x7fb7bc13a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7ba59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7bb59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35802/0 (socket says 172.21.15.44:35802) 2020-11-20T12:02:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7bad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7bc0a34d0 0x7fb7bc13a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:56080/0 (socket says 172.21.15.44:56080) 2020-11-20T12:02:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.979+0000 7fb7bb59e700 1 -- 172.21.15.44:0/2360533151 learned_addr learned my addr 172.21.15.44:0/2360533151 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:51.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7bb59e700 1 -- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 msgr2=0x7fb7bc145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7bb59e700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7bb59e700 1 -- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7bc0a34d0 msgr2=0x7fb7bc13a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7bb59e700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7bc0a34d0 0x7fb7bc13a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7bb59e700 1 -- 172.21.15.44:0/2360533151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc1469a0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7bb59e700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a1250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb7b400ca60 tx=0x7fb7b4007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:51.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7b2ffd700 1 -- 172.21.15.44:0/2360533151 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400a770 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7b2ffd700 1 -- 172.21.15.44:0/2360533151 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b400a8d0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7b2ffd700 1 -- 172.21.15.44:0/2360533151 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400ac50 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc0a1740 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.980+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc0a2cd0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.981+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7bc0a18d0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.981+0000 7fb7b2ffd700 1 -- 172.21.15.44:0/2360533151 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb7b4019030 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.981+0000 7fb7b2ffd700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb798056d20 0x7fb7980591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.981+0000 7fb7b2ffd700 1 -- 172.21.15.44:0/2360533151 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fb7b406f1b0 con 0x7fb7bc0a8b20 2020-11-20T12:02:51.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.982+0000 7fb7bad9d700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb798056d20 0x7fb7980591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:51.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.982+0000 7fb7bad9d700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb798056d20 0x7fb7980591e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb7bc143990 tx=0x7fb7bc067e80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:51.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:51.985+0000 7fb7b2ffd700 1 -- 172.21.15.44:0/2360533151 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb7b4043fd0 con 0x7fb7bc0a8b20 2020-11-20T12:02:52.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.159+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb7bc0615e0 con 0x7fb7bc0a8b20 2020-11-20T12:02:52.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.611+0000 7fb7b2ffd700 1 -- 172.21.15.44:0/2360533151 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb7b4044b40 con 0x7fb7bc0a8b20 2020-11-20T12:02:52.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb798056d20 msgr2=0x7fb7980591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:52.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb798056d20 0x7fb7980591e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb7bc143990 tx=0x7fb7bc067e80).stop 2020-11-20T12:02:52.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 msgr2=0x7fb7bc0a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:52.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a1250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb7b400ca60 tx=0x7fb7b4007f40).stop 2020-11-20T12:02:52.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 shutdown_connections 2020-11-20T12:02:52.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb798056d20 0x7fb7980591e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7bc0a8b20 0x7fb7bc0a1250 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7bc0a34d0 0x7fb7bc13a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 --2- 172.21.15.44:0/2360533151 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb7bc0a6ba0 0x7fb7bc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 >> 172.21.15.44:0/2360533151 conn(0x7fb7bc0aceb0 msgr2=0x7fb7bc13bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.615+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 shutdown_connections 2020-11-20T12:02:52.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.616+0000 7fb7c11ed700 1 -- 172.21.15.44:0/2360533151 wait complete. 2020-11-20T12:02:52.622 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'rbd' created 2020-11-20T12:02:52.633 INFO:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-20T12:02:52.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.766+0000 7fc05428d700 1 Processor -- start 2020-11-20T12:02:52.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.766+0000 7fc05428d700 1 -- start start 2020-11-20T12:02:52.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.766+0000 7fc05428d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc04c0ad550 0x7fc04c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:52.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.766+0000 7fc05428d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc04c0a8ea0 0x7fc04c1420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:52.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.766+0000 7fc05428d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c0a7200 con 0x7fc04c0ad930 2020-11-20T12:02:52.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.766+0000 7fc05428d700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c135240 con 0x7fc04c0a8ea0 2020-11-20T12:02:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.766+0000 7fc05428d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c0a6cc0 con 0x7fc04c0ad550 2020-11-20T12:02:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc04c0a8ea0 0x7fc04c1420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc04c0a8ea0 0x7fc04c1420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35810/0 (socket says 172.21.15.44:35810) 2020-11-20T12:02:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 -- 172.21.15.44:0/2554147005 learned_addr learned my addr 172.21.15.44:0/2554147005 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:52.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc052029700 1 --2- 172.21.15.44:0/2554147005 >> v2:172.21.15.79:3300/0 conn(0x7fc04c0ad550 0x7fc04c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 -- 172.21.15.44:0/2554147005 >> v2:172.21.15.79:3300/0 conn(0x7fc04c0ad550 msgr2=0x7fc04c0a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 --2- 172.21.15.44:0/2554147005 >> v2:172.21.15.79:3300/0 conn(0x7fc04c0ad550 0x7fc04c0a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 -- 172.21.15.44:0/2554147005 >> v1:172.21.15.79:6789/0 conn(0x7fc04c0ad930 legacy=0x7fc04c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:52.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 -- 172.21.15.44:0/2554147005 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0a6f60 con 0x7fc04c0a8ea0 2020-11-20T12:02:52.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc052029700 1 --2- 172.21.15.44:0/2554147005 >> v2:172.21.15.79:3300/0 conn(0x7fc04c0ad550 0x7fc04c0a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:02:52.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc05282a700 1 --2- 172.21.15.44:0/2554147005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0a8ea0 0x7fc04c1420b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc044009d50 tx=0x7fc044009290).ready entity=mon.0 client_cookie=2b7dff250bfc5f9b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:52.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc051027700 1 -- 172.21.15.44:0/2554147005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400bc40 con 0x7fc04c0a8ea0 2020-11-20T12:02:52.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc051027700 1 -- 172.21.15.44:0/2554147005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04400bda0 con 0x7fc04c0a8ea0 2020-11-20T12:02:52.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.767+0000 7fc051027700 1 -- 172.21.15.44:0/2554147005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044011610 con 0x7fc04c0a8ea0 2020-11-20T12:02:52.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 -- 172.21.15.44:0/2554147005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0a8ea0 msgr2=0x7fc04c1420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:52.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 --2- 172.21.15.44:0/2554147005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0a8ea0 0x7fc04c1420b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc044009d50 tx=0x7fc044009290).stop 2020-11-20T12:02:52.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 -- 172.21.15.44:0/2554147005 shutdown_connections 2020-11-20T12:02:52.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 --2- 172.21.15.44:0/2554147005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0a8ea0 0x7fc04c1420b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 --2- 172.21.15.44:0/2554147005 >> v2:172.21.15.79:3300/0 conn(0x7fc04c0ad550 0x7fc04c0a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 -- 172.21.15.44:0/2554147005 >> 172.21.15.44:0/2554147005 conn(0x7fc04c0624b0 msgr2=0x7fc04c0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:52.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 -- 172.21.15.44:0/2554147005 shutdown_connections 2020-11-20T12:02:52.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.768+0000 7fc05428d700 1 -- 172.21.15.44:0/2554147005 wait complete. 2020-11-20T12:02:52.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 Processor -- start 2020-11-20T12:02:52.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 -- start start 2020-11-20T12:02:52.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc04c0a8ea0 0x7fc04c09da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:52.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc04c0ad550 0x7fc04c131730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:52.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0ad930 0x7fc04c0a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:52.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c135590 con 0x7fc04c0ad930 2020-11-20T12:02:52.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c09e3d0 con 0x7fc04c0a8ea0 2020-11-20T12:02:52.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05428d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c09e670 con 0x7fc04c0ad550 2020-11-20T12:02:52.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05282a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0ad930 0x7fc04c0a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:52.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05282a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0ad930 0x7fc04c0a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35816/0 (socket says 172.21.15.44:35816) 2020-11-20T12:02:52.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc05282a700 1 -- 172.21.15.44:0/767224804 learned_addr learned my addr 172.21.15.44:0/767224804 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:52.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc052029700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc04c0a8ea0 0x7fc04c09da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:52.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.769+0000 7fc051828700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc04c0ad550 0x7fc04c131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:52.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05282a700 1 -- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc04c0ad550 msgr2=0x7fc04c131730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:52.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05282a700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc04c0ad550 0x7fc04c131730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05282a700 1 -- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc04c0a8ea0 msgr2=0x7fc04c09da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:52.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05282a700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc04c0a8ea0 0x7fc04c09da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:52.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05282a700 1 -- 172.21.15.44:0/767224804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c09e910 con 0x7fc04c0ad930 2020-11-20T12:02:52.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05282a700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0ad930 0x7fc04c0a02f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc0440092d0 tx=0x7fc044016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:52.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc03a7fc700 1 -- 172.21.15.44:0/767224804 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400bc40 con 0x7fc04c0ad930 2020-11-20T12:02:52.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc03a7fc700 1 -- 172.21.15.44:0/767224804 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc044003290 con 0x7fc04c0ad930 2020-11-20T12:02:52.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c1301f0 con 0x7fc04c0ad930 2020-11-20T12:02:52.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc03a7fc700 1 -- 172.21.15.44:0/767224804 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044010c70 con 0x7fc04c0ad930 2020-11-20T12:02:52.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.770+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c131c20 con 0x7fc04c0ad930 2020-11-20T12:02:52.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.771+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c09eaa0 con 0x7fc04c0ad930 2020-11-20T12:02:52.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.771+0000 7fc03a7fc700 1 -- 172.21.15.44:0/767224804 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc044017070 con 0x7fc04c0ad930 2020-11-20T12:02:52.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.771+0000 7fc03a7fc700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc024056d20 0x7fc0240591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:52.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.772+0000 7fc03a7fc700 1 -- 172.21.15.44:0/767224804 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7fc044071cd0 con 0x7fc04c0ad930 2020-11-20T12:02:52.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.772+0000 7fc052029700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc024056d20 0x7fc0240591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:52.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.772+0000 7fc052029700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc024056d20 0x7fc0240591e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc03c00ba80 tx=0x7fc03c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:52.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.776+0000 7fc03a7fc700 1 -- 172.21.15.44:0/767224804 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc0440474e0 con 0x7fc04c0ad930 2020-11-20T12:02:52.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:52.950+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc04c0615e0 con 0x7fc04c0ad930 2020-11-20T12:02:53.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.612+0000 7fc03a7fc700 1 -- 172.21.15.44:0/767224804 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc044015030 con 0x7fc04c0ad930 2020-11-20T12:02:53.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc024056d20 msgr2=0x7fc0240591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc024056d20 0x7fc0240591e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc03c00ba80 tx=0x7fc03c008040).stop 2020-11-20T12:02:53.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0ad930 msgr2=0x7fc04c0a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0ad930 0x7fc04c0a02f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc0440092d0 tx=0x7fc044016000).stop 2020-11-20T12:02:53.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 shutdown_connections 2020-11-20T12:02:53.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc024056d20 0x7fc0240591e0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0x7fc03c00ba80 tx=0x7fc03c008040).stop 2020-11-20T12:02:53.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc04c0ad930 0x7fc04c0a02f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc04c0a8ea0 0x7fc04c09da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 --2- 172.21.15.44:0/767224804 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc04c0ad550 0x7fc04c131730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.615+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 >> 172.21.15.44:0/767224804 conn(0x7fc04c0624b0 msgr2=0x7fc04c0a93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.616+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 shutdown_connections 2020-11-20T12:02:53.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.616+0000 7fc05428d700 1 -- 172.21.15.44:0/767224804 wait complete. 2020-11-20T12:02:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-20T12:02:53.635 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-20T12:02:53.636 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:02:53.636 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:02:53.639 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:02:53.639 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:02:53.640 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:02:53.642 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:02:53.642 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-20T12:02:53.643 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:02:53.721 INFO:tasks.ceph.mds.b.smithi079.stdout:starting mds.b at 2020-11-20T12:02:53.724 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2020-11-20T12:02:53.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.776+0000 7f62cc287700 1 Processor -- start 2020-11-20T12:02:53.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.776+0000 7f62cc287700 1 -- start start 2020-11-20T12:02:53.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62cc287700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 0x7f62c40a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62cc287700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c40a8870 con 0x7f62c413e040 2020-11-20T12:02:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62cc287700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c40a9f00 con 0x7f62c4060880 2020-11-20T12:02:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62cc287700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40a8aa0 con 0x7f62c4068f60 2020-11-20T12:02:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62c9822700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f62c413e040 0x7f62c4060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39580/0 (socket says 172.21.15.44:39580) 2020-11-20T12:02:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62c9822700 1 -- 172.21.15.44:0/4139547864 learned_addr learned my addr 172.21.15.44:0/4139547864 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:02:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999373766 0 0) 0x7f62c40a8870 con 0x7f62c413e040 2020-11-20T12:02:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62ca023700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 0x7f62c40a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b0003240 con 0x7f62c413e040 2020-11-20T12:02:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.777+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4096773960 0 0) 0x7f62b0003240 con 0x7f62c413e040 2020-11-20T12:02:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 msgr2=0x7f62c40a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 0x7f62c40a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 >> v1:172.21.15.79:6789/0 conn(0x7f62c4060880 legacy=0x7f62c4145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c40a8d40 con 0x7f62c413e040 2020-11-20T12:02:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f62b80014e0 con 0x7f62c413e040 2020-11-20T12:02:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 >> v1:172.21.15.44:6789/0 conn(0x7f62c413e040 legacy=0x7f62c4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 0x7f62b0003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b0003c50 0x7f62b00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62b00049d0 0x7f62b0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40a8aa0 con 0x7f62b00049d0 2020-11-20T12:02:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0004170 con 0x7f62b0003c50 2020-11-20T12:02:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0004330 con 0x7f62c4068f60 2020-11-20T12:02:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca824700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62b00049d0 0x7f62b0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9822700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b0003c50 0x7f62b00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca023700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 0x7f62b0003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca824700 1 -- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 msgr2=0x7f62b0003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca824700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c4068f60 0x7f62b0003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca824700 1 -- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b0003c50 msgr2=0x7f62b00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca824700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b0003c50 0x7f62b00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca824700 1 -- 172.21.15.44:0/4139547864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b0003240 con 0x7f62b00049d0 2020-11-20T12:02:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62ca824700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62b00049d0 0x7f62b0004e50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f62b4001a30 tx=0x7f62b4007b70).ready entity=mon.0 client_cookie=5072931a62666e47 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400c7d0 con 0x7f62b00049d0 2020-11-20T12:02:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b400c930 con 0x7f62b00049d0 2020-11-20T12:02:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.778+0000 7f62c9021700 1 -- 172.21.15.44:0/4139547864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b400ccf0 con 0x7f62b00049d0 2020-11-20T12:02:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.779+0000 7f62cc287700 1 -- 172.21.15.44:0/4139547864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62b00049d0 msgr2=0x7f62b0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.779+0000 7f62cc287700 1 --2- 172.21.15.44:0/4139547864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62b00049d0 0x7f62b0004e50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f62b4001a30 tx=0x7f62b4007b70).stop 2020-11-20T12:02:53.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.779+0000 7f62ca824700 1 -- 172.21.15.44:0/4139547864 reap_dead start 2020-11-20T12:02:53.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.779+0000 7f62cc287700 1 -- 172.21.15.44:0/4139547864 shutdown_connections 2020-11-20T12:02:53.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.779+0000 7f62cc287700 1 -- 172.21.15.44:0/4139547864 >> 172.21.15.44:0/4139547864 conn(0x7f62c40ad6f0 msgr2=0x7f62c40adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:53.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.779+0000 7f62cc287700 1 -- 172.21.15.44:0/4139547864 shutdown_connections 2020-11-20T12:02:53.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.779+0000 7f62cc287700 1 -- 172.21.15.44:0/4139547864 wait complete. 2020-11-20T12:02:53.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 Processor -- start 2020-11-20T12:02:53.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 -- start start 2020-11-20T12:02:53.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b00049d0 0x7f62c4130870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c40aa2e0 0x7f62c413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62c4130e30 0x7f62c4131290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40aa760 con 0x7f62c4130e30 2020-11-20T12:02:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c413dbd0 con 0x7f62b00049d0 2020-11-20T12:02:53.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62cc287700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c413bc40 con 0x7f62c40aa2e0 2020-11-20T12:02:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62ca824700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62c4130e30 0x7f62c4131290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62ca824700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62c4130e30 0x7f62c4131290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35848/0 (socket says 172.21.15.44:35848) 2020-11-20T12:02:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62ca824700 1 -- 172.21.15.44:0/4100309339 learned_addr learned my addr 172.21.15.44:0/4100309339 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62c9822700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c40aa2e0 0x7f62c413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.780+0000 7f62ca023700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b00049d0 0x7f62c4130870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62ca824700 1 -- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c40aa2e0 msgr2=0x7f62c413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62ca824700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c40aa2e0 0x7f62c413d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62ca824700 1 -- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b00049d0 msgr2=0x7f62c4130870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62ca824700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b00049d0 0x7f62c4130870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62ca824700 1 -- 172.21.15.44:0/4100309339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c412f840 con 0x7f62c4130e30 2020-11-20T12:02:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62c9822700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c40aa2e0 0x7f62c413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62ca824700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62c4130e30 0x7f62c4131290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f62b4010000 tx=0x7f62b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62c27fc700 1 -- 172.21.15.44:0/4100309339 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4013bd0 con 0x7f62c4130e30 2020-11-20T12:02:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62c27fc700 1 -- 172.21.15.44:0/4100309339 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b4013d30 con 0x7f62c4130e30 2020-11-20T12:02:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c413bf10 con 0x7f62c4130e30 2020-11-20T12:02:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62c27fc700 1 -- 172.21.15.44:0/4100309339 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4013ea0 con 0x7f62c4130e30 2020-11-20T12:02:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.781+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c41303d0 con 0x7f62c4130e30 2020-11-20T12:02:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.782+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c41306c0 con 0x7f62c4130e30 2020-11-20T12:02:53.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.782+0000 7f62c27fc700 1 -- 172.21.15.44:0/4100309339 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f62b401b040 con 0x7f62c4130e30 2020-11-20T12:02:53.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.783+0000 7f62c27fc700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f629c056d20 0x7f629c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:53.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.783+0000 7f62c27fc700 1 -- 172.21.15.44:0/4100309339 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f62b4070660 con 0x7f62c4130e30 2020-11-20T12:02:53.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.783+0000 7f62ca023700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f629c056d20 0x7f629c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:53.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.783+0000 7f62ca023700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f629c056d20 0x7f629c0591e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f62bc006e10 tx=0x7f62bc0091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:53.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.786+0000 7f62c27fc700 1 -- 172.21.15.44:0/4100309339 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f62b4045de0 con 0x7f62c4130e30 2020-11-20T12:02:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.959+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62c4063f00 con 0x7f62c4130e30 2020-11-20T12:02:53.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.960+0000 7f62c27fc700 1 -- 172.21.15.44:0/4100309339 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13637 (secure 0 0 0) 0x7f62b4049400 con 0x7f62c4130e30 2020-11-20T12:02:53.964 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:53.964 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:02:53.609751+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:02:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f629c056d20 msgr2=0x7f629c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f629c056d20 0x7f629c0591e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f62bc006e10 tx=0x7f62bc0091d0).stop 2020-11-20T12:02:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62c4130e30 msgr2=0x7f62c4131290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62c4130e30 0x7f62c4131290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f62b4010000 tx=0x7f62b4016000).stop 2020-11-20T12:02:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 shutdown_connections 2020-11-20T12:02:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f629c056d20 0x7f629c0591e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62c4130e30 0x7f62c4131290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b00049d0 0x7f62c4130870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 --2- 172.21.15.44:0/4100309339 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62c40aa2e0 0x7f62c413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 >> 172.21.15.44:0/4100309339 conn(0x7f62c40ad6f0 msgr2=0x7f62c413e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:53.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 shutdown_connections 2020-11-20T12:02:53.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:53.962+0000 7f62cc287700 1 -- 172.21.15.44:0/4100309339 wait complete. 2020-11-20T12:02:53.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:02:44.611733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:02:52.195125+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:02:53.977 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:02:54.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.102+0000 7fe5ea37c700 1 Processor -- start 2020-11-20T12:02:54.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5ea37c700 1 -- start start 2020-11-20T12:02:54.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5ea37c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe5e413ab60 0x7fe5e40a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5ea37c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40aaa00 con 0x7fe5e40b1970 2020-11-20T12:02:54.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5ea37c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40a9610 con 0x7fe5e40b1d50 2020-11-20T12:02:54.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5ea37c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e413afd0 con 0x7fe5e413ab60 2020-11-20T12:02:54.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5e8919700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe5e413ab60 0x7fe5e40a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5e3fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe5e40b1970 0x7fe5e413b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39600/0 (socket says 172.21.15.44:39600) 2020-11-20T12:02:54.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.103+0000 7fe5e3fff700 1 -- 172.21.15.44:0/521036420 learned_addr learned my addr 172.21.15.44:0/521036420 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:02:54.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e8919700 1 -- 172.21.15.44:0/521036420 >> v1:172.21.15.79:6789/0 conn(0x7fe5e40b1d50 legacy=0x7fe5e4143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e8919700 1 -- 172.21.15.44:0/521036420 >> v1:172.21.15.44:6789/0 conn(0x7fe5e40b1970 legacy=0x7fe5e413b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e8919700 1 -- 172.21.15.44:0/521036420 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40a98b0 con 0x7fe5e413ab60 2020-11-20T12:02:54.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e2ffd700 1 -- 172.21.15.44:0/521036420 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240552619 0 0) 0x7fe5e40aaa00 con 0x7fe5e40b1970 2020-11-20T12:02:54.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e8919700 1 --2- 172.21.15.44:0/521036420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e413ab60 0x7fe5e40a86a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe5dc009c90 tx=0x7fe5dc0091d0).ready entity=mon.0 client_cookie=cc52988353eec296 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e2ffd700 1 -- 172.21.15.44:0/521036420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00bbc0 con 0x7fe5e413ab60 2020-11-20T12:02:54.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e2ffd700 1 -- 172.21.15.44:0/521036420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc00bd20 con 0x7fe5e413ab60 2020-11-20T12:02:54.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5e2ffd700 1 -- 172.21.15.44:0/521036420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc011610 con 0x7fe5e413ab60 2020-11-20T12:02:54.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5ea37c700 1 -- 172.21.15.44:0/521036420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e413ab60 msgr2=0x7fe5e40a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/521036420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e413ab60 0x7fe5e40a86a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe5dc009c90 tx=0x7fe5dc0091d0).stop 2020-11-20T12:02:54.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5ea37c700 1 -- 172.21.15.44:0/521036420 shutdown_connections 2020-11-20T12:02:54.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/521036420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e413ab60 0x7fe5e40a86a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5ea37c700 1 -- 172.21.15.44:0/521036420 >> 172.21.15.44:0/521036420 conn(0x7fe5e4060c40 msgr2=0x7fe5e40b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:54.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5ea37c700 1 -- 172.21.15.44:0/521036420 shutdown_connections 2020-11-20T12:02:54.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.104+0000 7fe5ea37c700 1 -- 172.21.15.44:0/521036420 wait complete. 2020-11-20T12:02:54.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.105+0000 7fe5ea37c700 1 Processor -- start 2020-11-20T12:02:54.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5ea37c700 1 -- start start 2020-11-20T12:02:54.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5ea37c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e40b1970 0x7fe5e40a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5ea37c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e40b1d50 0x7fe5e409fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5ea37c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe5e413ab60 0x7fe5e41365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5ea37c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40a0a60 con 0x7fe5e40b1970 2020-11-20T12:02:54.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5ea37c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40a0300 con 0x7fe5e40b1d50 2020-11-20T12:02:54.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5ea37c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40a0520 con 0x7fe5e413ab60 2020-11-20T12:02:54.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e40b1970 0x7fe5e40a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e40b1970 0x7fe5e40a74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35858/0 (socket says 172.21.15.44:35858) 2020-11-20T12:02:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 -- 172.21.15.44:0/1069953693 learned_addr learned my addr 172.21.15.44:0/1069953693 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e8919700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe5e413ab60 0x7fe5e41365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e37fe700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e40b1d50 0x7fe5e409fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 -- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe5e413ab60 msgr2=0x7fe5e41365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe5e413ab60 0x7fe5e41365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 -- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e40b1d50 msgr2=0x7fe5e409fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e40b1d50 0x7fe5e409fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.106+0000 7fe5e3fff700 1 -- 172.21.15.44:0/1069953693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40a07c0 con 0x7fe5e40b1970 2020-11-20T12:02:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.107+0000 7fe5e3fff700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e40b1970 0x7fe5e40a74e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe5d400c760 tx=0x7fe5d4007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.107+0000 7fe5c3fff700 1 -- 172.21.15.44:0/1069953693 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40072e0 con 0x7fe5e40b1970 2020-11-20T12:02:54.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.107+0000 7fe5c3fff700 1 -- 172.21.15.44:0/1069953693 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4007440 con 0x7fe5e40b1970 2020-11-20T12:02:54.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.107+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e40a22b0 con 0x7fe5e40b1970 2020-11-20T12:02:54.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.107+0000 7fe5c3fff700 1 -- 172.21.15.44:0/1069953693 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4005820 con 0x7fe5e40b1970 2020-11-20T12:02:54.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.107+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e4136ce0 con 0x7fe5e40b1970 2020-11-20T12:02:54.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.108+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e40619a0 con 0x7fe5e40b1970 2020-11-20T12:02:54.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.108+0000 7fe5c3fff700 1 -- 172.21.15.44:0/1069953693 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe5d4010070 con 0x7fe5e40b1970 2020-11-20T12:02:54.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.108+0000 7fe5c3fff700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe5bc056d20 0x7fe5bc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.108+0000 7fe5c3fff700 1 -- 172.21.15.44:0/1069953693 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe5d406f310 con 0x7fe5e40b1970 2020-11-20T12:02:54.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.108+0000 7fe5e37fe700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe5bc056d20 0x7fe5bc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.109+0000 7fe5e37fe700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe5bc056d20 0x7fe5bc0591e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe5e4138070 tx=0x7fe5d8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.112+0000 7fe5c3fff700 1 -- 172.21.15.44:0/1069953693 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe5d4044a90 con 0x7fe5e40b1970 2020-11-20T12:02:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.286+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe5e40a2440 con 0x7fe5e40b1970 2020-11-20T12:02:54.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.287+0000 7fe5c3fff700 1 -- 172.21.15.44:0/1069953693 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe5e40a2440 con 0x7fe5e40b1970 2020-11-20T12:02:54.291 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2020-11-20T12:02:54.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe5bc056d20 msgr2=0x7fe5bc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe5bc056d20 0x7fe5bc0591e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe5e4138070 tx=0x7fe5d8009040).stop 2020-11-20T12:02:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e40b1970 msgr2=0x7fe5e40a74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e40b1970 0x7fe5e40a74e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe5d400c760 tx=0x7fe5d4007a60).stop 2020-11-20T12:02:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 shutdown_connections 2020-11-20T12:02:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe5bc056d20 0x7fe5bc0591e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe5e413ab60 0x7fe5e41365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5e40b1970 0x7fe5e40a74e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 --2- 172.21.15.44:0/1069953693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e40b1d50 0x7fe5e409fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 >> 172.21.15.44:0/1069953693 conn(0x7fe5e4060c40 msgr2=0x7fe5e413c280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:54.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 shutdown_connections 2020-11-20T12:02:54.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.288+0000 7fe5ea37c700 1 -- 172.21.15.44:0/1069953693 wait complete. 2020-11-20T12:02:54.303 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:02:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.428+0000 7f41fbbe3700 1 Processor -- start 2020-11-20T12:02:54.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41fbbe3700 1 -- start start 2020-11-20T12:02:54.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41fbbe3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41f40adc80 0x7f41f40a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41fbbe3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f41f413ab50 0x7f41f413afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41fbbe3700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f40ab560 con 0x7f41f40ae060 2020-11-20T12:02:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41fbbe3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f40a8770 con 0x7f41f413ab50 2020-11-20T12:02:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41fbbe3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f40a8f60 con 0x7f41f40adc80 2020-11-20T12:02:54.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41fa180700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f41f413ab50 0x7f41f413afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41f917e700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f41f40ae060 0x7f41f40aadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39614/0 (socket says 172.21.15.44:39614) 2020-11-20T12:02:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41f917e700 1 -- 172.21.15.44:0/420562087 learned_addr learned my addr 172.21.15.44:0/420562087 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:02:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.429+0000 7f41f997f700 1 --2- 172.21.15.44:0/420562087 >> v2:172.21.15.79:3300/0 conn(0x7f41f40adc80 0x7f41f40a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fa180700 1 -- 172.21.15.44:0/420562087 >> v2:172.21.15.79:3300/0 conn(0x7f41f40adc80 msgr2=0x7f41f40a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:54.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fa180700 1 --2- 172.21.15.44:0/420562087 >> v2:172.21.15.79:3300/0 conn(0x7f41f40adc80 0x7f41f40a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fa180700 1 -- 172.21.15.44:0/420562087 >> v1:172.21.15.44:6789/0 conn(0x7f41f40ae060 legacy=0x7f41f40aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fa180700 1 -- 172.21.15.44:0/420562087 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40a9200 con 0x7f41f413ab50 2020-11-20T12:02:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41f897d700 1 -- 172.21.15.44:0/420562087 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290254341 0 0) 0x7f41f40ab560 con 0x7f41f40ae060 2020-11-20T12:02:54.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fa180700 1 --2- 172.21.15.44:0/420562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f413afe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f41f4060ef0 tx=0x7f41ec0091d0).ready entity=mon.0 client_cookie=745ca9b58da2ae91 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41f897d700 1 -- 172.21.15.44:0/420562087 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec00ba70 con 0x7f41f413ab50 2020-11-20T12:02:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41f897d700 1 -- 172.21.15.44:0/420562087 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec00bbd0 con 0x7f41f413ab50 2020-11-20T12:02:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41f897d700 1 -- 172.21.15.44:0/420562087 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec011400 con 0x7f41f413ab50 2020-11-20T12:02:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 -- 172.21.15.44:0/420562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 msgr2=0x7f41f413afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/420562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f413afe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f41f4060ef0 tx=0x7f41ec0091d0).stop 2020-11-20T12:02:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 -- 172.21.15.44:0/420562087 shutdown_connections 2020-11-20T12:02:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/420562087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f413afe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/420562087 >> v2:172.21.15.79:3300/0 conn(0x7f41f40adc80 0x7f41f40a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 -- 172.21.15.44:0/420562087 >> 172.21.15.44:0/420562087 conn(0x7f41f4060170 msgr2=0x7f41f4060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 -- 172.21.15.44:0/420562087 shutdown_connections 2020-11-20T12:02:54.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.430+0000 7f41fbbe3700 1 -- 172.21.15.44:0/420562087 wait complete. 2020-11-20T12:02:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.431+0000 7f41fbbe3700 1 Processor -- start 2020-11-20T12:02:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.431+0000 7f41fbbe3700 1 -- start start 2020-11-20T12:02:54.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fbbe3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41f40adc80 0x7f41f409e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fbbe3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41f40ae060 0x7f41f40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fbbe3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f40a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fbbe3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40a6d00 con 0x7f41f413ab50 2020-11-20T12:02:54.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fbbe3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40a77c0 con 0x7f41f40ae060 2020-11-20T12:02:54.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fbbe3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40a7a60 con 0x7f41f40adc80 2020-11-20T12:02:54.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f40a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f40a6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35874/0 (socket says 172.21.15.44:35874) 2020-11-20T12:02:54.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 -- 172.21.15.44:0/3409023309 learned_addr learned my addr 172.21.15.44:0/3409023309 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:54.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41f917e700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41f40ae060 0x7f41f40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41f997f700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41f40adc80 0x7f41f409e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 -- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41f40adc80 msgr2=0x7f41f409e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41f40adc80 0x7f41f409e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 -- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41f40ae060 msgr2=0x7f41f40a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41f40ae060 0x7f41f40a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 -- 172.21.15.44:0/3409023309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40a7d00 con 0x7f41f413ab50 2020-11-20T12:02:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.432+0000 7f41fa180700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f40a6810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f41ec009fd0 tx=0x7f41ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.433+0000 7f41e5ffb700 1 -- 172.21.15.44:0/3409023309 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec01c5f0 con 0x7f41f413ab50 2020-11-20T12:02:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.433+0000 7f41e5ffb700 1 -- 172.21.15.44:0/3409023309 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec01c750 con 0x7f41f413ab50 2020-11-20T12:02:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.433+0000 7f41e5ffb700 1 -- 172.21.15.44:0/3409023309 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec01cad0 con 0x7f41f413ab50 2020-11-20T12:02:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.433+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f40a74b0 con 0x7f41f413ab50 2020-11-20T12:02:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.433+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f413e660 con 0x7f41f413ab50 2020-11-20T12:02:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.434+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f413e940 con 0x7f41f413ab50 2020-11-20T12:02:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.435+0000 7f41e5ffb700 1 -- 172.21.15.44:0/3409023309 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f41ec024030 con 0x7f41f413ab50 2020-11-20T12:02:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.435+0000 7f41e5ffb700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f41d0056d20 0x7f41d00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.435+0000 7f41e5ffb700 1 -- 172.21.15.44:0/3409023309 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f41ec072040 con 0x7f41f413ab50 2020-11-20T12:02:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.435+0000 7f41f997f700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f41d0056d20 0x7f41d00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.435+0000 7f41f997f700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f41d0056d20 0x7f41d00591e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f41e8006fd0 tx=0x7f41e8006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.438+0000 7f41e5ffb700 1 -- 172.21.15.44:0/3409023309 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f41ec0478d0 con 0x7f41f413ab50 2020-11-20T12:02:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.613+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f41f40696f0 con 0x7f41f413ab50 2020-11-20T12:02:54.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.614+0000 7f41e5ffb700 1 -- 172.21.15.44:0/3409023309 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f41ec02c070 con 0x7f41f413ab50 2020-11-20T12:02:54.618 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:02:54.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f41d0056d20 msgr2=0x7f41d00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f41d0056d20 0x7f41d00591e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f41e8006fd0 tx=0x7f41e8006df0).stop 2020-11-20T12:02:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 msgr2=0x7f41f40a6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f40a6810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f41ec009fd0 tx=0x7f41ec016000).stop 2020-11-20T12:02:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 shutdown_connections 2020-11-20T12:02:54.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f41d0056d20 0x7f41d00591e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41f413ab50 0x7f41f40a6810 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41f40adc80 0x7f41f409e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 --2- 172.21.15.44:0/3409023309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41f40ae060 0x7f41f40a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 >> 172.21.15.44:0/3409023309 conn(0x7f41f4060170 msgr2=0x7f41f40af080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 shutdown_connections 2020-11-20T12:02:54.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.615+0000 7f41fbbe3700 1 -- 172.21.15.44:0/3409023309 wait complete. 2020-11-20T12:02:54.631 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:02:54.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f92153bb700 1 Processor -- start 2020-11-20T12:02:54.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f92153bb700 1 -- start start 2020-11-20T12:02:54.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f92153bb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9210060330 0x7f92100a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f92153bb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92100a4c30 0x7f921013a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f92153bb700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100a6260 con 0x7f921013ae80 2020-11-20T12:02:54.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f92153bb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92100a52e0 con 0x7f92100a4c30 2020-11-20T12:02:54.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f92153bb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92100aa730 con 0x7f9210060330 2020-11-20T12:02:54.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f920f7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f921013ae80 0x7f92100aa020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39630/0 (socket says 172.21.15.44:39630) 2020-11-20T12:02:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.755+0000 7f920f7fe700 1 -- 172.21.15.44:0/3486380940 learned_addr learned my addr 172.21.15.44:0/3486380940 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:02:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920e7fc700 1 --2- 172.21.15.44:0/3486380940 >> v2:172.21.15.79:3300/0 conn(0x7f92100a4c30 0x7f921013a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920effd700 1 --2- 172.21.15.44:0/3486380940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9210060330 0x7f92100a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920dffb700 1 -- 172.21.15.44:0/3486380940 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715741124 0 0) 0x7f92100a6260 con 0x7f921013ae80 2020-11-20T12:02:54.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920dffb700 1 -- 172.21.15.44:0/3486380940 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91fc003240 con 0x7f921013ae80 2020-11-20T12:02:54.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920effd700 1 -- 172.21.15.44:0/3486380940 >> v2:172.21.15.79:3300/0 conn(0x7f92100a4c30 msgr2=0x7f921013a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:54.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920effd700 1 --2- 172.21.15.44:0/3486380940 >> v2:172.21.15.79:3300/0 conn(0x7f92100a4c30 0x7f921013a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920effd700 1 -- 172.21.15.44:0/3486380940 >> v1:172.21.15.44:6789/0 conn(0x7f921013ae80 legacy=0x7f92100aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920dffb700 1 -- 172.21.15.44:0/3486380940 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2041843012 0 0) 0x7f91fc003240 con 0x7f921013ae80 2020-11-20T12:02:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920effd700 1 -- 172.21.15.44:0/3486380940 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100a6490 con 0x7f9210060330 2020-11-20T12:02:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920e7fc700 1 --2- 172.21.15.44:0/3486380940 >> v2:172.21.15.79:3300/0 conn(0x7f92100a4c30 0x7f921013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920effd700 1 --2- 172.21.15.44:0/3486380940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9210060330 0x7f92100a4740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f920000b9a0 tx=0x7f9200006df0).ready entity=mon.2 client_cookie=cb3368ce0b83a70 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920dffb700 1 -- 172.21.15.44:0/3486380940 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208001860 con 0x7f9210060330 2020-11-20T12:02:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920dffb700 1 -- 172.21.15.44:0/3486380940 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9200011400 con 0x7f9210060330 2020-11-20T12:02:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.756+0000 7f920dffb700 1 -- 172.21.15.44:0/3486380940 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200011780 con 0x7f9210060330 2020-11-20T12:02:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 -- 172.21.15.44:0/3486380940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9210060330 msgr2=0x7f92100a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 --2- 172.21.15.44:0/3486380940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9210060330 0x7f92100a4740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f920000b9a0 tx=0x7f9200006df0).stop 2020-11-20T12:02:54.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 -- 172.21.15.44:0/3486380940 shutdown_connections 2020-11-20T12:02:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 --2- 172.21.15.44:0/3486380940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9210060330 0x7f92100a4740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 --2- 172.21.15.44:0/3486380940 >> v2:172.21.15.79:3300/0 conn(0x7f92100a4c30 0x7f921013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 -- 172.21.15.44:0/3486380940 >> 172.21.15.44:0/3486380940 conn(0x7f9210062690 msgr2=0x7f92100a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 -- 172.21.15.44:0/3486380940 shutdown_connections 2020-11-20T12:02:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.757+0000 7f92153bb700 1 -- 172.21.15.44:0/3486380940 wait complete. 2020-11-20T12:02:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 Processor -- start 2020-11-20T12:02:54.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 -- start start 2020-11-20T12:02:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9210060330 0x7f92100a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92100a4c30 0x7f9210135540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921013ae80 0x7f92100a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210136e90 con 0x7f9210060330 2020-11-20T12:02:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210137080 con 0x7f92100a4c30 2020-11-20T12:02:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.758+0000 7f92153bb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9210137320 con 0x7f921013ae80 2020-11-20T12:02:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9210060330 0x7f92100a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9210060330 0x7f92100a29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35884/0 (socket says 172.21.15.44:35884) 2020-11-20T12:02:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 -- 172.21.15.44:0/1578967395 learned_addr learned my addr 172.21.15.44:0/1578967395 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920e7fc700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92100a4c30 0x7f9210135540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920f7fe700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921013ae80 0x7f92100a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 -- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921013ae80 msgr2=0x7f92100a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921013ae80 0x7f92100a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 -- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92100a4c30 msgr2=0x7f9210135540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92100a4c30 0x7f9210135540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 -- 172.21.15.44:0/1578967395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92101375c0 con 0x7f9210060330 2020-11-20T12:02:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f920effd700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9210060330 0x7f92100a29b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f9200006fd0 tx=0x7f9200013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f91f6ffd700 1 -- 172.21.15.44:0/1578967395 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200010a60 con 0x7f9210060330 2020-11-20T12:02:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f91f6ffd700 1 -- 172.21.15.44:0/1578967395 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9200010bc0 con 0x7f9210060330 2020-11-20T12:02:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9210133420 con 0x7f9210060330 2020-11-20T12:02:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f91f6ffd700 1 -- 172.21.15.44:0/1578967395 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920001e460 con 0x7f9210060330 2020-11-20T12:02:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.759+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9210137ab0 con 0x7f9210060330 2020-11-20T12:02:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.760+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9210137d90 con 0x7f9210060330 2020-11-20T12:02:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.760+0000 7f91f6ffd700 1 -- 172.21.15.44:0/1578967395 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9200025030 con 0x7f9210060330 2020-11-20T12:02:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.761+0000 7f91f6ffd700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91ec056d20 0x7f91ec0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.761+0000 7f91f6ffd700 1 -- 172.21.15.44:0/1578967395 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9200070d20 con 0x7f9210060330 2020-11-20T12:02:54.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.761+0000 7f920e7fc700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91ec056d20 0x7f91ec0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:54.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.761+0000 7f920e7fc700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91ec056d20 0x7f91ec0591e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9210136c30 tx=0x7f9204006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:54.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.764+0000 7f91f6ffd700 1 -- 172.21.15.44:0/1578967395 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9200046550 con 0x7f9210060330 2020-11-20T12:02:54.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.941+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f92100a3030 con 0x7f9210060330 2020-11-20T12:02:54.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.941+0000 7f91f6ffd700 1 -- 172.21.15.44:0/1578967395 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9200023030 con 0x7f9210060330 2020-11-20T12:02:54.945 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:02:54.946 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2020-11-20T12:02:54.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:02:54.946 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2020-11-20T12:02:54.946 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": "device_health_metrics" 2020-11-20T12:02:54.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:02:54.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:02:54.947 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 2, 2020-11-20T12:02:54.947 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": "rbd" 2020-11-20T12:02:54.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:02:54.948 INFO:teuthology.orchestra.run.smithi044.stdout:] 2020-11-20T12:02:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91ec056d20 msgr2=0x7f91ec0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91ec056d20 0x7f91ec0591e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9210136c30 tx=0x7f9204006df0).stop 2020-11-20T12:02:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9210060330 msgr2=0x7f92100a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9210060330 0x7f92100a29b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f9200006fd0 tx=0x7f9200013000).stop 2020-11-20T12:02:54.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 shutdown_connections 2020-11-20T12:02:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91ec056d20 0x7f91ec0591e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921013ae80 0x7f92100a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9210060330 0x7f92100a29b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 --2- 172.21.15.44:0/1578967395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92100a4c30 0x7f9210135540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 >> 172.21.15.44:0/1578967395 conn(0x7f9210062690 msgr2=0x7f921013a760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 shutdown_connections 2020-11-20T12:02:54.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:54.943+0000 7f92153bb700 1 -- 172.21.15.44:0/1578967395 wait complete. 2020-11-20T12:02:54.958 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:02:54.959 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:02:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.084+0000 7fcb7ce44700 1 Processor -- start 2020-11-20T12:02:55.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.084+0000 7fcb7ce44700 1 -- start start 2020-11-20T12:02:55.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb7ce44700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb78142ae0 0x7fcb7813be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb7ce44700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb7813c360 0x7fcb78149770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb7ce44700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7813c7f0 con 0x7fcb780a3ad0 2020-11-20T12:02:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb7ce44700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb780a57f0 con 0x7fcb7813c360 2020-11-20T12:02:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb7ce44700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780a5a90 con 0x7fcb78142ae0 2020-11-20T12:02:55.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb6ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb78142ae0 0x7fcb7813be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb76d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb7813c360 0x7fcb78149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb6ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb78142ae0 0x7fcb7813be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:47332/0 (socket says 172.21.15.44:47332) 2020-11-20T12:02:55.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb6ffff700 1 -- 172.21.15.44:0/1865495169 learned_addr learned my addr 172.21.15.44:0/1865495169 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:55.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb6ffff700 1 -- 172.21.15.44:0/1865495169 >> v2:172.21.15.79:3300/0 conn(0x7fcb7813c360 msgr2=0x7fcb78149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:55.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb6ffff700 1 --2- 172.21.15.44:0/1865495169 >> v2:172.21.15.79:3300/0 conn(0x7fcb7813c360 0x7fcb78149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb6ffff700 1 -- 172.21.15.44:0/1865495169 >> v1:172.21.15.79:6789/0 conn(0x7fcb780a3ad0 legacy=0x7fcb781423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb6ffff700 1 -- 172.21.15.44:0/1865495169 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780a5d30 con 0x7fcb78142ae0 2020-11-20T12:02:55.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.085+0000 7fcb76d9d700 1 --2- 172.21.15.44:0/1865495169 >> v2:172.21.15.79:3300/0 conn(0x7fcb7813c360 0x7fcb78149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:02:55.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb6ffff700 1 --2- 172.21.15.44:0/1865495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb78142ae0 0x7fcb7813be70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcb60006fd0 tx=0x7fcb60006eb0).ready entity=mon.2 client_cookie=21f169f0d8ff5722 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:55.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb75d9b700 1 -- 172.21.15.44:0/1865495169 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60009810 con 0x7fcb78142ae0 2020-11-20T12:02:55.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb75d9b700 1 -- 172.21.15.44:0/1865495169 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb60009970 con 0x7fcb78142ae0 2020-11-20T12:02:55.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb75d9b700 1 -- 172.21.15.44:0/1865495169 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60009cf0 con 0x7fcb78142ae0 2020-11-20T12:02:55.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 -- 172.21.15.44:0/1865495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb78142ae0 msgr2=0x7fcb7813be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/1865495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb78142ae0 0x7fcb7813be70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcb60006fd0 tx=0x7fcb60006eb0).stop 2020-11-20T12:02:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 -- 172.21.15.44:0/1865495169 shutdown_connections 2020-11-20T12:02:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/1865495169 >> v2:172.21.15.79:3300/0 conn(0x7fcb7813c360 0x7fcb78149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/1865495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb78142ae0 0x7fcb7813be70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 -- 172.21.15.44:0/1865495169 >> 172.21.15.44:0/1865495169 conn(0x7fcb780624b0 msgr2=0x7fcb780ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:55.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 -- 172.21.15.44:0/1865495169 shutdown_connections 2020-11-20T12:02:55.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.086+0000 7fcb7ce44700 1 -- 172.21.15.44:0/1865495169 wait complete. 2020-11-20T12:02:55.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.087+0000 7fcb7ce44700 1 Processor -- start 2020-11-20T12:02:55.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.087+0000 7fcb7ce44700 1 -- start start 2020-11-20T12:02:55.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb780a3ad0 0x7fcb780ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb7813c360 0x7fcb781517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb78142ae0 0x7fcb78151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7814e770 con 0x7fcb780a3ad0 2020-11-20T12:02:55.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7814e9d0 con 0x7fcb78142ae0 2020-11-20T12:02:55.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7814ec70 con 0x7fcb7813c360 2020-11-20T12:02:55.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb780a3ad0 0x7fcb780ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb780a3ad0 0x7fcb780ac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35898/0 (socket says 172.21.15.44:35898) 2020-11-20T12:02:55.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 -- 172.21.15.44:0/3633308322 learned_addr learned my addr 172.21.15.44:0/3633308322 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb6ffff700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb7813c360 0x7fcb781517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb76d9d700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb78142ae0 0x7fcb78151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 -- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb7813c360 msgr2=0x7fcb781517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb7813c360 0x7fcb781517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 -- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb78142ae0 msgr2=0x7fcb78151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb78142ae0 0x7fcb78151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 -- 172.21.15.44:0/3633308322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7814ef10 con 0x7fcb780a3ad0 2020-11-20T12:02:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7659c700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb780a3ad0 0x7fcb780ac3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fcb6800c710 tx=0x7fcb68007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:55.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb6e7fc700 1 -- 172.21.15.44:0/3633308322 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800e4a0 con 0x7fcb780a3ad0 2020-11-20T12:02:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb6e7fc700 1 -- 172.21.15.44:0/3633308322 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6800e600 con 0x7fcb780a3ad0 2020-11-20T12:02:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb780aad30 con 0x7fcb780a3ad0 2020-11-20T12:02:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb6e7fc700 1 -- 172.21.15.44:0/3633308322 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800e9a0 con 0x7fcb780a3ad0 2020-11-20T12:02:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.088+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7813b6f0 con 0x7fcb780a3ad0 2020-11-20T12:02:55.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.089+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb780615e0 con 0x7fcb780a3ad0 2020-11-20T12:02:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.090+0000 7fcb6e7fc700 1 -- 172.21.15.44:0/3633308322 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcb68019030 con 0x7fcb780a3ad0 2020-11-20T12:02:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.090+0000 7fcb6e7fc700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb54056d20 0x7fcb540591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.090+0000 7fcb6e7fc700 1 -- 172.21.15.44:0/3633308322 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fcb6806f160 con 0x7fcb780a3ad0 2020-11-20T12:02:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.090+0000 7fcb6ffff700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb54056d20 0x7fcb540591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.090+0000 7fcb6ffff700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb54056d20 0x7fcb540591e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fcb60006ef0 tx=0x7fcb60015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:55.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.093+0000 7fcb6e7fc700 1 -- 172.21.15.44:0/3633308322 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcb680448e0 con 0x7fcb780a3ad0 2020-11-20T12:02:55.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.267+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fcb780446d0 con 0x7fcb780a3ad0 2020-11-20T12:02:55.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.741+0000 7fcb6e7fc700 1 -- 172.21.15.44:0/3633308322 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcb68017020 con 0x7fcb780a3ad0 2020-11-20T12:02:55.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.744+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb54056d20 msgr2=0x7fcb540591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.744+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb54056d20 0x7fcb540591e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fcb60006ef0 tx=0x7fcb60015000).stop 2020-11-20T12:02:55.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.744+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb780a3ad0 msgr2=0x7fcb780ac3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.744+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb780a3ad0 0x7fcb780ac3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fcb6800c710 tx=0x7fcb68007c10).stop 2020-11-20T12:02:55.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 shutdown_connections 2020-11-20T12:02:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb54056d20 0x7fcb540591e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb78142ae0 0x7fcb78151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb780a3ad0 0x7fcb780ac3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 --2- 172.21.15.44:0/3633308322 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcb7813c360 0x7fcb781517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 >> 172.21.15.44:0/3633308322 conn(0x7fcb780624b0 msgr2=0x7fcb780a60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 shutdown_connections 2020-11-20T12:02:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.745+0000 7fcb7ce44700 1 -- 172.21.15.44:0/3633308322 wait complete. 2020-11-20T12:02:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2020-11-20T12:02:55.764 INFO:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2020-11-20T12:02:55.764 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2020-11-20T12:02:55.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.900+0000 7f08dc287700 1 Processor -- start 2020-11-20T12:02:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.900+0000 7f08dc287700 1 -- start start 2020-11-20T12:02:55.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08dc287700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f08d40ad550 0x7f08d40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08dc287700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d40a7200 con 0x7f08d40a8ea0 2020-11-20T12:02:55.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08dc287700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d4135240 con 0x7f08d40ad930 2020-11-20T12:02:55.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08dc287700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d40a6cc0 con 0x7f08d40ad550 2020-11-20T12:02:55.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08da824700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f08d40a8ea0 0x7f08d41420b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39658/0 (socket says 172.21.15.44:39658) 2020-11-20T12:02:55.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08da824700 1 -- 172.21.15.44:0/3959332025 learned_addr learned my addr 172.21.15.44:0/3959332025 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:02:55.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08da023700 1 --2- 172.21.15.44:0/3959332025 >> v2:172.21.15.79:3300/0 conn(0x7f08d40ad550 0x7f08d40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288773923 0 0) 0x7f08d40a7200 con 0x7f08d40a8ea0 2020-11-20T12:02:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08bc003240 con 0x7f08d40a8ea0 2020-11-20T12:02:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1504408994 0 0) 0x7f08bc003240 con 0x7f08d40a8ea0 2020-11-20T12:02:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 >> v2:172.21.15.79:3300/0 conn(0x7f08d40ad550 msgr2=0x7f08d40a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:55.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 --2- 172.21.15.44:0/3959332025 >> v2:172.21.15.79:3300/0 conn(0x7f08d40ad550 0x7f08d40a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 >> v1:172.21.15.79:6789/0 conn(0x7f08d40ad930 legacy=0x7f08d40a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40a6f60 con 0x7f08d40a8ea0 2020-11-20T12:02:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f08cc001860 con 0x7f08d40a8ea0 2020-11-20T12:02:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 >> v1:172.21.15.44:6789/0 conn(0x7f08d40a8ea0 legacy=0x7f08d41420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08bc003730 0x7f08bc003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08bc0044e0 0x7f08bc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc004e50 0x7f08bc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40a6cc0 con 0x7f08bc003730 2020-11-20T12:02:55.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc004260 con 0x7f08bc004e50 2020-11-20T12:02:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.901+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc00bc20 con 0x7f08bc0044e0 2020-11-20T12:02:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da023700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08bc003730 0x7f08bc003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da824700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc004e50 0x7f08bc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08d9822700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08bc0044e0 0x7f08bc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da023700 1 -- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08bc0044e0 msgr2=0x7f08bc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:55.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da023700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08bc0044e0 0x7f08bc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da023700 1 -- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc004e50 msgr2=0x7f08bc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:55.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da023700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc004e50 0x7f08bc00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da023700 1 -- 172.21.15.44:0/3959332025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc003240 con 0x7f08bc003730 2020-11-20T12:02:55.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da824700 1 -- 172.21.15.44:0/3959332025 reap_dead start 2020-11-20T12:02:55.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08da023700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08bc003730 0x7f08bc003b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f08c400bbe0 tx=0x7f08c4006df0).ready entity=mon.0 client_cookie=9e7b1af54e63d992 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:55.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400b2f0 con 0x7f08bc003730 2020-11-20T12:02:55.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c400ea30 con 0x7f08bc003730 2020-11-20T12:02:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.902+0000 7f08d9021700 1 -- 172.21.15.44:0/3959332025 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400ebc0 con 0x7f08bc003730 2020-11-20T12:02:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.903+0000 7f08dc287700 1 -- 172.21.15.44:0/3959332025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08bc003730 msgr2=0x7f08bc003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.903+0000 7f08dc287700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08bc003730 0x7f08bc003b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f08c400bbe0 tx=0x7f08c4006df0).stop 2020-11-20T12:02:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.903+0000 7f08dc287700 1 -- 172.21.15.44:0/3959332025 shutdown_connections 2020-11-20T12:02:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.903+0000 7f08dc287700 1 --2- 172.21.15.44:0/3959332025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08bc003730 0x7f08bc003b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.903+0000 7f08dc287700 1 -- 172.21.15.44:0/3959332025 >> 172.21.15.44:0/3959332025 conn(0x7f08d40624b0 msgr2=0x7f08d40ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.903+0000 7f08dc287700 1 -- 172.21.15.44:0/3959332025 shutdown_connections 2020-11-20T12:02:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.903+0000 7f08dc287700 1 -- 172.21.15.44:0/3959332025 wait complete. 2020-11-20T12:02:55.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 Processor -- start 2020-11-20T12:02:55.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 -- start start 2020-11-20T12:02:55.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc003730 0x7f08d4066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d4139fa0 0x7f08d413aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08d413d530 0x7f08d413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d413b890 con 0x7f08d4139fa0 2020-11-20T12:02:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d413a4b0 con 0x7f08bc003730 2020-11-20T12:02:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08dc287700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d413a750 con 0x7f08d413d530 2020-11-20T12:02:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08d9822700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d4139fa0 0x7f08d413aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08d9822700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d4139fa0 0x7f08d413aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35920/0 (socket says 172.21.15.44:35920) 2020-11-20T12:02:55.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08d9822700 1 -- 172.21.15.44:0/2968672327 learned_addr learned my addr 172.21.15.44:0/2968672327 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:55.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08da023700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc003730 0x7f08d4066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.904+0000 7f08da824700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08d413d530 0x7f08d413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08d9822700 1 -- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08d413d530 msgr2=0x7f08d413b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08d9822700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08d413d530 0x7f08d413b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08d9822700 1 -- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc003730 msgr2=0x7f08d4066ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:55.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08d9822700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc003730 0x7f08d4066ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:55.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08d9822700 1 -- 172.21.15.44:0/2968672327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d413a9f0 con 0x7f08d4139fa0 2020-11-20T12:02:55.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08d9822700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d4139fa0 0x7f08d413aeb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f08d000f920 tx=0x7f08d0007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:55.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08c27fc700 1 -- 172.21.15.44:0/2968672327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0002760 con 0x7f08d4139fa0 2020-11-20T12:02:55.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08c27fc700 1 -- 172.21.15.44:0/2968672327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d00028c0 con 0x7f08d4139fa0 2020-11-20T12:02:55.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d4067900 con 0x7f08d4139fa0 2020-11-20T12:02:55.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08c27fc700 1 -- 172.21.15.44:0/2968672327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0002c40 con 0x7f08d4139fa0 2020-11-20T12:02:55.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.905+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d4148fb0 con 0x7f08d4139fa0 2020-11-20T12:02:55.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.906+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d413ab80 con 0x7f08d4139fa0 2020-11-20T12:02:55.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.906+0000 7f08c27fc700 1 -- 172.21.15.44:0/2968672327 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f08d001b030 con 0x7f08d4139fa0 2020-11-20T12:02:55.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.907+0000 7f08c27fc700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f08ac056d50 0x7f08ac059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:55.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.907+0000 7f08c27fc700 1 -- 172.21.15.44:0/2968672327 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f08d006f930 con 0x7f08d4139fa0 2020-11-20T12:02:55.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.907+0000 7f08da023700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f08ac056d50 0x7f08ac059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:55.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.907+0000 7f08da023700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f08ac056d50 0x7f08ac059210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f08c4006d00 tx=0x7f08c4019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:55.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:55.910+0000 7f08c27fc700 1 -- 172.21.15.44:0/2968672327 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f08d0045000 con 0x7f08d4139fa0 2020-11-20T12:02:56.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.098+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f08d40615e0 con 0x7f08d4139fa0 2020-11-20T12:02:56.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.743+0000 7f08c27fc700 1 -- 172.21.15.44:0/2968672327 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 154+0+0 (secure 0 0 0) 0x7f08d0019070 con 0x7f08d4139fa0 2020-11-20T12:02:56.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f08ac056d50 msgr2=0x7f08ac059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:56.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f08ac056d50 0x7f08ac059210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f08c4006d00 tx=0x7f08c4019000).stop 2020-11-20T12:02:56.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d4139fa0 msgr2=0x7f08d413aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:56.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d4139fa0 0x7f08d413aeb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f08d000f920 tx=0x7f08d0007b90).stop 2020-11-20T12:02:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 shutdown_connections 2020-11-20T12:02:56.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f08ac056d50 0x7f08ac059210 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f08d413d530 0x7f08d413b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08bc003730 0x7f08d4066ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 --2- 172.21.15.44:0/2968672327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08d4139fa0 0x7f08d413aeb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 >> 172.21.15.44:0/2968672327 conn(0x7f08d40624b0 msgr2=0x7f08d405ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:56.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.749+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 shutdown_connections 2020-11-20T12:02:56.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.750+0000 7f08dc287700 1 -- 172.21.15.44:0/2968672327 wait complete. 2020-11-20T12:02:56.769 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 erasure cephfs_data 2020-11-20T12:02:56.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.904+0000 7f1f9e006700 1 Processor -- start 2020-11-20T12:02:56.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.904+0000 7f1f9e006700 1 -- start start 2020-11-20T12:02:56.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f9e006700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060170 0x7f1f980a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:56.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f9e006700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1f98137cc0 0x7f1f98141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:56.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f9e006700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f980a4280 con 0x7f1f98060550 2020-11-20T12:02:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f9e006700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980a5b20 con 0x7f1f98137cc0 2020-11-20T12:02:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f9e006700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980a5d10 con 0x7f1f98060170 2020-11-20T12:02:56.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f97fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1f98137cc0 0x7f1f98141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:56.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f96ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1f98060550 0x7f1f981375b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39676/0 (socket says 172.21.15.44:39676) 2020-11-20T12:02:56.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f96ffd700 1 -- 172.21.15.44:0/58138426 learned_addr learned my addr 172.21.15.44:0/58138426 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:02:56.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f977fe700 1 --2- 172.21.15.44:0/58138426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060170 0x7f1f980a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:56.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f97fff700 1 -- 172.21.15.44:0/58138426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060170 msgr2=0x7f1f980a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:56.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f97fff700 1 --2- 172.21.15.44:0/58138426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060170 0x7f1f980a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f97fff700 1 -- 172.21.15.44:0/58138426 >> v1:172.21.15.44:6789/0 conn(0x7f1f98060550 legacy=0x7f1f981375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:56.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f97fff700 1 -- 172.21.15.44:0/58138426 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f980a6840 con 0x7f1f98137cc0 2020-11-20T12:02:56.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f967fc700 1 -- 172.21.15.44:0/58138426 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265105395 0 0) 0x7f1f980a4280 con 0x7f1f98060550 2020-11-20T12:02:56.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.905+0000 7f1f97fff700 1 --2- 172.21.15.44:0/58138426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98137cc0 0x7f1f98141890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1f90009c90 tx=0x7f1f900091d0).ready entity=mon.0 client_cookie=a54f9ac705394948 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:56.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f967fc700 1 -- 172.21.15.44:0/58138426 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000bbc0 con 0x7f1f98137cc0 2020-11-20T12:02:56.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f967fc700 1 -- 172.21.15.44:0/58138426 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9000bd20 con 0x7f1f98137cc0 2020-11-20T12:02:56.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f967fc700 1 -- 172.21.15.44:0/58138426 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90011610 con 0x7f1f98137cc0 2020-11-20T12:02:56.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 -- 172.21.15.44:0/58138426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98137cc0 msgr2=0x7f1f98141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:56.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 --2- 172.21.15.44:0/58138426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98137cc0 0x7f1f98141890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1f90009c90 tx=0x7f1f900091d0).stop 2020-11-20T12:02:56.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 -- 172.21.15.44:0/58138426 shutdown_connections 2020-11-20T12:02:56.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 --2- 172.21.15.44:0/58138426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98137cc0 0x7f1f98141890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 --2- 172.21.15.44:0/58138426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060170 0x7f1f980a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 -- 172.21.15.44:0/58138426 >> 172.21.15.44:0/58138426 conn(0x7f1f980624b0 msgr2=0x7f1f980ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:56.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 -- 172.21.15.44:0/58138426 shutdown_connections 2020-11-20T12:02:56.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.906+0000 7f1f9e006700 1 -- 172.21.15.44:0/58138426 wait complete. 2020-11-20T12:02:56.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.907+0000 7f1f9e006700 1 Processor -- start 2020-11-20T12:02:56.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.907+0000 7f1f9e006700 1 -- start start 2020-11-20T12:02:56.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f9e006700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98060170 0x7f1f98131bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:56.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f9e006700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060550 0x7f1f9812b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:56.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f9e006700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f98137cc0 0x7f1f98064f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:56.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f9e006700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98130290 con 0x7f1f98060170 2020-11-20T12:02:56.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f9e006700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980669a0 con 0x7f1f98137cc0 2020-11-20T12:02:56.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f9e006700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98066c40 con 0x7f1f98060550 2020-11-20T12:02:56.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98060170 0x7f1f98131bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:56.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98060170 0x7f1f98131bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35932/0 (socket says 172.21.15.44:35932) 2020-11-20T12:02:56.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 -- 172.21.15.44:0/1236972414 learned_addr learned my addr 172.21.15.44:0/1236972414 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:56.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f97fff700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f98137cc0 0x7f1f98064f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:56.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f96ffd700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060550 0x7f1f9812b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:56.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 -- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060550 msgr2=0x7f1f9812b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:56.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060550 0x7f1f9812b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 -- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f98137cc0 msgr2=0x7f1f98064f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:56.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f98137cc0 0x7f1f98064f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:56.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 -- 172.21.15.44:0/1236972414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98066ee0 con 0x7f1f98060170 2020-11-20T12:02:56.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f977fe700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98060170 0x7f1f98131bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1f8800c8f0 tx=0x7f1f88008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:56.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.908+0000 7f1f777fe700 1 -- 172.21.15.44:0/1236972414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88007710 con 0x7f1f98060170 2020-11-20T12:02:56.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.909+0000 7f1f777fe700 1 -- 172.21.15.44:0/1236972414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f88007870 con 0x7f1f98060170 2020-11-20T12:02:56.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.909+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f981316a0 con 0x7f1f98060170 2020-11-20T12:02:56.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.909+0000 7f1f777fe700 1 -- 172.21.15.44:0/1236972414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88007c10 con 0x7f1f98060170 2020-11-20T12:02:56.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.909+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f98067a00 con 0x7f1f98060170 2020-11-20T12:02:56.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.909+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f980a6330 con 0x7f1f98060170 2020-11-20T12:02:56.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.910+0000 7f1f777fe700 1 -- 172.21.15.44:0/1236972414 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1f8801b460 con 0x7f1f98060170 2020-11-20T12:02:56.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.910+0000 7f1f777fe700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f70056d50 0x7f1f70059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:56.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.910+0000 7f1f777fe700 1 -- 172.21.15.44:0/1236972414 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5863+0+0 (secure 0 0 0) 0x7f1f8806ffd0 con 0x7f1f98060170 2020-11-20T12:02:56.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.910+0000 7f1f96ffd700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f70056d50 0x7f1f70059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:56.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.911+0000 7f1f96ffd700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f70056d50 0x7f1f70059210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1f980a9320 tx=0x7f1f8c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:56.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:56.914+0000 7f1f777fe700 1 -- 172.21.15.44:0/1236972414 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1f88045490 con 0x7f1f98060170 2020-11-20T12:02:57.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:57.088+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7f1f980615e0 con 0x7f1f98060170 2020-11-20T12:02:58.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.761+0000 7f1f777fe700 1 -- 172.21.15.44:0/1236972414 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v23) v1 ==== 188+0+0 (secure 0 0 0) 0x7f1f88048ab0 con 0x7f1f98060170 2020-11-20T12:02:58.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f70056d50 msgr2=0x7f1f70059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f70056d50 0x7f1f70059210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1f980a9320 tx=0x7f1f8c009040).stop 2020-11-20T12:02:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98060170 msgr2=0x7f1f98131bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:58.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98060170 0x7f1f98131bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1f8800c8f0 tx=0x7f1f88008040).stop 2020-11-20T12:02:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 shutdown_connections 2020-11-20T12:02:58.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f70056d50 0x7f1f70059210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f98137cc0 0x7f1f98064f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f98060170 0x7f1f98131bb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 --2- 172.21.15.44:0/1236972414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f98060550 0x7f1f9812b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.765+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 >> 172.21.15.44:0/1236972414 conn(0x7f1f980624b0 msgr2=0x7f1f980a85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:58.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.768+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 shutdown_connections 2020-11-20T12:02:58.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.768+0000 7f1f9e006700 1 -- 172.21.15.44:0/1236972414 wait complete. 2020-11-20T12:02:58.773 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2020-11-20T12:02:58.787 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data allow_ec_overwrites true 2020-11-20T12:02:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.919+0000 7fb9cf843700 1 Processor -- start 2020-11-20T12:02:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.919+0000 7fb9cf843700 1 -- start start 2020-11-20T12:02:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9cf843700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad550 0x7fb9c80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9cf843700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb9c80ad930 0x7fb9c80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:58.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9cf843700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80a7200 con 0x7fb9c80a8ea0 2020-11-20T12:02:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9cf843700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c8135240 con 0x7fb9c80ad930 2020-11-20T12:02:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9cf843700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80a6cc0 con 0x7fb9c80ad550 2020-11-20T12:02:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9ccdde700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb9c80ad930 0x7fb9c80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9ccdde700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb9c80ad930 0x7fb9c80a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35942/0 (socket says 172.21.15.44:35942) 2020-11-20T12:02:58.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.920+0000 7fb9ccdde700 1 -- 172.21.15.44:0/1533848825 learned_addr learned my addr 172.21.15.44:0/1533848825 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cd5df700 1 --2- 172.21.15.44:0/1533848825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad550 0x7fb9c80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9ccdde700 1 -- 172.21.15.44:0/1533848825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad550 msgr2=0x7fb9c80a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9ccdde700 1 --2- 172.21.15.44:0/1533848825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad550 0x7fb9c80a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9ccdde700 1 -- 172.21.15.44:0/1533848825 >> v1:172.21.15.79:6789/0 conn(0x7fb9c80a8ea0 legacy=0x7fb9c813ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:02:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9ccdde700 1 -- 172.21.15.44:0/1533848825 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80a6f60 con 0x7fb9c80ad930 2020-11-20T12:02:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9ccdde700 1 --2- 172.21.15.44:0/1533848825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad930 0x7fb9c80a3700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb9c4006fd0 tx=0x7fb9c4006eb0).ready entity=mon.0 client_cookie=79b3c9822e17cc4e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9bbfff700 1 -- 172.21.15.44:0/1533848825 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4009920 con 0x7fb9c80ad930 2020-11-20T12:02:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9bbfff700 1 -- 172.21.15.44:0/1533848825 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4009a80 con 0x7fb9c80ad930 2020-11-20T12:02:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9bbfff700 1 -- 172.21.15.44:0/1533848825 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4009e00 con 0x7fb9c80ad930 2020-11-20T12:02:58.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 -- 172.21.15.44:0/1533848825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad930 msgr2=0x7fb9c80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:58.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 --2- 172.21.15.44:0/1533848825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad930 0x7fb9c80a3700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb9c4006fd0 tx=0x7fb9c4006eb0).stop 2020-11-20T12:02:58.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 -- 172.21.15.44:0/1533848825 shutdown_connections 2020-11-20T12:02:58.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 --2- 172.21.15.44:0/1533848825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad550 0x7fb9c80a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 --2- 172.21.15.44:0/1533848825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad930 0x7fb9c80a3700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 -- 172.21.15.44:0/1533848825 >> 172.21.15.44:0/1533848825 conn(0x7fb9c80624b0 msgr2=0x7fb9c80ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:58.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 -- 172.21.15.44:0/1533848825 shutdown_connections 2020-11-20T12:02:58.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.921+0000 7fb9cf843700 1 -- 172.21.15.44:0/1533848825 wait complete. 2020-11-20T12:02:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.922+0000 7fb9cf843700 1 Processor -- start 2020-11-20T12:02:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.922+0000 7fb9cf843700 1 -- start start 2020-11-20T12:02:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9cf843700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9c80a8ea0 0x7fb9c8066bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9cf843700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad550 0x7fb9c8130700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:58.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9cf843700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad930 0x7fb9c8138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9cf843700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80661a0 con 0x7fb9c80ad550 2020-11-20T12:02:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9cf843700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80670b0 con 0x7fb9c80a8ea0 2020-11-20T12:02:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9cf843700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8131260 con 0x7fb9c80ad930 2020-11-20T12:02:58.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9ccdde700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad550 0x7fb9c8130700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:58.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9ccdde700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad550 0x7fb9c8130700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35948/0 (socket says 172.21.15.44:35948) 2020-11-20T12:02:58.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9ccdde700 1 -- 172.21.15.44:0/73279933 learned_addr learned my addr 172.21.15.44:0/73279933 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:58.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.923+0000 7fb9cdde0700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad930 0x7fb9c8138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:58.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9cd5df700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9c80a8ea0 0x7fb9c8066bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:58.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9ccdde700 1 -- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad930 msgr2=0x7fb9c8138f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9ccdde700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad930 0x7fb9c8138f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9ccdde700 1 -- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9c80a8ea0 msgr2=0x7fb9c8066bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9ccdde700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9c80a8ea0 0x7fb9c8066bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9ccdde700 1 -- 172.21.15.44:0/73279933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8065a60 con 0x7fb9c80ad550 2020-11-20T12:02:58.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9cd5df700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9c80a8ea0 0x7fb9c8066bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:02:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9ccdde700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad550 0x7fb9c8130700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fb9c4006ef0 tx=0x7fb9c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9b97fa700 1 -- 172.21.15.44:0/73279933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c401e460 con 0x7fb9c80ad550 2020-11-20T12:02:58.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c8065d00 con 0x7fb9c80ad550 2020-11-20T12:02:58.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c8065440 con 0x7fb9c80ad550 2020-11-20T12:02:58.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9b97fa700 1 -- 172.21.15.44:0/73279933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c401e5c0 con 0x7fb9c80ad550 2020-11-20T12:02:58.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.924+0000 7fb9b97fa700 1 -- 172.21.15.44:0/73279933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c401e940 con 0x7fb9c80ad550 2020-11-20T12:02:58.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.925+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c8065e90 con 0x7fb9c80ad550 2020-11-20T12:02:58.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.925+0000 7fb9b97fa700 1 -- 172.21.15.44:0/73279933 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb9c4025030 con 0x7fb9c80ad550 2020-11-20T12:02:58.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.926+0000 7fb9b97fa700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb9a0056d50 0x7fb9a0059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:58.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.926+0000 7fb9b97fa700 1 -- 172.21.15.44:0/73279933 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7fb9c40718a0 con 0x7fb9c80ad550 2020-11-20T12:02:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.926+0000 7fb9cd5df700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb9a0056d50 0x7fb9a0059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:58.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.926+0000 7fb9cd5df700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb9a0056d50 0x7fb9a0059210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb9bc00c880 tx=0x7fb9bc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:58.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:58.930+0000 7fb9b97fa700 1 -- 172.21.15.44:0/73279933 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb9c4046cb0 con 0x7fb9c80ad550 2020-11-20T12:02:59.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.105+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb9c80615e0 con 0x7fb9c80ad550 2020-11-20T12:02:59.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.767+0000 7fb9b97fa700 1 -- 172.21.15.44:0/73279933 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb9c4023070 con 0x7fb9c80ad550 2020-11-20T12:02:59.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.773+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb9a0056d50 msgr2=0x7fb9a0059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:59.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.773+0000 7fb9cf843700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb9a0056d50 0x7fb9a0059210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb9bc00c880 tx=0x7fb9bc008040).stop 2020-11-20T12:02:59.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.773+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad550 msgr2=0x7fb9c8130700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:59.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.773+0000 7fb9cf843700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad550 0x7fb9c8130700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fb9c4006ef0 tx=0x7fb9c4015000).stop 2020-11-20T12:02:59.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.776+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 shutdown_connections 2020-11-20T12:02:59.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.776+0000 7fb9cf843700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb9a0056d50 0x7fb9a0059210 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:59.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.776+0000 7fb9cf843700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb9c80ad930 0x7fb9c8138f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:59.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.776+0000 7fb9cf843700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb9c80a8ea0 0x7fb9c8066bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:59.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.776+0000 7fb9cf843700 1 --2- 172.21.15.44:0/73279933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9c80ad550 0x7fb9c8130700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:59.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.776+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 >> 172.21.15.44:0/73279933 conn(0x7fb9c80624b0 msgr2=0x7fb9c80a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:59.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.777+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 shutdown_connections 2020-11-20T12:02:59.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.777+0000 7fb9cf843700 1 -- 172.21.15.44:0/73279933 wait complete. 2020-11-20T12:02:59.782 INFO:teuthology.orchestra.run.smithi044.stderr:set pool 4 allow_ec_overwrites to true 2020-11-20T12:02:59.796 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:02:59.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.932+0000 7f5f9ad7b700 1 Processor -- start 2020-11-20T12:02:59.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.932+0000 7f5f9ad7b700 1 -- start start 2020-11-20T12:02:59.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.932+0000 7f5f9ad7b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5f940a8ea0 0x7f5f941430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:59.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f9ad7b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f940a7200 con 0x7f5f940ad550 2020-11-20T12:02:59.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f9ad7b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f94135240 con 0x7f5f940ad930 2020-11-20T12:02:59.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f9ad7b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f940a6cc0 con 0x7f5f940a8ea0 2020-11-20T12:02:59.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f99318700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5f940a8ea0 0x7f5f941430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:59.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f99318700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5f940a8ea0 0x7f5f941430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35954/0 (socket says 172.21.15.44:35954) 2020-11-20T12:02:59.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f98b17700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5f940ad550 0x7f5f940a5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39702/0 (socket says 172.21.15.44:39702) 2020-11-20T12:02:59.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f99318700 1 -- 172.21.15.44:0/2589461952 learned_addr learned my addr 172.21.15.44:0/2589461952 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:59.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f99318700 1 -- 172.21.15.44:0/2589461952 >> v1:172.21.15.79:6789/0 conn(0x7f5f940ad930 legacy=0x7f5f940a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:59.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f99318700 1 -- 172.21.15.44:0/2589461952 >> v1:172.21.15.44:6789/0 conn(0x7f5f940ad550 legacy=0x7f5f940a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:59.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f937fe700 1 -- 172.21.15.44:0/2589461952 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470168711 0 0) 0x7f5f940a7200 con 0x7f5f940ad550 2020-11-20T12:02:59.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f99318700 1 -- 172.21.15.44:0/2589461952 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940a6f60 con 0x7f5f940a8ea0 2020-11-20T12:02:59.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f99318700 1 --2- 172.21.15.44:0/2589461952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940a8ea0 0x7f5f941430c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5f84006fd0 tx=0x7f5f84006eb0).ready entity=mon.0 client_cookie=a28123fd8ec1d3a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:59.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f937fe700 1 -- 172.21.15.44:0/2589461952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840098d0 con 0x7f5f940a8ea0 2020-11-20T12:02:59.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f937fe700 1 -- 172.21.15.44:0/2589461952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84009a30 con 0x7f5f940a8ea0 2020-11-20T12:02:59.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.933+0000 7f5f937fe700 1 -- 172.21.15.44:0/2589461952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84009db0 con 0x7f5f940a8ea0 2020-11-20T12:02:59.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.934+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2589461952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940a8ea0 msgr2=0x7f5f941430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:59.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.934+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2589461952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940a8ea0 0x7f5f941430c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5f84006fd0 tx=0x7f5f84006eb0).stop 2020-11-20T12:02:59.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.934+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2589461952 shutdown_connections 2020-11-20T12:02:59.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.934+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2589461952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940a8ea0 0x7f5f941430c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:59.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.934+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2589461952 >> 172.21.15.44:0/2589461952 conn(0x7f5f940624b0 msgr2=0x7f5f940ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:02:59.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.934+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2589461952 shutdown_connections 2020-11-20T12:02:59.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.934+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2589461952 wait complete. 2020-11-20T12:02:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.935+0000 7f5f9ad7b700 1 Processor -- start 2020-11-20T12:02:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.935+0000 7f5f9ad7b700 1 -- start start 2020-11-20T12:02:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.935+0000 7f5f9ad7b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f940a8ea0 0x7f5f9413e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:59.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.935+0000 7f5f9ad7b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940ad550 0x7f5f94146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.935+0000 7f5f9ad7b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f940ad930 0x7f5f9409e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.935+0000 7f5f9ad7b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9413d030 con 0x7f5f940ad550 2020-11-20T12:02:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.935+0000 7f5f9ad7b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9413eb90 con 0x7f5f940ad930 2020-11-20T12:02:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f9ad7b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9413d290 con 0x7f5f940a8ea0 2020-11-20T12:02:59.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940ad550 0x7f5f94146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940ad550 0x7f5f94146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35962/0 (socket says 172.21.15.44:35962) 2020-11-20T12:02:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 -- 172.21.15.44:0/2236666550 learned_addr learned my addr 172.21.15.44:0/2236666550 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:02:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f98b17700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f940a8ea0 0x7f5f9413e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f99318700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f940ad930 0x7f5f9409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:59.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 -- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f940a8ea0 msgr2=0x7f5f9413e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f940a8ea0 0x7f5f9413e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 -- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f940ad930 msgr2=0x7f5f9409e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:02:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f940ad930 0x7f5f9409e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:02:59.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 -- 172.21.15.44:0/2236666550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9413d530 con 0x7f5f940ad550 2020-11-20T12:02:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f93fff700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940ad550 0x7f5f94146850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5f88001db0 tx=0x7f5f88007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f90ff9700 1 -- 172.21.15.44:0/2236666550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88015070 con 0x7f5f940ad550 2020-11-20T12:02:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f90ff9700 1 -- 172.21.15.44:0/2236666550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f88007020 con 0x7f5f940ad550 2020-11-20T12:02:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f94146d40 con 0x7f5f940ad550 2020-11-20T12:02:59.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f90ff9700 1 -- 172.21.15.44:0/2236666550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f880073a0 con 0x7f5f940ad550 2020-11-20T12:02:59.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.936+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9413c360 con 0x7f5f940ad550 2020-11-20T12:02:59.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.937+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f940615e0 con 0x7f5f940ad550 2020-11-20T12:02:59.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.938+0000 7f5f90ff9700 1 -- 172.21.15.44:0/2236666550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f5f880108b0 con 0x7f5f940ad550 2020-11-20T12:02:59.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.938+0000 7f5f90ff9700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f70056d20 0x7f5f700591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:02:59.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.938+0000 7f5f90ff9700 1 -- 172.21.15.44:0/2236666550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f5f88001e40 con 0x7f5f940ad550 2020-11-20T12:02:59.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.938+0000 7f5f98b17700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f70056d20 0x7f5f700591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:02:59.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.938+0000 7f5f98b17700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f70056d20 0x7f5f700591e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5f8c001520 tx=0x7f5f8c007da0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:02:59.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:02:59.942+0000 7f5f90ff9700 1 -- 172.21.15.44:0/2236666550 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5f88046e90 con 0x7f5f940ad550 2020-11-20T12:03:00.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.177+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f5f940a69b0 con 0x7f5f940ad550 2020-11-20T12:03:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.180+0000 7f5f90ff9700 1 -- 172.21.15.44:0/2236666550 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7f5f8804a4b0 con 0x7f5f940ad550 2020-11-20T12:03:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f70056d20 msgr2=0x7f5f700591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f70056d20 0x7f5f700591e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5f8c001520 tx=0x7f5f8c007da0).stop 2020-11-20T12:03:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940ad550 msgr2=0x7f5f94146850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940ad550 0x7f5f94146850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5f88001db0 tx=0x7f5f88007c20).stop 2020-11-20T12:03:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 shutdown_connections 2020-11-20T12:03:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f70056d20 0x7f5f700591e0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0x7f5f8c001520 tx=0x7f5f8c007da0).stop 2020-11-20T12:03:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f940ad930 0x7f5f9409e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f940a8ea0 0x7f5f9413e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 --2- 172.21.15.44:0/2236666550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f940ad550 0x7f5f94146850 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0x7f5f88001db0 tx=0x7f5f88007c20).stop 2020-11-20T12:03:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.184+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 >> 172.21.15.44:0/2236666550 conn(0x7f5f940624b0 msgr2=0x7f5f9405ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.187+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 shutdown_connections 2020-11-20T12:03:00.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.187+0000 7f5f9ad7b700 1 -- 172.21.15.44:0/2236666550 wait complete. 2020-11-20T12:03:00.198 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 3 and data pool 4 2020-11-20T12:03:00.209 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.331+0000 7f9c7b47b700 1 Processor -- start 2020-11-20T12:03:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c7b47b700 1 -- start start 2020-11-20T12:03:00.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c7b47b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c7b47b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9c740aa400 0x7f9c740a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c7b47b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9c7413ac60 0x7f9c740a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c7b47b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c740a9770 con 0x7f9c7413ac60 2020-11-20T12:03:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c7b47b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c740a6da0 con 0x7f9c740aa400 2020-11-20T12:03:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c7b47b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740a7040 con 0x7f9c740aa020 2020-11-20T12:03:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.332+0000 7f9c79a18700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9c7413ac60 0x7f9c740a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9c7413ac60 0x7f9c740a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35982/0 (socket says 172.21.15.44:35982) 2020-11-20T12:03:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 -- 172.21.15.44:0/242693211 learned_addr learned my addr 172.21.15.44:0/242693211 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c78a16700 1 --2- 172.21.15.44:0/242693211 >> v2:172.21.15.79:3300/0 conn(0x7f9c740aa400 0x7f9c740a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79217700 1 --2- 172.21.15.44:0/242693211 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 -- 172.21.15.44:0/242693211 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 msgr2=0x7f9c740a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 --2- 172.21.15.44:0/242693211 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 -- 172.21.15.44:0/242693211 >> v2:172.21.15.79:3300/0 conn(0x7f9c740aa400 msgr2=0x7f9c740a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 --2- 172.21.15.44:0/242693211 >> v2:172.21.15.79:3300/0 conn(0x7f9c740aa400 0x7f9c740a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 -- 172.21.15.44:0/242693211 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740a6a90 con 0x7f9c7413ac60 2020-11-20T12:03:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c79a18700 1 --2- 172.21.15.44:0/242693211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7413ac60 0x7f9c740a3700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f9c74062b30 tx=0x7f9c6c009290).ready entity=mon.0 client_cookie=72cfddf88eae1158 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c67fff700 1 -- 172.21.15.44:0/242693211 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00bbc0 con 0x7f9c7413ac60 2020-11-20T12:03:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c67fff700 1 -- 172.21.15.44:0/242693211 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c00bd20 con 0x7f9c7413ac60 2020-11-20T12:03:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.333+0000 7f9c67fff700 1 -- 172.21.15.44:0/242693211 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c011610 con 0x7f9c7413ac60 2020-11-20T12:03:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 -- 172.21.15.44:0/242693211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7413ac60 msgr2=0x7f9c740a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/242693211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7413ac60 0x7f9c740a3700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f9c74062b30 tx=0x7f9c6c009290).stop 2020-11-20T12:03:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 -- 172.21.15.44:0/242693211 shutdown_connections 2020-11-20T12:03:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/242693211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c7413ac60 0x7f9c740a3700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/242693211 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/242693211 >> v2:172.21.15.79:3300/0 conn(0x7f9c740aa400 0x7f9c740a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 -- 172.21.15.44:0/242693211 >> 172.21.15.44:0/242693211 conn(0x7f9c74062690 msgr2=0x7f9c740a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 -- 172.21.15.44:0/242693211 shutdown_connections 2020-11-20T12:03:00.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.334+0000 7f9c7b47b700 1 -- 172.21.15.44:0/242693211 wait complete. 2020-11-20T12:03:00.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 Processor -- start 2020-11-20T12:03:00.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 -- start start 2020-11-20T12:03:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c740aa400 0x7f9c740aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c7413ac60 0x7f9c740ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c74135300 con 0x7f9c740aa400 2020-11-20T12:03:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740ad670 con 0x7f9c7413ac60 2020-11-20T12:03:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c7b47b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740a85d0 con 0x7f9c740aa020 2020-11-20T12:03:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c78a16700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c740aa400 0x7f9c740aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c78a16700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c740aa400 0x7f9c740aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35986/0 (socket says 172.21.15.44:35986) 2020-11-20T12:03:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.335+0000 7f9c78a16700 1 -- 172.21.15.44:0/1576539089 learned_addr learned my addr 172.21.15.44:0/1576539089 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c79217700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c79a18700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c7413ac60 0x7f9c740ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c78a16700 1 -- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 msgr2=0x7f9c740ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c78a16700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c78a16700 1 -- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c7413ac60 msgr2=0x7f9c740ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c78a16700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c7413ac60 0x7f9c740ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c78a16700 1 -- 172.21.15.44:0/1576539089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740adaa0 con 0x7f9c740aa400 2020-11-20T12:03:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c79a18700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c7413ac60 0x7f9c740ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c78a16700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c740aa400 0x7f9c740aac60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f9c7000d900 tx=0x7f9c70008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c657fa700 1 -- 172.21.15.44:0/1576539089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000f5e0 con 0x7f9c740aa400 2020-11-20T12:03:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c657fa700 1 -- 172.21.15.44:0/1576539089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7000f740 con 0x7f9c740aa400 2020-11-20T12:03:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c740add40 con 0x7f9c740aa400 2020-11-20T12:03:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c657fa700 1 -- 172.21.15.44:0/1576539089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000fb00 con 0x7f9c740aa400 2020-11-20T12:03:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.336+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c740673c0 con 0x7f9c740aa400 2020-11-20T12:03:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.337+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c740446d0 con 0x7f9c740aa400 2020-11-20T12:03:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.337+0000 7f9c657fa700 1 -- 172.21.15.44:0/1576539089 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9c7001c460 con 0x7f9c740aa400 2020-11-20T12:03:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.337+0000 7f9c657fa700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c4c056d20 0x7f9c4c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.338+0000 7f9c657fa700 1 -- 172.21.15.44:0/1576539089 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9c70070020 con 0x7f9c740aa400 2020-11-20T12:03:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.338+0000 7f9c79217700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c4c056d20 0x7f9c4c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.338+0000 7f9c79217700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c4c056d20 0x7f9c4c0591e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9c6800ba00 tx=0x7f9c68008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.341+0000 7f9c657fa700 1 -- 172.21.15.44:0/1576539089 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9c700452b0 con 0x7f9c740aa400 2020-11-20T12:03:00.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.514+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c740613b0 con 0x7f9c740aa400 2020-11-20T12:03:00.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.515+0000 7f9c657fa700 1 -- 172.21.15.44:0/1576539089 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7f9c700488d0 con 0x7f9c740aa400 2020-11-20T12:03:00.519 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:00.519 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c4c056d20 msgr2=0x7f9c4c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c4c056d20 0x7f9c4c0591e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9c6800ba00 tx=0x7f9c68008040).stop 2020-11-20T12:03:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c740aa400 msgr2=0x7f9c740aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c740aa400 0x7f9c740aac60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f9c7000d900 tx=0x7f9c70008040).stop 2020-11-20T12:03:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 shutdown_connections 2020-11-20T12:03:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c4c056d20 0x7f9c4c0591e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9c7413ac60 0x7f9c740ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9c740aa020 0x7f9c740ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 --2- 172.21.15.44:0/1576539089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c740aa400 0x7f9c740aac60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 >> 172.21.15.44:0/1576539089 conn(0x7f9c74062690 msgr2=0x7f9c74135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 shutdown_connections 2020-11-20T12:03:00.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.516+0000 7f9c7b47b700 1 -- 172.21.15.44:0/1576539089 wait complete. 2020-11-20T12:03:00.532 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5d5e71700 1 Processor -- start 2020-11-20T12:03:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5d5e71700 1 -- start start 2020-11-20T12:03:00.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5d5e71700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d00aa020 0x7fc5d00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5d5e71700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc5d00aa400 0x7fc5d00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5d5e71700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d00a9770 con 0x7fc5d013ac60 2020-11-20T12:03:00.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5d5e71700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00a6da0 con 0x7fc5d00aa400 2020-11-20T12:03:00.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5d5e71700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00a7040 con 0x7fc5d00aa020 2020-11-20T12:03:00.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5ceffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc5d00aa400 0x7fc5d00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5cffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc5d013ac60 0x7fc5d00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39742/0 (socket says 172.21.15.44:39742) 2020-11-20T12:03:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5cffff700 1 -- 172.21.15.44:0/3236282366 learned_addr learned my addr 172.21.15.44:0/3236282366 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.656+0000 7fc5ceffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc5d00aa400 0x7fc5d00a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35996/0 (socket says 172.21.15.44:35996) 2020-11-20T12:03:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5cf7fe700 1 --2- 172.21.15.44:0/3236282366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d00aa020 0x7fc5d00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ceffd700 1 -- 172.21.15.44:0/3236282366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d00aa020 msgr2=0x7fc5d00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:00.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ceffd700 1 --2- 172.21.15.44:0/3236282366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d00aa020 0x7fc5d00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ceffd700 1 -- 172.21.15.44:0/3236282366 >> v1:172.21.15.44:6789/0 conn(0x7fc5d013ac60 legacy=0x7fc5d00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ceffd700 1 -- 172.21.15.44:0/3236282366 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d00a6a90 con 0x7fc5d00aa400 2020-11-20T12:03:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ce7fc700 1 -- 172.21.15.44:0/3236282366 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271277828 0 0) 0x7fc5c8001450 con 0x7fc5d013ac60 2020-11-20T12:03:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ceffd700 1 --2- 172.21.15.44:0/3236282366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d00a4fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc5c400dc30 tx=0x7fc5c4006e80).ready entity=mon.0 client_cookie=1ab05fdaf8092355 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:00.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ce7fc700 1 -- 172.21.15.44:0/3236282366 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400bc60 con 0x7fc5d00aa400 2020-11-20T12:03:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ce7fc700 1 -- 172.21.15.44:0/3236282366 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c400bdc0 con 0x7fc5d00aa400 2020-11-20T12:03:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5ce7fc700 1 -- 172.21.15.44:0/3236282366 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4010680 con 0x7fc5d00aa400 2020-11-20T12:03:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5d5e71700 1 -- 172.21.15.44:0/3236282366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 msgr2=0x7fc5d00a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.657+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/3236282366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d00a4fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fc5c400dc30 tx=0x7fc5c4006e80).stop 2020-11-20T12:03:00.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.658+0000 7fc5d5e71700 1 -- 172.21.15.44:0/3236282366 shutdown_connections 2020-11-20T12:03:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.658+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/3236282366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d00aa020 0x7fc5d00a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.658+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/3236282366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d00a4fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.658+0000 7fc5d5e71700 1 -- 172.21.15.44:0/3236282366 >> 172.21.15.44:0/3236282366 conn(0x7fc5d0062690 msgr2=0x7fc5d00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.658+0000 7fc5d5e71700 1 -- 172.21.15.44:0/3236282366 shutdown_connections 2020-11-20T12:03:00.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.658+0000 7fc5d5e71700 1 -- 172.21.15.44:0/3236282366 wait complete. 2020-11-20T12:03:00.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 Processor -- start 2020-11-20T12:03:00.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 -- start start 2020-11-20T12:03:00.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d00aa020 0x7fc5d00acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d0065770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d013ac60 0x7fc5d00a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00acff0 con 0x7fc5d00aa400 2020-11-20T12:03:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d014da20 con 0x7fc5d00aa020 2020-11-20T12:03:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5d5e71700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d014dcc0 con 0x7fc5d013ac60 2020-11-20T12:03:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5ceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d0065770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5ceffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d0065770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36002/0 (socket says 172.21.15.44:36002) 2020-11-20T12:03:00.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5ceffd700 1 -- 172.21.15.44:0/1783897217 learned_addr learned my addr 172.21.15.44:0/1783897217 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:00.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5cffff700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d013ac60 0x7fc5d00a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.659+0000 7fc5cf7fe700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d00aa020 0x7fc5d00acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5ceffd700 1 -- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d013ac60 msgr2=0x7fc5d00a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5ceffd700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d013ac60 0x7fc5d00a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5ceffd700 1 -- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d00aa020 msgr2=0x7fc5d00acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5ceffd700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d00aa020 0x7fc5d00acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5ceffd700 1 -- 172.21.15.44:0/1783897217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d014df60 con 0x7fc5d00aa400 2020-11-20T12:03:00.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5ceffd700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d0065770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc5c4012000 tx=0x7fc5c4006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:00.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5af7fe700 1 -- 172.21.15.44:0/1783897217 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400bc60 con 0x7fc5d00aa400 2020-11-20T12:03:00.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5af7fe700 1 -- 172.21.15.44:0/1783897217 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c401bab0 con 0x7fc5d00aa400 2020-11-20T12:03:00.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d0065c60 con 0x7fc5d00aa400 2020-11-20T12:03:00.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5af7fe700 1 -- 172.21.15.44:0/1783897217 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c401be30 con 0x7fc5d00aa400 2020-11-20T12:03:00.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.660+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d014d660 con 0x7fc5d00aa400 2020-11-20T12:03:00.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.661+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d014d880 con 0x7fc5d00aa400 2020-11-20T12:03:00.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.661+0000 7fc5af7fe700 1 -- 172.21.15.44:0/1783897217 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc5c4018610 con 0x7fc5d00aa400 2020-11-20T12:03:00.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.662+0000 7fc5af7fe700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc5a8056d20 0x7fc5a80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:00.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.662+0000 7fc5af7fe700 1 -- 172.21.15.44:0/1783897217 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc5c4072130 con 0x7fc5d00aa400 2020-11-20T12:03:00.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.662+0000 7fc5cf7fe700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc5a8056d20 0x7fc5a80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:00.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.662+0000 7fc5cf7fe700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc5a8056d20 0x7fc5a80591e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc5c000da50 tx=0x7fc5c0006f90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:00.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.665+0000 7fc5af7fe700 1 -- 172.21.15.44:0/1783897217 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc5d014d880 con 0x7fc5d00aa400 2020-11-20T12:03:00.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.842+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5d014d880 con 0x7fc5d00aa400 2020-11-20T12:03:00.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.843+0000 7fc5af7fe700 1 -- 172.21.15.44:0/1783897217 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7fc5d014d880 con 0x7fc5d00aa400 2020-11-20T12:03:00.847 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:00.847 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:00.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc5a8056d20 msgr2=0x7fc5a80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc5a8056d20 0x7fc5a80591e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc5c000da50 tx=0x7fc5c0006f90).stop 2020-11-20T12:03:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 msgr2=0x7fc5d0065770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d0065770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc5c4012000 tx=0x7fc5c4006e80).stop 2020-11-20T12:03:00.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 shutdown_connections 2020-11-20T12:03:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc5a8056d20 0x7fc5a80591e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5d013ac60 0x7fc5d00a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5d00aa020 0x7fc5d00acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 --2- 172.21.15.44:0/1783897217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5d00aa400 0x7fc5d0065770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:00.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 >> 172.21.15.44:0/1783897217 conn(0x7fc5d0062690 msgr2=0x7fc5d0060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 shutdown_connections 2020-11-20T12:03:00.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.845+0000 7fc5d5e71700 1 -- 172.21.15.44:0/1783897217 wait complete. 2020-11-20T12:03:00.860 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:03:00.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.993+0000 7fe47152e700 1 Processor -- start 2020-11-20T12:03:00.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.993+0000 7fe47152e700 1 -- start start 2020-11-20T12:03:01.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe47152e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0ad550 0x7fe46c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:01.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe47152e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe46c0ad930 0x7fe46c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:01.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe47152e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe46c0a8ea0 0x7fe46c0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe47152e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c0a7200 con 0x7fe46c0ad930 2020-11-20T12:03:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe47152e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c135240 con 0x7fe46c0a8ea0 2020-11-20T12:03:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe47152e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0a6cc0 con 0x7fe46c0ad550 2020-11-20T12:03:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe46c0ad930 0x7fe46c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:01.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe46c0ad930 0x7fe46c0a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36006/0 (socket says 172.21.15.44:36006) 2020-11-20T12:03:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 -- 172.21.15.44:0/1675004146 learned_addr learned my addr 172.21.15.44:0/1675004146 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46affd700 1 --2- 172.21.15.44:0/1675004146 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0ad550 0x7fe46c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46b7fe700 1 --2- 172.21.15.44:0/1675004146 >> v2:172.21.15.79:3300/0 conn(0x7fe46c0a8ea0 0x7fe46c0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:01.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 -- 172.21.15.44:0/1675004146 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0ad550 msgr2=0x7fe46c0a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 --2- 172.21.15.44:0/1675004146 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0ad550 0x7fe46c0a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 -- 172.21.15.44:0/1675004146 >> v2:172.21.15.79:3300/0 conn(0x7fe46c0a8ea0 msgr2=0x7fe46c0a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 --2- 172.21.15.44:0/1675004146 >> v2:172.21.15.79:3300/0 conn(0x7fe46c0a8ea0 0x7fe46c0a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.994+0000 7fe46a7fc700 1 -- 172.21.15.44:0/1675004146 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0a6f60 con 0x7fe46c0ad930 2020-11-20T12:03:01.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe46a7fc700 1 --2- 172.21.15.44:0/1675004146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c0a3700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe46c0a5cd0 tx=0x7fe45c009290).ready entity=mon.0 client_cookie=c9dc226dff439d58 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe469ffb700 1 -- 172.21.15.44:0/1675004146 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00bc50 con 0x7fe46c0ad930 2020-11-20T12:03:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe469ffb700 1 -- 172.21.15.44:0/1675004146 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c00bdb0 con 0x7fe46c0ad930 2020-11-20T12:03:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe469ffb700 1 -- 172.21.15.44:0/1675004146 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011610 con 0x7fe46c0ad930 2020-11-20T12:03:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 -- 172.21.15.44:0/1675004146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 msgr2=0x7fe46c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 --2- 172.21.15.44:0/1675004146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c0a3700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe46c0a5cd0 tx=0x7fe45c009290).stop 2020-11-20T12:03:01.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 -- 172.21.15.44:0/1675004146 shutdown_connections 2020-11-20T12:03:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 --2- 172.21.15.44:0/1675004146 >> v2:172.21.15.79:3300/0 conn(0x7fe46c0a8ea0 0x7fe46c0a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 --2- 172.21.15.44:0/1675004146 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0ad550 0x7fe46c0a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 --2- 172.21.15.44:0/1675004146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c0a3700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:01.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 -- 172.21.15.44:0/1675004146 >> 172.21.15.44:0/1675004146 conn(0x7fe46c0624b0 msgr2=0x7fe46c0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 -- 172.21.15.44:0/1675004146 shutdown_connections 2020-11-20T12:03:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.995+0000 7fe47152e700 1 -- 172.21.15.44:0/1675004146 wait complete. 2020-11-20T12:03:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.996+0000 7fe47152e700 1 Processor -- start 2020-11-20T12:03:01.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.996+0000 7fe47152e700 1 -- start start 2020-11-20T12:03:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe47152e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0a8ea0 0x7fe46c1374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe47152e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe46c0ad550 0x7fe46c12bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe47152e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c065710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe47152e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0652b0 con 0x7fe46c0ad930 2020-11-20T12:03:01.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe47152e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c137b90 con 0x7fe46c0ad550 2020-11-20T12:03:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe47152e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c137e30 con 0x7fe46c0a8ea0 2020-11-20T12:03:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c065710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36014/0 (socket says 172.21.15.44:36014) 2020-11-20T12:03:01.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 -- 172.21.15.44:0/1895615052 learned_addr learned my addr 172.21.15.44:0/1895615052 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46a7fc700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe46c0ad550 0x7fe46c12bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46affd700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0a8ea0 0x7fe46c1374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 -- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0a8ea0 msgr2=0x7fe46c1374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0a8ea0 0x7fe46c1374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:01.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 -- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe46c0ad550 msgr2=0x7fe46c12bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:01.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe46c0ad550 0x7fe46c12bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:01.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 -- 172.21.15.44:0/1895615052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0679d0 con 0x7fe46c0ad930 2020-11-20T12:03:01.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46a7fc700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe46c0ad550 0x7fe46c12bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:01.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46affd700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0a8ea0 0x7fe46c1374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:01.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.997+0000 7fe46b7fe700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c065710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe46400a730 tx=0x7fe464008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.998+0000 7fe452ffd700 1 -- 172.21.15.44:0/1895615052 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46400e5d0 con 0x7fe46c0ad930 2020-11-20T12:03:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.998+0000 7fe452ffd700 1 -- 172.21.15.44:0/1895615052 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46400c030 con 0x7fe46c0ad930 2020-11-20T12:03:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.998+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c067c80 con 0x7fe46c0ad930 2020-11-20T12:03:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.998+0000 7fe452ffd700 1 -- 172.21.15.44:0/1895615052 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46400e980 con 0x7fe46c0ad930 2020-11-20T12:03:01.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.998+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c065e40 con 0x7fe46c0ad930 2020-11-20T12:03:01.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.999+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c1379e0 con 0x7fe46c0ad930 2020-11-20T12:03:01.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.999+0000 7fe452ffd700 1 -- 172.21.15.44:0/1895615052 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe464019030 con 0x7fe46c0ad930 2020-11-20T12:03:01.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.999+0000 7fe452ffd700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe448056d50 0x7fe448059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:01.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.999+0000 7fe452ffd700 1 -- 172.21.15.44:0/1895615052 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe46406f9f0 con 0x7fe46c0ad930 2020-11-20T12:03:01.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:00.999+0000 7fe46affd700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe448056d50 0x7fe448059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:01.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:01.000+0000 7fe46affd700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe448056d50 0x7fe448059210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe46000b7f0 tx=0x7fe460008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:01.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:01.003+0000 7fe452ffd700 1 -- 172.21.15.44:0/1895615052 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe464071e60 con 0x7fe46c0ad930 2020-11-20T12:03:01.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:01.236+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe46c0615e0 con 0x7fe46c0ad930 2020-11-20T12:03:02.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.184+0000 7fe452ffd700 1 -- 172.21.15.44:0/1895615052 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe464044ce0 con 0x7fe46c0ad930 2020-11-20T12:03:02.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.186+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe448056d50 msgr2=0x7fe448059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:02.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.186+0000 7fe47152e700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe448056d50 0x7fe448059210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe46000b7f0 tx=0x7fe460008040).stop 2020-11-20T12:03:02.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.186+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 msgr2=0x7fe46c065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:02.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.186+0000 7fe47152e700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c065710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe46400a730 tx=0x7fe464008040).stop 2020-11-20T12:03:02.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.186+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 shutdown_connections 2020-11-20T12:03:02.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.186+0000 7fe47152e700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe448056d50 0x7fe448059210 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.187+0000 7fe47152e700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c0ad930 0x7fe46c065710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.187+0000 7fe47152e700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe46c0a8ea0 0x7fe46c1374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.187+0000 7fe47152e700 1 --2- 172.21.15.44:0/1895615052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe46c0ad550 0x7fe46c12bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.187+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 >> 172.21.15.44:0/1895615052 conn(0x7fe46c0624b0 msgr2=0x7fe46c0a7710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:02.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.187+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 shutdown_connections 2020-11-20T12:03:02.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.187+0000 7fe47152e700 1 -- 172.21.15.44:0/1895615052 wait complete. 2020-11-20T12:03:02.204 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2020-11-20T12:03:02.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.337+0000 7fd131144700 1 Processor -- start 2020-11-20T12:03:02.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.337+0000 7fd131144700 1 -- start start 2020-11-20T12:03:02.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd131144700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:02.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd131144700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd12c0ad930 0x7fd12c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:02.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd131144700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd12c0a8ea0 0x7fd12c0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:02.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd131144700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0a7200 con 0x7fd12c0ad930 2020-11-20T12:03:02.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd131144700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c135240 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd131144700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0a6cc0 con 0x7fd12c0ad550 2020-11-20T12:03:02.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd12a59c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd12c0ad930 0x7fd12c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:02.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd12a59c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd12c0ad930 0x7fd12c0a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36018/0 (socket says 172.21.15.44:36018) 2020-11-20T12:03:02.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd12a59c700 1 -- 172.21.15.44:0/2546728807 learned_addr learned my addr 172.21.15.44:0/2546728807 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:02.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd12ad9d700 1 --2- 172.21.15.44:0/2546728807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:02.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.338+0000 7fd12b59e700 1 --2- 172.21.15.44:0/2546728807 >> v2:172.21.15.79:3300/0 conn(0x7fd12c0a8ea0 0x7fd12c0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:02.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd12a59c700 1 -- 172.21.15.44:0/2546728807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 msgr2=0x7fd12c0a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:02.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd12a59c700 1 --2- 172.21.15.44:0/2546728807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c0a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd12a59c700 1 -- 172.21.15.44:0/2546728807 >> v2:172.21.15.79:3300/0 conn(0x7fd12c0a8ea0 msgr2=0x7fd12c0a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:02.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd12a59c700 1 --2- 172.21.15.44:0/2546728807 >> v2:172.21.15.79:3300/0 conn(0x7fd12c0a8ea0 0x7fd12c0a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd12a59c700 1 -- 172.21.15.44:0/2546728807 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c0a6f60 con 0x7fd12c0ad930 2020-11-20T12:03:02.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd12b59e700 1 --2- 172.21.15.44:0/2546728807 >> v2:172.21.15.79:3300/0 conn(0x7fd12c0a8ea0 0x7fd12c0a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:02.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd12a59c700 1 --2- 172.21.15.44:0/2546728807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0ad930 0x7fd12c0a3700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd12c0a5cd0 tx=0x7fd11c006eb0).ready entity=mon.0 client_cookie=efb590f6b3add2c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:02.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd129d9b700 1 -- 172.21.15.44:0/2546728807 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c009960 con 0x7fd12c0ad930 2020-11-20T12:03:02.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd129d9b700 1 -- 172.21.15.44:0/2546728807 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c009ac0 con 0x7fd12c0ad930 2020-11-20T12:03:02.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd129d9b700 1 -- 172.21.15.44:0/2546728807 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c009e40 con 0x7fd12c0ad930 2020-11-20T12:03:02.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 -- 172.21.15.44:0/2546728807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0ad930 msgr2=0x7fd12c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:02.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 --2- 172.21.15.44:0/2546728807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0ad930 0x7fd12c0a3700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd12c0a5cd0 tx=0x7fd11c006eb0).stop 2020-11-20T12:03:02.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 -- 172.21.15.44:0/2546728807 shutdown_connections 2020-11-20T12:03:02.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 --2- 172.21.15.44:0/2546728807 >> v2:172.21.15.79:3300/0 conn(0x7fd12c0a8ea0 0x7fd12c0a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 --2- 172.21.15.44:0/2546728807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c0a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 --2- 172.21.15.44:0/2546728807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0ad930 0x7fd12c0a3700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 -- 172.21.15.44:0/2546728807 >> 172.21.15.44:0/2546728807 conn(0x7fd12c0624b0 msgr2=0x7fd12c0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:02.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 -- 172.21.15.44:0/2546728807 shutdown_connections 2020-11-20T12:03:02.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.339+0000 7fd131144700 1 -- 172.21.15.44:0/2546728807 wait complete. 2020-11-20T12:03:02.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.340+0000 7fd131144700 1 Processor -- start 2020-11-20T12:03:02.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.340+0000 7fd131144700 1 -- start start 2020-11-20T12:03:02.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd131144700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0a8ea0 0x7fd12c13d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:02.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd131144700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c09e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:02.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd131144700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd12c0ad930 0x7fd12c0a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:02.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd131144700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c13a9f0 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd131144700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c13d810 con 0x7fd12c0ad930 2020-11-20T12:03:02.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd131144700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c13bbc0 con 0x7fd12c0ad550 2020-11-20T12:03:02.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0a8ea0 0x7fd12c13d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:02.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0a8ea0 0x7fd12c13d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36022/0 (socket says 172.21.15.44:36022) 2020-11-20T12:03:02.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 -- 172.21.15.44:0/3151153805 learned_addr learned my addr 172.21.15.44:0/3151153805 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:02.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12a59c700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c09e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:02.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12b59e700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd12c0ad930 0x7fd12c0a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:02.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 -- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 msgr2=0x7fd12c09e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:02.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c09e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 -- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd12c0ad930 msgr2=0x7fd12c0a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:02.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd12c0ad930 0x7fd12c0a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:02.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.341+0000 7fd12ad9d700 1 -- 172.21.15.44:0/3151153805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c13bdf0 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.342+0000 7fd12ad9d700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0a8ea0 0x7fd12c13d320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd12000db90 tx=0x7fd120008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:02.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.342+0000 7fd112ffd700 1 -- 172.21.15.44:0/3151153805 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120009790 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.342+0000 7fd112ffd700 1 -- 172.21.15.44:0/3151153805 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1200098f0 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.342+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c13c330 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.342+0000 7fd112ffd700 1 -- 172.21.15.44:0/3151153805 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120009c90 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.342+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c13b070 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.343+0000 7fd112ffd700 1 -- 172.21.15.44:0/3151153805 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd120013480 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.343+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c0acc20 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.343+0000 7fd112ffd700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd108056d20 0x7fd1080591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:02.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.343+0000 7fd112ffd700 1 -- 172.21.15.44:0/3151153805 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd1200706b0 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.343+0000 7fd12a59c700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd108056d20 0x7fd1080591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.344+0000 7fd12a59c700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd108056d20 0x7fd1080591e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd12c0a4cb0 tx=0x7fd11c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:02.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.347+0000 7fd112ffd700 1 -- 172.21.15.44:0/3151153805 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd120045830 con 0x7fd12c0a8ea0 2020-11-20T12:03:02.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:02.580+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fd12c0ad0f0 con 0x7fd12c0a8ea0 2020-11-20T12:03:03.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.189+0000 7fd112ffd700 1 -- 172.21.15.44:0/3151153805 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v7) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd120048e50 con 0x7fd12c0a8ea0 2020-11-20T12:03:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd108056d20 msgr2=0x7fd1080591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd108056d20 0x7fd1080591e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd12c0a4cb0 tx=0x7fd11c015000).stop 2020-11-20T12:03:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0a8ea0 msgr2=0x7fd12c13d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0a8ea0 0x7fd12c13d320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd12000db90 tx=0x7fd120008040).stop 2020-11-20T12:03:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 shutdown_connections 2020-11-20T12:03:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd108056d20 0x7fd1080591e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd12c0ad930 0x7fd12c0a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd12c0a8ea0 0x7fd12c13d320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 --2- 172.21.15.44:0/3151153805 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd12c0ad550 0x7fd12c09e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 >> 172.21.15.44:0/3151153805 conn(0x7fd12c0624b0 msgr2=0x7fd12c13df50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 shutdown_connections 2020-11-20T12:03:03.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.192+0000 7fd131144700 1 -- 172.21.15.44:0/3151153805 wait complete. 2020-11-20T12:03:03.603 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:03:03.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.753+0000 7f18a7e2a700 1 Processor -- start 2020-11-20T12:03:03.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.753+0000 7f18a7e2a700 1 -- start start 2020-11-20T12:03:03.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a7e2a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 0x7f18a00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a7e2a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f18a013ac60 0x7f18a00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a7e2a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a00a9770 con 0x7f18a00aa400 2020-11-20T12:03:03.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a7e2a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a00a56c0 con 0x7f18a013ac60 2020-11-20T12:03:03.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a7e2a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00a6da0 con 0x7f18a00aa020 2020-11-20T12:03:03.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a53c5700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f18a00aa400 0x7f18a00a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39778/0 (socket says 172.21.15.44:39778) 2020-11-20T12:03:03.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a53c5700 1 -- 172.21.15.44:0/999637862 learned_addr learned my addr 172.21.15.44:0/999637862 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:03.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a5bc6700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 0x7f18a00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793757484 0 0) 0x7f189c001450 con 0x7f18a00aa400 2020-11-20T12:03:03.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a63c7700 1 --2- 172.21.15.44:0/999637862 >> v2:172.21.15.79:3300/0 conn(0x7f18a013ac60 0x7f18a00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1888003240 con 0x7f18a00aa400 2020-11-20T12:03:03.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4057536943 0 0) 0x7f1888003240 con 0x7f18a00aa400 2020-11-20T12:03:03.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 msgr2=0x7f18a00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:03.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 0x7f18a00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 >> v2:172.21.15.79:3300/0 conn(0x7f18a013ac60 msgr2=0x7f18a00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:03.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 --2- 172.21.15.44:0/999637862 >> v2:172.21.15.79:3300/0 conn(0x7f18a013ac60 0x7f18a00a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00a7040 con 0x7f18a00aa400 2020-11-20T12:03:03.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.754+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f189c0019f0 con 0x7f18a00aa400 2020-11-20T12:03:03.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 >> v1:172.21.15.44:6789/0 conn(0x7f18a00aa400 legacy=0x7f18a00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:03.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 0x7f1888003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18880044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18880049d0 0x7f1888004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00a56c0 con 0x7f1888003c40 2020-11-20T12:03:03.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00a6da0 con 0x7f18880049d0 2020-11-20T12:03:03.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1888004200 con 0x7f18a00aa020 2020-11-20T12:03:03.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a53c5700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18880044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a5bc6700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 0x7f1888003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a63c7700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18880049d0 0x7f1888004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a53c5700 1 -- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 msgr2=0x7f1888003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:03.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a53c5700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00aa020 0x7f1888003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a53c5700 1 -- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18880049d0 msgr2=0x7f1888004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:03.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a53c5700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18880049d0 0x7f1888004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a53c5700 1 -- 172.21.15.44:0/999637862 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1888003240 con 0x7f1888003c40 2020-11-20T12:03:03.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a53c5700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18880044e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f18a0062aa0 tx=0x7f18a00a5f30).ready entity=mon.0 client_cookie=8a384002bd07283 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:03.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c005b80 con 0x7f1888003c40 2020-11-20T12:03:03.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c005ce0 con 0x7f1888003c40 2020-11-20T12:03:03.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.755+0000 7f18a4bc4700 1 -- 172.21.15.44:0/999637862 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c007360 con 0x7f1888003c40 2020-11-20T12:03:03.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.756+0000 7f18a7e2a700 1 -- 172.21.15.44:0/999637862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 msgr2=0x7f18880044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:03.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.756+0000 7f18a7e2a700 1 --2- 172.21.15.44:0/999637862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18880044e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f18a0062aa0 tx=0x7f18a00a5f30).stop 2020-11-20T12:03:03.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.756+0000 7f18a63c7700 1 -- 172.21.15.44:0/999637862 reap_dead start 2020-11-20T12:03:03.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.756+0000 7f18a7e2a700 1 -- 172.21.15.44:0/999637862 shutdown_connections 2020-11-20T12:03:03.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.756+0000 7f18a7e2a700 1 -- 172.21.15.44:0/999637862 >> 172.21.15.44:0/999637862 conn(0x7f18a0062690 msgr2=0x7f18a00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:03.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.756+0000 7f18a7e2a700 1 -- 172.21.15.44:0/999637862 shutdown_connections 2020-11-20T12:03:03.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.756+0000 7f18a7e2a700 1 -- 172.21.15.44:0/999637862 wait complete. 2020-11-20T12:03:03.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.757+0000 7f18a7e2a700 1 Processor -- start 2020-11-20T12:03:03.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.757+0000 7f18a7e2a700 1 -- start start 2020-11-20T12:03:03.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a7e2a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18a0067500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a7e2a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18a009d470 0x7f18a00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a7e2a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00a2f60 0x7f18a0130c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a7e2a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a009ea90 con 0x7f1888003c40 2020-11-20T12:03:03.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a7e2a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00ad480 con 0x7f18a009d470 2020-11-20T12:03:03.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a7e2a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a009ec80 con 0x7f18a00a2f60 2020-11-20T12:03:03.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18a0067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18a0067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36042/0 (socket says 172.21.15.44:36042) 2020-11-20T12:03:03.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 -- 172.21.15.44:0/464693251 learned_addr learned my addr 172.21.15.44:0/464693251 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:03.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a63c7700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00a2f60 0x7f18a0130c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a53c5700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18a009d470 0x7f18a00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 -- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00a2f60 msgr2=0x7f18a0130c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:03.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00a2f60 0x7f18a0130c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 -- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18a009d470 msgr2=0x7f18a00a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:03.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18a009d470 0x7f18a00a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:03.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 -- 172.21.15.44:0/464693251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a009ef20 con 0x7f1888003c40 2020-11-20T12:03:03.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a53c5700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18a009d470 0x7f18a00a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:03.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a5bc6700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18a0067500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1890011cf0 tx=0x7f1890006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:03.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f188dffb700 1 -- 172.21.15.44:0/464693251 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890011ea0 con 0x7f1888003c40 2020-11-20T12:03:03.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f188dffb700 1 -- 172.21.15.44:0/464693251 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189000fca0 con 0x7f1888003c40 2020-11-20T12:03:03.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f188dffb700 1 -- 172.21.15.44:0/464693251 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890014440 con 0x7f1888003c40 2020-11-20T12:03:03.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.758+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a009f1c0 con 0x7f1888003c40 2020-11-20T12:03:03.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.759+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a00680f0 con 0x7f1888003c40 2020-11-20T12:03:03.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.760+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a0061650 con 0x7f1888003c40 2020-11-20T12:03:03.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.760+0000 7f188dffb700 1 -- 172.21.15.44:0/464693251 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f18900147f0 con 0x7f1888003c40 2020-11-20T12:03:03.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.760+0000 7f188dffb700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1878056d20 0x7f18780591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.760+0000 7f188dffb700 1 -- 172.21.15.44:0/464693251 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1890070bb0 con 0x7f1888003c40 2020-11-20T12:03:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.760+0000 7f18a53c5700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1878056d20 0x7f18780591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.761+0000 7f18a53c5700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1878056d20 0x7f18780591e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f189c0021f0 tx=0x7f189c014000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:03.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.764+0000 7f188dffb700 1 -- 172.21.15.44:0/464693251 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1890077080 con 0x7f1888003c40 2020-11-20T12:03:04.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.999+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18a013aa40 con 0x7f1888003c40 2020-11-20T12:03:04.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:03.999+0000 7f188dffb700 1 -- 172.21.15.44:0/464693251 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2131 (secure 0 0 0) 0x7f1890018020 con 0x7f1888003c40 2020-11-20T12:03:04.004 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:04.004 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/4254760701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":4254760701},{"type":"v1","addr":"172.21.15.79:6835","nonce":4254760701}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:03:00.178663+0000","modified":"2020-11-20T12:03:03.184476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/2557888073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2557888073},{"type":"v1","addr":"172.21.15.44:6833","nonce":2557888073}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:03:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1878056d20 msgr2=0x7f18780591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1878056d20 0x7f18780591e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f189c0021f0 tx=0x7f189c014000).stop 2020-11-20T12:03:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 msgr2=0x7f18a0067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18a0067500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1890011cf0 tx=0x7f1890006df0).stop 2020-11-20T12:03:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 shutdown_connections 2020-11-20T12:03:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1878056d20 0x7f18780591e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f18a00a2f60 0x7f18a0130c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1888003c40 0x7f18a0067500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 --2- 172.21.15.44:0/464693251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18a009d470 0x7f18a00a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 >> 172.21.15.44:0/464693251 conn(0x7f18a0062690 msgr2=0x7f18a0134a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 shutdown_connections 2020-11-20T12:03:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.001+0000 7f18a7e2a700 1 -- 172.21.15.44:0/464693251 wait complete. 2020-11-20T12:03:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2020-11-20T12:03:04.016 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:04.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.153+0000 7fcde6798700 1 Processor -- start 2020-11-20T12:03:04.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.153+0000 7fcde6798700 1 -- start start 2020-11-20T12:03:04.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.153+0000 7fcde6798700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcde00a3e10 0x7fcde00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.153+0000 7fcde6798700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde00a9770 con 0x7fcde00aa400 2020-11-20T12:03:04.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.153+0000 7fcde6798700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde00a6da0 con 0x7fcde00aa020 2020-11-20T12:03:04.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.153+0000 7fcde6798700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde00a56c0 con 0x7fcde00a3e10 2020-11-20T12:03:04.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcddf7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcde00aa400 0x7fcde00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39798/0 (socket says 172.21.15.44:39798) 2020-11-20T12:03:04.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcddf7fe700 1 -- 172.21.15.44:0/1280643334 learned_addr learned my addr 172.21.15.44:0/1280643334 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:04.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcde4d35700 1 --2- 172.21.15.44:0/1280643334 >> v2:172.21.15.79:3300/0 conn(0x7fcde00a3e10 0x7fcde00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcddeffd700 1 -- 172.21.15.44:0/1280643334 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498019652 0 0) 0x7fcdd40010d0 con 0x7fcde00aa400 2020-11-20T12:03:04.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcddeffd700 1 -- 172.21.15.44:0/1280643334 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdcc003240 con 0x7fcde00aa400 2020-11-20T12:03:04.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcde4d35700 1 -- 172.21.15.44:0/1280643334 >> v1:172.21.15.79:6789/0 conn(0x7fcde00aa020 legacy=0x7fcde00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcde4d35700 1 -- 172.21.15.44:0/1280643334 >> v1:172.21.15.44:6789/0 conn(0x7fcde00aa400 legacy=0x7fcde00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcde4d35700 1 -- 172.21.15.44:0/1280643334 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00a7040 con 0x7fcde00a3e10 2020-11-20T12:03:04.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcde4d35700 1 --2- 172.21.15.44:0/1280643334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00a3e10 0x7fcde00a9280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fcdd8009c90 tx=0x7fcdd80091d0).ready entity=mon.1 client_cookie=d6e112ab49a0536 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcddeffd700 1 -- 172.21.15.44:0/1280643334 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800ba70 con 0x7fcde00a3e10 2020-11-20T12:03:04.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcddeffd700 1 -- 172.21.15.44:0/1280643334 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd800bbd0 con 0x7fcde00a3e10 2020-11-20T12:03:04.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.154+0000 7fcddeffd700 1 -- 172.21.15.44:0/1280643334 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011400 con 0x7fcde00a3e10 2020-11-20T12:03:04.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.155+0000 7fcde6798700 1 -- 172.21.15.44:0/1280643334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00a3e10 msgr2=0x7fcde00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.155+0000 7fcde6798700 1 --2- 172.21.15.44:0/1280643334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00a3e10 0x7fcde00a9280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fcdd8009c90 tx=0x7fcdd80091d0).stop 2020-11-20T12:03:04.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.155+0000 7fcde6798700 1 -- 172.21.15.44:0/1280643334 shutdown_connections 2020-11-20T12:03:04.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.155+0000 7fcde6798700 1 --2- 172.21.15.44:0/1280643334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00a3e10 0x7fcde00a9280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.155+0000 7fcde6798700 1 -- 172.21.15.44:0/1280643334 >> 172.21.15.44:0/1280643334 conn(0x7fcde0062690 msgr2=0x7fcde00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:04.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.155+0000 7fcde6798700 1 -- 172.21.15.44:0/1280643334 shutdown_connections 2020-11-20T12:03:04.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.155+0000 7fcde6798700 1 -- 172.21.15.44:0/1280643334 wait complete. 2020-11-20T12:03:04.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.156+0000 7fcde6798700 1 Processor -- start 2020-11-20T12:03:04.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.156+0000 7fcde6798700 1 -- start start 2020-11-20T12:03:04.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcde00a3e10 0x7fcde0134790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00aa020 0x7fcde00ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcde00aa400 0x7fcde014b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00ac1b0 con 0x7fcde00aa400 2020-11-20T12:03:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00ac450 con 0x7fcde00aa020 2020-11-20T12:03:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00ac6f0 con 0x7fcde00a3e10 2020-11-20T12:03:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcde00aa400 0x7fcde014b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcde00aa400 0x7fcde014b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36060/0 (socket says 172.21.15.44:36060) 2020-11-20T12:03:04.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 -- 172.21.15.44:0/2680889021 learned_addr learned my addr 172.21.15.44:0/2680889021 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcddf7fe700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00aa020 0x7fcde00ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcddffff700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcde00a3e10 0x7fcde0134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 -- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcde00a3e10 msgr2=0x7fcde0134790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcde00a3e10 0x7fcde0134790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 -- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00aa020 msgr2=0x7fcde00ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00aa020 0x7fcde00ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 -- 172.21.15.44:0/2680889021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00ad860 con 0x7fcde00aa400 2020-11-20T12:03:04.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcddffff700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcde00a3e10 0x7fcde0134790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcddf7fe700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00aa020 0x7fcde00ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde4d35700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcde00aa400 0x7fcde014b2e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fcde0061280 tx=0x7fcdd8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcdc7fff700 1 -- 172.21.15.44:0/2680889021 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8010700 con 0x7fcde00aa400 2020-11-20T12:03:04.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcdc7fff700 1 -- 172.21.15.44:0/2680889021 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8010860 con 0x7fcde00aa400 2020-11-20T12:03:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde00abc30 con 0x7fcde00aa400 2020-11-20T12:03:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcdc7fff700 1 -- 172.21.15.44:0/2680889021 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8010be0 con 0x7fcde00aa400 2020-11-20T12:03:04.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.157+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde014ba10 con 0x7fcde00aa400 2020-11-20T12:03:04.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.158+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde00ac8b0 con 0x7fcde00aa400 2020-11-20T12:03:04.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.159+0000 7fcdc7fff700 1 -- 172.21.15.44:0/2680889021 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcdd8024030 con 0x7fcde00aa400 2020-11-20T12:03:04.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.159+0000 7fcdc7fff700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdbc056d20 0x7fcdbc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.159+0000 7fcdc7fff700 1 -- 172.21.15.44:0/2680889021 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcdd8071c60 con 0x7fcde00aa400 2020-11-20T12:03:04.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.159+0000 7fcddffff700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdbc056d20 0x7fcdbc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.160+0000 7fcddffff700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdbc056d20 0x7fcdbc0591e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcdd00014d0 tx=0x7fcdd0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.163+0000 7fcdc7fff700 1 -- 172.21.15.44:0/2680889021 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcdd8047000 con 0x7fcde00aa400 2020-11-20T12:03:04.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.337+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcde00abfb0 con 0x7fcde00aa400 2020-11-20T12:03:04.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.338+0000 7fcdc7fff700 1 -- 172.21.15.44:0/2680889021 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7fcdd80302a0 con 0x7fcde00aa400 2020-11-20T12:03:04.343 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:04.343 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:04.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdbc056d20 msgr2=0x7fcdbc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdbc056d20 0x7fcdbc0591e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcdd00014d0 tx=0x7fcdd0009040).stop 2020-11-20T12:03:04.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcde00aa400 msgr2=0x7fcde014b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcde00aa400 0x7fcde014b2e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fcde0061280 tx=0x7fcdd8016000).stop 2020-11-20T12:03:04.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 shutdown_connections 2020-11-20T12:03:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdbc056d20 0x7fcdbc0591e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcde00aa400 0x7fcde014b2e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcde00a3e10 0x7fcde0134790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 --2- 172.21.15.44:0/2680889021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcde00aa020 0x7fcde00ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 >> 172.21.15.44:0/2680889021 conn(0x7fcde0062690 msgr2=0x7fcde0139180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:04.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 shutdown_connections 2020-11-20T12:03:04.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.340+0000 7fcde6798700 1 -- 172.21.15.44:0/2680889021 wait complete. 2020-11-20T12:03:04.356 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:04.356 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-20T12:03:04.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.481+0000 7ffbc7966700 1 Processor -- start 2020-11-20T12:03:04.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.481+0000 7ffbc7966700 1 -- start start 2020-11-20T12:03:04.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.481+0000 7ffbc7966700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ffbc00aa020 0x7ffbc00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.481+0000 7ffbc7966700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa400 0x7ffbc00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.481+0000 7ffbc7966700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc00a9770 con 0x7ffbc013ac60 2020-11-20T12:03:04.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.481+0000 7ffbc7966700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc00a6da0 con 0x7ffbc00aa020 2020-11-20T12:03:04.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.481+0000 7ffbc7966700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00a7040 con 0x7ffbc00aa400 2020-11-20T12:03:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5f03700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ffbc013ac60 0x7ffbc00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39816/0 (socket says 172.21.15.44:39816) 2020-11-20T12:03:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5702700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ffbc00aa020 0x7ffbc00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5f03700 1 -- 172.21.15.44:0/3350397451 learned_addr learned my addr 172.21.15.44:0/3350397451 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:04.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc4f01700 1 --2- 172.21.15.44:0/3350397451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa400 0x7ffbc00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5702700 1 -- 172.21.15.44:0/3350397451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa400 msgr2=0x7ffbc00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:04.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5702700 1 --2- 172.21.15.44:0/3350397451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa400 0x7ffbc00a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5702700 1 -- 172.21.15.44:0/3350397451 >> v1:172.21.15.44:6789/0 conn(0x7ffbc013ac60 legacy=0x7ffbc00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbaffff700 1 -- 172.21.15.44:0/3350397451 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166148217 0 0) 0x7ffbb8001450 con 0x7ffbc013ac60 2020-11-20T12:03:04.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5702700 1 -- 172.21.15.44:0/3350397451 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00a6a90 con 0x7ffbc00aa020 2020-11-20T12:03:04.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc4f01700 1 --2- 172.21.15.44:0/3350397451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa400 0x7ffbc00a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:04.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbc5702700 1 --2- 172.21.15.44:0/3350397451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa020 0x7ffbc00a9280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7ffbb000b9a0 tx=0x7ffbb0006df0).ready entity=mon.0 client_cookie=2d24e293ec9bec63 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbaffff700 1 -- 172.21.15.44:0/3350397451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00097b0 con 0x7ffbc00aa020 2020-11-20T12:03:04.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbaffff700 1 -- 172.21.15.44:0/3350397451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb0009910 con 0x7ffbc00aa020 2020-11-20T12:03:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.482+0000 7ffbaffff700 1 -- 172.21.15.44:0/3350397451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0009c90 con 0x7ffbc00aa020 2020-11-20T12:03:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 -- 172.21.15.44:0/3350397451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa020 msgr2=0x7ffbc00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 --2- 172.21.15.44:0/3350397451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa020 0x7ffbc00a9280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7ffbb000b9a0 tx=0x7ffbb0006df0).stop 2020-11-20T12:03:04.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 -- 172.21.15.44:0/3350397451 shutdown_connections 2020-11-20T12:03:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 --2- 172.21.15.44:0/3350397451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa020 0x7ffbc00a9280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 --2- 172.21.15.44:0/3350397451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa400 0x7ffbc00a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 -- 172.21.15.44:0/3350397451 >> 172.21.15.44:0/3350397451 conn(0x7ffbc0062690 msgr2=0x7ffbc00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:04.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 -- 172.21.15.44:0/3350397451 shutdown_connections 2020-11-20T12:03:04.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.483+0000 7ffbc7966700 1 -- 172.21.15.44:0/3350397451 wait complete. 2020-11-20T12:03:04.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.484+0000 7ffbc7966700 1 Processor -- start 2020-11-20T12:03:04.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.484+0000 7ffbc7966700 1 -- start start 2020-11-20T12:03:04.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.484+0000 7ffbc7966700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa020 0x7ffbc00a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.484+0000 7ffbc7966700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa400 0x7ffbc009ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc7966700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc013ac60 0x7ffbc009e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc7966700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc009e7a0 con 0x7ffbc00aa400 2020-11-20T12:03:04.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc7966700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00a29d0 con 0x7ffbc013ac60 2020-11-20T12:03:04.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc7966700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00a2c30 con 0x7ffbc00aa020 2020-11-20T12:03:04.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa400 0x7ffbc009ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa400 0x7ffbc009ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36072/0 (socket says 172.21.15.44:36072) 2020-11-20T12:03:04.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 -- 172.21.15.44:0/4025181552 learned_addr learned my addr 172.21.15.44:0/4025181552 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:04.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc5702700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa020 0x7ffbc00a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc5f03700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc013ac60 0x7ffbc009e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 -- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa020 msgr2=0x7ffbc00a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa020 0x7ffbc00a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 -- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc013ac60 msgr2=0x7ffbc009e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc013ac60 0x7ffbc009e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 -- 172.21.15.44:0/4025181552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00a2ed0 con 0x7ffbc00aa400 2020-11-20T12:03:04.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc4f01700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa400 0x7ffbc009ddc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ffbbc00cb50 tx=0x7ffbbc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbad7fa700 1 -- 172.21.15.44:0/4025181552 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0077d0 con 0x7ffbc00aa400 2020-11-20T12:03:04.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbad7fa700 1 -- 172.21.15.44:0/4025181552 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc007930 con 0x7ffbc00aa400 2020-11-20T12:03:04.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbad7fa700 1 -- 172.21.15.44:0/4025181552 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc007cb0 con 0x7ffbc00aa400 2020-11-20T12:03:04.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc00a3170 con 0x7ffbc00aa400 2020-11-20T12:03:04.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.485+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc00adc10 con 0x7ffbc00aa400 2020-11-20T12:03:04.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.486+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc009dc30 con 0x7ffbc00aa400 2020-11-20T12:03:04.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.486+0000 7ffbad7fa700 1 -- 172.21.15.44:0/4025181552 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ffbbc011620 con 0x7ffbc00aa400 2020-11-20T12:03:04.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.487+0000 7ffbad7fa700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb98056d20 0x7ffb980591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.487+0000 7ffbad7fa700 1 -- 172.21.15.44:0/4025181552 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ffbbc0702e0 con 0x7ffbc00aa400 2020-11-20T12:03:04.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.487+0000 7ffbc5702700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb98056d20 0x7ffb980591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.487+0000 7ffbc5702700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb98056d20 0x7ffb980591e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ffbb000b150 tx=0x7ffbb0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.490+0000 7ffbad7fa700 1 -- 172.21.15.44:0/4025181552 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffbbc0455d0 con 0x7ffbc00aa400 2020-11-20T12:03:04.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.664+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbc0138c00 con 0x7ffbc00aa400 2020-11-20T12:03:04.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.665+0000 7ffbad7fa700 1 -- 172.21.15.44:0/4025181552 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7ffbbc048bf0 con 0x7ffbc00aa400 2020-11-20T12:03:04.669 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:04.669 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb98056d20 msgr2=0x7ffb980591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb98056d20 0x7ffb980591e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ffbb000b150 tx=0x7ffbb0015000).stop 2020-11-20T12:03:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa400 msgr2=0x7ffbc009ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa400 0x7ffbc009ddc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ffbbc00cb50 tx=0x7ffbbc008040).stop 2020-11-20T12:03:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 shutdown_connections 2020-11-20T12:03:04.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb98056d20 0x7ffb980591e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffbc013ac60 0x7ffbc009e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffbc00aa020 0x7ffbc00a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 --2- 172.21.15.44:0/4025181552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbc00aa400 0x7ffbc009ddc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 >> 172.21.15.44:0/4025181552 conn(0x7ffbc0062690 msgr2=0x7ffbc0060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:04.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.666+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 shutdown_connections 2020-11-20T12:03:04.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.667+0000 7ffbc7966700 1 -- 172.21.15.44:0/4025181552 wait complete. 2020-11-20T12:03:04.682 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:02:44.611733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:02:52.195125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:02:55.303534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:02:57.791405+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:03:04.683 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:03:04.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.820+0000 7efd17a59700 1 Processor -- start 2020-11-20T12:03:04.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.820+0000 7efd17a59700 1 -- start start 2020-11-20T12:03:04.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.820+0000 7efd17a59700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efd10060170 0x7efd100a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.820+0000 7efd17a59700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efd10060550 0x7efd101375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.820+0000 7efd17a59700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd100a4280 con 0x7efd10137aa0 2020-11-20T12:03:04.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.820+0000 7efd17a59700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd100a5b20 con 0x7efd10060170 2020-11-20T12:03:04.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.820+0000 7efd17a59700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd10137f10 con 0x7efd10060550 2020-11-20T12:03:04.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efd10060170 0x7efd100a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efd10060170 0x7efd100a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36078/0 (socket says 172.21.15.44:36078) 2020-11-20T12:03:04.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 -- 172.21.15.44:0/3142453607 learned_addr learned my addr 172.21.15.44:0/3142453607 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:04.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd14ff4700 1 --2- 172.21.15.44:0/3142453607 >> v2:172.21.15.79:3300/0 conn(0x7efd10060550 0x7efd101375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 -- 172.21.15.44:0/3142453607 >> v2:172.21.15.79:3300/0 conn(0x7efd10060550 msgr2=0x7efd101375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:04.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 --2- 172.21.15.44:0/3142453607 >> v2:172.21.15.79:3300/0 conn(0x7efd10060550 0x7efd101375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 -- 172.21.15.44:0/3142453607 >> v1:172.21.15.79:6789/0 conn(0x7efd10137aa0 legacy=0x7efd1013a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 -- 172.21.15.44:0/3142453607 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100a5d10 con 0x7efd10060170 2020-11-20T12:03:04.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd157f5700 1 --2- 172.21.15.44:0/3142453607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060170 0x7efd100a46b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7efd10063240 tx=0x7efd04006eb0).ready entity=mon.0 client_cookie=89b6a86ec6ff943e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd03fff700 1 -- 172.21.15.44:0/3142453607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04009810 con 0x7efd10060170 2020-11-20T12:03:04.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd03fff700 1 -- 172.21.15.44:0/3142453607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd04009970 con 0x7efd10060170 2020-11-20T12:03:04.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.821+0000 7efd03fff700 1 -- 172.21.15.44:0/3142453607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04009cf0 con 0x7efd10060170 2020-11-20T12:03:04.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 -- 172.21.15.44:0/3142453607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060170 msgr2=0x7efd100a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 --2- 172.21.15.44:0/3142453607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060170 0x7efd100a46b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7efd10063240 tx=0x7efd04006eb0).stop 2020-11-20T12:03:04.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 -- 172.21.15.44:0/3142453607 shutdown_connections 2020-11-20T12:03:04.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 --2- 172.21.15.44:0/3142453607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060170 0x7efd100a46b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 --2- 172.21.15.44:0/3142453607 >> v2:172.21.15.79:3300/0 conn(0x7efd10060550 0x7efd101375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 -- 172.21.15.44:0/3142453607 >> 172.21.15.44:0/3142453607 conn(0x7efd100624b0 msgr2=0x7efd100ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:04.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 -- 172.21.15.44:0/3142453607 shutdown_connections 2020-11-20T12:03:04.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.822+0000 7efd17a59700 1 -- 172.21.15.44:0/3142453607 wait complete. 2020-11-20T12:03:04.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 Processor -- start 2020-11-20T12:03:04.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 -- start start 2020-11-20T12:03:04.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efd10060170 0x7efd10066660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060550 0x7efd1014d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd10137aa0 0x7efd101360f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd10065f60 con 0x7efd10060550 2020-11-20T12:03:04.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd10067560 con 0x7efd10137aa0 2020-11-20T12:03:04.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.823+0000 7efd17a59700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd100661c0 con 0x7efd10060170 2020-11-20T12:03:04.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060550 0x7efd1014d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060550 0x7efd1014d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36086/0 (socket says 172.21.15.44:36086) 2020-11-20T12:03:04.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 -- 172.21.15.44:0/1152738489 learned_addr learned my addr 172.21.15.44:0/1152738489 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:04.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd15ff6700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd10137aa0 0x7efd101360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd157f5700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efd10060170 0x7efd10066660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 -- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efd10060170 msgr2=0x7efd10066660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efd10060170 0x7efd10066660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 -- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd10137aa0 msgr2=0x7efd101360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:04.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd10137aa0 0x7efd101360f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:04.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 -- 172.21.15.44:0/1152738489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1012f440 con 0x7efd10060550 2020-11-20T12:03:04.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd15ff6700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd10137aa0 0x7efd101360f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:04.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd14ff4700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060550 0x7efd1014d2a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7efd0c00eba0 tx=0x7efd0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd017fa700 1 -- 172.21.15.44:0/1152738489 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00ca60 con 0x7efd10060550 2020-11-20T12:03:04.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd017fa700 1 -- 172.21.15.44:0/1152738489 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c00cbc0 con 0x7efd10060550 2020-11-20T12:03:04.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd101365e0 con 0x7efd10060550 2020-11-20T12:03:04.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd017fa700 1 -- 172.21.15.44:0/1152738489 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0123f0 con 0x7efd10060550 2020-11-20T12:03:04.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.824+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd10066e20 con 0x7efd10060550 2020-11-20T12:03:04.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.825+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10066380 con 0x7efd10060550 2020-11-20T12:03:04.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.825+0000 7efd017fa700 1 -- 172.21.15.44:0/1152738489 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7efd0c0127a0 con 0x7efd10060550 2020-11-20T12:03:04.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.826+0000 7efd017fa700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efce8056d20 0x7efce80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:04.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.826+0000 7efd017fa700 1 -- 172.21.15.44:0/1152738489 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efd0c0705d0 con 0x7efd10060550 2020-11-20T12:03:04.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.826+0000 7efd157f5700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efce8056d20 0x7efce80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:04.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.826+0000 7efd157f5700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efce8056d20 0x7efce80591e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7efd0400b1b0 tx=0x7efd04015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:04.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:04.829+0000 7efd017fa700 1 -- 172.21.15.44:0/1152738489 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efd0c0458c0 con 0x7efd10060550 2020-11-20T12:03:05.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.005+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efd100a3710 con 0x7efd10060550 2020-11-20T12:03:05.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.006+0000 7efd017fa700 1 -- 172.21.15.44:0/1152738489 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7efd0c048ee0 con 0x7efd10060550 2020-11-20T12:03:05.010 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2020-11-20T12:03:05.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efce8056d20 msgr2=0x7efce80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efce8056d20 0x7efce80591e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7efd0400b1b0 tx=0x7efd04015000).stop 2020-11-20T12:03:05.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060550 msgr2=0x7efd1014d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060550 0x7efd1014d2a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7efd0c00eba0 tx=0x7efd0c006df0).stop 2020-11-20T12:03:05.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 shutdown_connections 2020-11-20T12:03:05.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efce8056d20 0x7efce80591e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd10137aa0 0x7efd101360f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efd10060170 0x7efd10066660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 --2- 172.21.15.44:0/1152738489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd10060550 0x7efd1014d2a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 >> 172.21.15.44:0/1152738489 conn(0x7efd100624b0 msgr2=0x7efd100a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:05.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 shutdown_connections 2020-11-20T12:03:05.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.007+0000 7efd17a59700 1 -- 172.21.15.44:0/1152738489 wait complete. 2020-11-20T12:03:05.022 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:03:05.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.149+0000 7f6dbc276700 1 Processor -- start 2020-11-20T12:03:05.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.149+0000 7f6dbc276700 1 -- start start 2020-11-20T12:03:05.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.149+0000 7f6dbc276700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.149+0000 7f6dbc276700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6db40a8ea0 0x7f6db41420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.149+0000 7f6dbc276700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40a7200 con 0x7f6db40ad930 2020-11-20T12:03:05.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.149+0000 7f6dbc276700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db4135240 con 0x7f6db40a8ea0 2020-11-20T12:03:05.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.149+0000 7f6dbc276700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40a6cc0 con 0x7f6db40ad550 2020-11-20T12:03:05.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6db40a8ea0 0x7f6db41420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6db40a8ea0 0x7f6db41420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36092/0 (socket says 172.21.15.44:36092) 2020-11-20T12:03:05.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 -- 172.21.15.44:0/3398594657 learned_addr learned my addr 172.21.15.44:0/3398594657 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:05.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba012700 1 --2- 172.21.15.44:0/3398594657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 -- 172.21.15.44:0/3398594657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 msgr2=0x7f6db40a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:05.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 --2- 172.21.15.44:0/3398594657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db40a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 -- 172.21.15.44:0/3398594657 >> v1:172.21.15.79:6789/0 conn(0x7f6db40ad930 legacy=0x7f6db40a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:05.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 -- 172.21.15.44:0/3398594657 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40a6f60 con 0x7f6db40a8ea0 2020-11-20T12:03:05.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6dba813700 1 --2- 172.21.15.44:0/3398594657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db41420b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6dac009d50 tx=0x7f6dac009290).ready entity=mon.0 client_cookie=3c45a22f0e1fafb5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6db9010700 1 -- 172.21.15.44:0/3398594657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00bd10 con 0x7f6db40a8ea0 2020-11-20T12:03:05.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6db9010700 1 -- 172.21.15.44:0/3398594657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00be70 con 0x7f6db40a8ea0 2020-11-20T12:03:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.150+0000 7f6db9010700 1 -- 172.21.15.44:0/3398594657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac011610 con 0x7f6db40a8ea0 2020-11-20T12:03:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 -- 172.21.15.44:0/3398594657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 msgr2=0x7f6db41420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3398594657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db41420b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6dac009d50 tx=0x7f6dac009290).stop 2020-11-20T12:03:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 -- 172.21.15.44:0/3398594657 shutdown_connections 2020-11-20T12:03:05.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3398594657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db41420b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3398594657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db40a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 -- 172.21.15.44:0/3398594657 >> 172.21.15.44:0/3398594657 conn(0x7f6db40624b0 msgr2=0x7f6db40ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:05.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 -- 172.21.15.44:0/3398594657 shutdown_connections 2020-11-20T12:03:05.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.151+0000 7f6dbc276700 1 -- 172.21.15.44:0/3398594657 wait complete. 2020-11-20T12:03:05.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 Processor -- start 2020-11-20T12:03:05.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 -- start start 2020-11-20T12:03:05.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db409fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db413d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6db40ad930 0x7f6db4060360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db413d680 con 0x7f6db40a8ea0 2020-11-20T12:03:05.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db413d920 con 0x7f6db40ad930 2020-11-20T12:03:05.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dbc276700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4147a70 con 0x7f6db40ad550 2020-11-20T12:03:05.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dba012700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db409fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dba012700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db409fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36098/0 (socket says 172.21.15.44:36098) 2020-11-20T12:03:05.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.152+0000 7f6dba012700 1 -- 172.21.15.44:0/3844816394 learned_addr learned my addr 172.21.15.44:0/3844816394 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:05.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6db9811700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db413d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dba813700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6db40ad930 0x7f6db4060360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dba012700 1 -- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 msgr2=0x7f6db413d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dba012700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db413d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dba012700 1 -- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6db40ad930 msgr2=0x7f6db4060360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dba012700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6db40ad930 0x7f6db4060360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dba012700 1 -- 172.21.15.44:0/3844816394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db409dcb0 con 0x7f6db40a8ea0 2020-11-20T12:03:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dba012700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db409fd80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6db000db60 tx=0x7f6db0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6da27fc700 1 -- 172.21.15.44:0/3844816394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0009790 con 0x7f6db40a8ea0 2020-11-20T12:03:05.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6da27fc700 1 -- 172.21.15.44:0/3844816394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00098f0 con 0x7f6db40a8ea0 2020-11-20T12:03:05.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db409df60 con 0x7f6db40a8ea0 2020-11-20T12:03:05.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6da27fc700 1 -- 172.21.15.44:0/3844816394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0009c90 con 0x7f6db40a8ea0 2020-11-20T12:03:05.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.153+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db41480b0 con 0x7f6db40a8ea0 2020-11-20T12:03:05.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.154+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db413dae0 con 0x7f6db40a8ea0 2020-11-20T12:03:05.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.154+0000 7f6da27fc700 1 -- 172.21.15.44:0/3844816394 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f6db0013480 con 0x7f6db40a8ea0 2020-11-20T12:03:05.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.154+0000 7f6da27fc700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d8c056d50 0x7f6d8c059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.155+0000 7f6da27fc700 1 -- 172.21.15.44:0/3844816394 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6db0070680 con 0x7f6db40a8ea0 2020-11-20T12:03:05.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.155+0000 7f6db9811700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d8c056d50 0x7f6d8c059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.155+0000 7f6db9811700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d8c056d50 0x7f6d8c059210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6db4060b30 tx=0x7f6da4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.158+0000 7f6da27fc700 1 -- 172.21.15.44:0/3844816394 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6db0045910 con 0x7f6db40a8ea0 2020-11-20T12:03:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.335+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6db40615e0 con 0x7f6db40a8ea0 2020-11-20T12:03:05.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.335+0000 7f6da27fc700 1 -- 172.21.15.44:0/3844816394 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6db0048f30 con 0x7f6db40a8ea0 2020-11-20T12:03:05.339 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d8c056d50 msgr2=0x7f6d8c059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d8c056d50 0x7f6d8c059210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6db4060b30 tx=0x7f6da4009040).stop 2020-11-20T12:03:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 msgr2=0x7f6db409fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db409fd80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6db000db60 tx=0x7f6db0008040).stop 2020-11-20T12:03:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 shutdown_connections 2020-11-20T12:03:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d8c056d50 0x7f6d8c059210 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6db40ad930 0x7f6db4060360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db40a8ea0 0x7f6db409fd80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 --2- 172.21.15.44:0/3844816394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6db40ad550 0x7f6db413d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 >> 172.21.15.44:0/3844816394 conn(0x7f6db40624b0 msgr2=0x7f6db40aa280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 shutdown_connections 2020-11-20T12:03:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.337+0000 7f6dbc276700 1 -- 172.21.15.44:0/3844816394 wait complete. 2020-11-20T12:03:05.352 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:03:05.360 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:03:05.396 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:03:05.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f177a9cc700 1 Processor -- start 2020-11-20T12:03:05.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f177a9cc700 1 -- start start 2020-11-20T12:03:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f177a9cc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f17740ad550 0x7f17740a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f177a9cc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17741420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f177a9cc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17740a7200 con 0x7f17740ad930 2020-11-20T12:03:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f177a9cc700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1774135240 con 0x7f17740ad550 2020-11-20T12:03:05.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f177a9cc700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17740a6cc0 con 0x7f17740a8ea0 2020-11-20T12:03:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f1773fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f17740ad550 0x7f17740a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f1773fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f17740ad550 0x7f17740a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36106/0 (socket says 172.21.15.44:36106) 2020-11-20T12:03:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.479+0000 7f1773fff700 1 -- 172.21.15.44:0/3304676577 learned_addr learned my addr 172.21.15.44:0/3304676577 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:05.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1778f69700 1 --2- 172.21.15.44:0/3304676577 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17741420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1773fff700 1 -- 172.21.15.44:0/3304676577 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 msgr2=0x7f17741420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1773fff700 1 --2- 172.21.15.44:0/3304676577 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17741420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1773fff700 1 -- 172.21.15.44:0/3304676577 >> v1:172.21.15.79:6789/0 conn(0x7f17740ad930 legacy=0x7f17740a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1773fff700 1 -- 172.21.15.44:0/3304676577 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740a6f60 con 0x7f17740ad550 2020-11-20T12:03:05.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1773fff700 1 --2- 172.21.15.44:0/3304676577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad550 0x7f17740a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1764006fd0 tx=0x7f1764006eb0).ready entity=mon.0 client_cookie=537de4848e55380a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1772ffd700 1 -- 172.21.15.44:0/3304676577 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764009810 con 0x7f17740ad550 2020-11-20T12:03:05.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1772ffd700 1 -- 172.21.15.44:0/3304676577 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764009970 con 0x7f17740ad550 2020-11-20T12:03:05.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.480+0000 7f1772ffd700 1 -- 172.21.15.44:0/3304676577 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764009cf0 con 0x7f17740ad550 2020-11-20T12:03:05.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 -- 172.21.15.44:0/3304676577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad550 msgr2=0x7f17740a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3304676577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad550 0x7f17740a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1764006fd0 tx=0x7f1764006eb0).stop 2020-11-20T12:03:05.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 -- 172.21.15.44:0/3304676577 shutdown_connections 2020-11-20T12:03:05.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3304676577 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17741420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3304676577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad550 0x7f17740a5070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 -- 172.21.15.44:0/3304676577 >> 172.21.15.44:0/3304676577 conn(0x7f17740624b0 msgr2=0x7f17740ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:05.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 -- 172.21.15.44:0/3304676577 shutdown_connections 2020-11-20T12:03:05.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.481+0000 7f177a9cc700 1 -- 172.21.15.44:0/3304676577 wait complete. 2020-11-20T12:03:05.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 Processor -- start 2020-11-20T12:03:05.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 -- start start 2020-11-20T12:03:05.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17740a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17740ad550 0x7f1774060330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad930 0x7f177409d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17740a3e30 con 0x7f17740ad930 2020-11-20T12:03:05.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177413db90 con 0x7f17740ad550 2020-11-20T12:03:05.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f177a9cc700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17740a4080 con 0x7f17740a8ea0 2020-11-20T12:03:05.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f1778f69700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad930 0x7f177409d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f1778f69700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad930 0x7f177409d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36116/0 (socket says 172.21.15.44:36116) 2020-11-20T12:03:05.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f1778f69700 1 -- 172.21.15.44:0/3108801630 learned_addr learned my addr 172.21.15.44:0/3108801630 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:05.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f1773fff700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17740a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.482+0000 7f17737fe700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17740ad550 0x7f1774060330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1778f69700 1 -- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 msgr2=0x7f17740a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1778f69700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17740a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1778f69700 1 -- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17740ad550 msgr2=0x7f1774060330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1778f69700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17740ad550 0x7f1774060330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1778f69700 1 -- 172.21.15.44:0/3108801630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740a4320 con 0x7f17740ad930 2020-11-20T12:03:05.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1778f69700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad930 0x7f177409d470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f176c00ef50 tx=0x7f176c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1753fff700 1 -- 172.21.15.44:0/3108801630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c00cb40 con 0x7f17740ad930 2020-11-20T12:03:05.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1753fff700 1 -- 172.21.15.44:0/3108801630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c00cca0 con 0x7f17740ad930 2020-11-20T12:03:05.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17740a0190 con 0x7f17740ad930 2020-11-20T12:03:05.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f1753fff700 1 -- 172.21.15.44:0/3108801630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c012620 con 0x7f17740ad930 2020-11-20T12:03:05.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.483+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177409e340 con 0x7f17740ad930 2020-11-20T12:03:05.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.484+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17740a03b0 con 0x7f17740ad930 2020-11-20T12:03:05.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.484+0000 7f1753fff700 1 -- 172.21.15.44:0/3108801630 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f176c022070 con 0x7f17740ad930 2020-11-20T12:03:05.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.484+0000 7f1753fff700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f174c056d20 0x7f174c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.485+0000 7f1753fff700 1 -- 172.21.15.44:0/3108801630 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f176c0708e0 con 0x7f17740ad930 2020-11-20T12:03:05.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.485+0000 7f1773fff700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f174c056d20 0x7f174c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.485+0000 7f1773fff700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f174c056d20 0x7f174c0591e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f176400b1b0 tx=0x7f1764015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.488+0000 7f1753fff700 1 -- 172.21.15.44:0/3108801630 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f176c045bd0 con 0x7f17740ad930 2020-11-20T12:03:05.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.662+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f177409e5c0 con 0x7f17740ad930 2020-11-20T12:03:05.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.662+0000 7f1753fff700 1 -- 172.21.15.44:0/3108801630 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f176c016020 con 0x7f17740ad930 2020-11-20T12:03:05.666 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f174c056d20 msgr2=0x7f174c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f174c056d20 0x7f174c0591e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f176400b1b0 tx=0x7f1764015000).stop 2020-11-20T12:03:05.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad930 msgr2=0x7f177409d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad930 0x7f177409d470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f176c00ef50 tx=0x7f176c0091d0).stop 2020-11-20T12:03:05.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 shutdown_connections 2020-11-20T12:03:05.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f174c056d20 0x7f174c0591e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17740ad930 0x7f177409d470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17740a8ea0 0x7f17740a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 --2- 172.21.15.44:0/3108801630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17740ad550 0x7f1774060330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.663+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 >> 172.21.15.44:0/3108801630 conn(0x7f17740624b0 msgr2=0x7f1774137ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:05.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.664+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 shutdown_connections 2020-11-20T12:03:05.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.664+0000 7f177a9cc700 1 -- 172.21.15.44:0/3108801630 wait complete. 2020-11-20T12:03:05.679 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:03:05.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe3e108700 1 Processor -- start 2020-11-20T12:03:05.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe3e108700 1 -- start start 2020-11-20T12:03:05.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe3e108700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe3813c580 0x7efe3814a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe3e108700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380a57f0 con 0x7efe380a3ad0 2020-11-20T12:03:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe3e108700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380a5a90 con 0x7efe38142ae0 2020-11-20T12:03:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe3e108700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe380a5d30 con 0x7efe3813c580 2020-11-20T12:03:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe377fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7efe380a3ad0 0x7efe381423d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39868/0 (socket says 172.21.15.44:39868) 2020-11-20T12:03:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.846+0000 7efe377fe700 1 -- 172.21.15.44:0/1085989926 learned_addr learned my addr 172.21.15.44:0/1085989926 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe37fff700 1 --2- 172.21.15.44:0/1085989926 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe3813c580 0x7efe3814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe36ffd700 1 -- 172.21.15.44:0/1085989926 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203823560 0 0) 0x7efe380a57f0 con 0x7efe380a3ad0 2020-11-20T12:03:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe36ffd700 1 -- 172.21.15.44:0/1085989926 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe18003240 con 0x7efe380a3ad0 2020-11-20T12:03:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe36ffd700 1 -- 172.21.15.44:0/1085989926 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151097912 0 0) 0x7efe380a5a90 con 0x7efe38142ae0 2020-11-20T12:03:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe36ffd700 1 -- 172.21.15.44:0/1085989926 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe380a57f0 con 0x7efe38142ae0 2020-11-20T12:03:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe37fff700 1 -- 172.21.15.44:0/1085989926 >> v1:172.21.15.79:6789/0 conn(0x7efe38142ae0 legacy=0x7efe3813be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe37fff700 1 -- 172.21.15.44:0/1085989926 >> v1:172.21.15.44:6789/0 conn(0x7efe380a3ad0 legacy=0x7efe381423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe37fff700 1 -- 172.21.15.44:0/1085989926 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380a3eb0 con 0x7efe3813c580 2020-11-20T12:03:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe37fff700 1 --2- 172.21.15.44:0/1085989926 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe3813c580 0x7efe3814a780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7efe30009cc0 tx=0x7efe300091d0).ready entity=mon.2 client_cookie=c2e72fb0a43e72ee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe36ffd700 1 -- 172.21.15.44:0/1085989926 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3000ba20 con 0x7efe3813c580 2020-11-20T12:03:05.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe36ffd700 1 -- 172.21.15.44:0/1085989926 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3000bb80 con 0x7efe3813c580 2020-11-20T12:03:05.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.847+0000 7efe36ffd700 1 -- 172.21.15.44:0/1085989926 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30011400 con 0x7efe3813c580 2020-11-20T12:03:05.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.848+0000 7efe3e108700 1 -- 172.21.15.44:0/1085989926 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe3813c580 msgr2=0x7efe3814a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.848+0000 7efe3e108700 1 --2- 172.21.15.44:0/1085989926 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe3813c580 0x7efe3814a780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7efe30009cc0 tx=0x7efe300091d0).stop 2020-11-20T12:03:05.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.848+0000 7efe3e108700 1 -- 172.21.15.44:0/1085989926 shutdown_connections 2020-11-20T12:03:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.848+0000 7efe3e108700 1 --2- 172.21.15.44:0/1085989926 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe3813c580 0x7efe3814a780 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.848+0000 7efe3e108700 1 -- 172.21.15.44:0/1085989926 >> 172.21.15.44:0/1085989926 conn(0x7efe380624b0 msgr2=0x7efe380ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.848+0000 7efe3e108700 1 -- 172.21.15.44:0/1085989926 shutdown_connections 2020-11-20T12:03:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.848+0000 7efe3e108700 1 -- 172.21.15.44:0/1085989926 wait complete. 2020-11-20T12:03:05.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 Processor -- start 2020-11-20T12:03:05.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 -- start start 2020-11-20T12:03:05.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe380a3ad0 0x7efe38144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe3813c580 0x7efe380ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38142ae0 0x7efe38151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380ac930 con 0x7efe38142ae0 2020-11-20T12:03:05.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3814e6c0 con 0x7efe3813c580 2020-11-20T12:03:05.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe3e108700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3814e960 con 0x7efe380a3ad0 2020-11-20T12:03:05.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe37fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38142ae0 0x7efe38151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe37fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38142ae0 0x7efe38151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36128/0 (socket says 172.21.15.44:36128) 2020-11-20T12:03:05.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.849+0000 7efe37fff700 1 -- 172.21.15.44:0/911777852 learned_addr learned my addr 172.21.15.44:0/911777852 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:05.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe377fe700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe380a3ad0 0x7efe38144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe2ffff700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe3813c580 0x7efe380ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe37fff700 1 -- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe380a3ad0 msgr2=0x7efe38144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe37fff700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe380a3ad0 0x7efe38144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe37fff700 1 -- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe3813c580 msgr2=0x7efe380ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:05.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe37fff700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe3813c580 0x7efe380ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:05.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe37fff700 1 -- 172.21.15.44:0/911777852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3814ec00 con 0x7efe38142ae0 2020-11-20T12:03:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe37fff700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38142ae0 0x7efe38151820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7efe38062980 tx=0x7efe30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe2f7fe700 1 -- 172.21.15.44:0/911777852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3001c5f0 con 0x7efe38142ae0 2020-11-20T12:03:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe2f7fe700 1 -- 172.21.15.44:0/911777852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3001c750 con 0x7efe38142ae0 2020-11-20T12:03:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe2f7fe700 1 -- 172.21.15.44:0/911777852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3001cad0 con 0x7efe38142ae0 2020-11-20T12:03:05.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3814f0a0 con 0x7efe38142ae0 2020-11-20T12:03:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.850+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3814e2d0 con 0x7efe38142ae0 2020-11-20T12:03:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.851+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3814e4f0 con 0x7efe38142ae0 2020-11-20T12:03:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.851+0000 7efe2f7fe700 1 -- 172.21.15.44:0/911777852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7efe30024030 con 0x7efe38142ae0 2020-11-20T12:03:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.851+0000 7efe2f7fe700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efe14056d50 0x7efe14059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:05.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.852+0000 7efe2f7fe700 1 -- 172.21.15.44:0/911777852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efe30071af0 con 0x7efe38142ae0 2020-11-20T12:03:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.852+0000 7efe377fe700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efe14056d50 0x7efe14059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.852+0000 7efe377fe700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efe14056d50 0x7efe14059210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7efe280055a0 tx=0x7efe28009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:05.855+0000 7efe2f7fe700 1 -- 172.21.15.44:0/911777852 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efe30046f10 con 0x7efe38142ae0 2020-11-20T12:03:06.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.031+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe380615e0 con 0x7efe38142ae0 2020-11-20T12:03:06.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.031+0000 7efe2f7fe700 1 -- 172.21.15.44:0/911777852 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7efe30022020 con 0x7efe38142ae0 2020-11-20T12:03:06.035 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:06.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efe14056d50 msgr2=0x7efe14059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efe14056d50 0x7efe14059210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7efe280055a0 tx=0x7efe28009040).stop 2020-11-20T12:03:06.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38142ae0 msgr2=0x7efe38151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38142ae0 0x7efe38151820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7efe38062980 tx=0x7efe30016000).stop 2020-11-20T12:03:06.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 shutdown_connections 2020-11-20T12:03:06.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efe14056d50 0x7efe14059210 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe38142ae0 0x7efe38151820 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efe380a3ad0 0x7efe38144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 --2- 172.21.15.44:0/911777852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe3813c580 0x7efe380ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.032+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 >> 172.21.15.44:0/911777852 conn(0x7efe380624b0 msgr2=0x7efe38060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:06.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.033+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 shutdown_connections 2020-11-20T12:03:06.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.033+0000 7efe3e108700 1 -- 172.21.15.44:0/911777852 wait complete. 2020-11-20T12:03:06.047 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-20T12:03:06.048 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-20T12:03:06.048 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2020-11-20T12:03:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.170+0000 7f7efdb4e700 1 Processor -- start 2020-11-20T12:03:06.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.170+0000 7f7efdb4e700 1 -- start start 2020-11-20T12:03:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7efdb4e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 0x7f7ef813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7efdb4e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7ef813f800 0x7f7ef8141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7efdb4e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef80a6260 con 0x7f7ef8060330 2020-11-20T12:03:06.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7efdb4e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef80a52e0 con 0x7f7ef813f800 2020-11-20T12:03:06.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7efdb4e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80a6490 con 0x7f7ef813a990 2020-11-20T12:03:06.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7ef77fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f7ef8060330 0x7f7ef80a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39880/0 (socket says 172.21.15.44:39880) 2020-11-20T12:03:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7ef77fe700 1 -- 172.21.15.44:0/1651511466 learned_addr learned my addr 172.21.15.44:0/1651511466 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7ef7fff700 1 --2- 172.21.15.44:0/1651511466 >> v2:172.21.15.79:3300/0 conn(0x7f7ef813f800 0x7f7ef8141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7ef6ffd700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 0x7f7ef813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373488921 0 0) 0x7f7ef80a6260 con 0x7f7ef8060330 2020-11-20T12:03:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee4003240 con 0x7f7ef8060330 2020-11-20T12:03:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.171+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 544417417 0 0) 0x7f7ee4003240 con 0x7f7ef8060330 2020-11-20T12:03:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 msgr2=0x7f7ef813adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 0x7f7ef813adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 >> v2:172.21.15.79:3300/0 conn(0x7f7ef813f800 msgr2=0x7f7ef8141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 --2- 172.21.15.44:0/1651511466 >> v2:172.21.15.79:3300/0 conn(0x7f7ef813f800 0x7f7ef8141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80a6730 con 0x7f7ef8060330 2020-11-20T12:03:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f7ee80014e0 con 0x7f7ef8060330 2020-11-20T12:03:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 >> v1:172.21.15.44:6789/0 conn(0x7f7ef8060330 legacy=0x7f7ef80a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 0x7f7ee4003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee4003c40 0x7f7ee40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ee40049d0 0x7f7ee4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80a52e0 con 0x7f7ee4003c40 2020-11-20T12:03:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80a6490 con 0x7f7ee40049d0 2020-11-20T12:03:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee4004200 con 0x7f7ef813a990 2020-11-20T12:03:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef6ffd700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee4003c40 0x7f7ee40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef77fe700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 0x7f7ee4003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef7fff700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ee40049d0 0x7f7ee4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef6ffd700 1 -- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 msgr2=0x7f7ee4003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:06.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef6ffd700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ef813a990 0x7f7ee4003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef6ffd700 1 -- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ee40049d0 msgr2=0x7f7ee4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:06.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef6ffd700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ee40049d0 0x7f7ee4004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef6ffd700 1 -- 172.21.15.44:0/1651511466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4003240 con 0x7f7ee4003c40 2020-11-20T12:03:06.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef6ffd700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee4003c40 0x7f7ee40044e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f7eec00bc00 tx=0x7f7eec006df0).ready entity=mon.0 client_cookie=6ac59576f89acf2a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00b310 con 0x7f7ee4003c40 2020-11-20T12:03:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec00e9d0 con 0x7f7ee4003c40 2020-11-20T12:03:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.172+0000 7f7ef67fc700 1 -- 172.21.15.44:0/1651511466 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00ed50 con 0x7f7ee4003c40 2020-11-20T12:03:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.173+0000 7f7efdb4e700 1 -- 172.21.15.44:0/1651511466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee4003c40 msgr2=0x7f7ee40044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.173+0000 7f7efdb4e700 1 --2- 172.21.15.44:0/1651511466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee4003c40 0x7f7ee40044e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f7eec00bc00 tx=0x7f7eec006df0).stop 2020-11-20T12:03:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.173+0000 7f7ef7fff700 1 -- 172.21.15.44:0/1651511466 reap_dead start 2020-11-20T12:03:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.173+0000 7f7efdb4e700 1 -- 172.21.15.44:0/1651511466 shutdown_connections 2020-11-20T12:03:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.173+0000 7f7efdb4e700 1 -- 172.21.15.44:0/1651511466 >> 172.21.15.44:0/1651511466 conn(0x7f7ef8062690 msgr2=0x7f7ef80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.173+0000 7f7efdb4e700 1 -- 172.21.15.44:0/1651511466 shutdown_connections 2020-11-20T12:03:06.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.173+0000 7f7efdb4e700 1 -- 172.21.15.44:0/1651511466 wait complete. 2020-11-20T12:03:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 Processor -- start 2020-11-20T12:03:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 -- start start 2020-11-20T12:03:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ee4003c40 0x7f7ef8135730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ef813e820 0x7f7ef80ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ef8130040 0x7f7ef81304a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8134250 con 0x7f7ef8130040 2020-11-20T12:03:06.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8134ab0 con 0x7f7ef813e820 2020-11-20T12:03:06.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.174+0000 7f7efdb4e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8134d50 con 0x7f7ee4003c40 2020-11-20T12:03:06.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ef8130040 0x7f7ef81304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ef8130040 0x7f7ef81304a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36146/0 (socket says 172.21.15.44:36146) 2020-11-20T12:03:06.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef77fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ee4003c40 0x7f7ef8135730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 -- 172.21.15.44:0/2285058154 learned_addr learned my addr 172.21.15.44:0/2285058154 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:06.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef6ffd700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ef813e820 0x7f7ef80ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 -- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ee4003c40 msgr2=0x7f7ef8135730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ee4003c40 0x7f7ef8135730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 -- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ef813e820 msgr2=0x7f7ef80ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ef813e820 0x7f7ef80ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 -- 172.21.15.44:0/2285058154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef813e380 con 0x7f7ef8130040 2020-11-20T12:03:06.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ef7fff700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ef8130040 0x7f7ef81304a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f7ef00090e0 tx=0x7f7ef00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ed77fe700 1 -- 172.21.15.44:0/2285058154 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000faa0 con 0x7f7ef8130040 2020-11-20T12:03:06.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ed77fe700 1 -- 172.21.15.44:0/2285058154 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef000fc00 con 0x7f7ef8130040 2020-11-20T12:03:06.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef8130990 con 0x7f7ef8130040 2020-11-20T12:03:06.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7ed77fe700 1 -- 172.21.15.44:0/2285058154 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0014440 con 0x7f7ef8130040 2020-11-20T12:03:06.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.175+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef8137ba0 con 0x7f7ef8130040 2020-11-20T12:03:06.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.176+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef813ec00 con 0x7f7ef8130040 2020-11-20T12:03:06.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.176+0000 7f7ed77fe700 1 -- 172.21.15.44:0/2285058154 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7ef0014830 con 0x7f7ef8130040 2020-11-20T12:03:06.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.177+0000 7f7ed77fe700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ed0056d20 0x7f7ed00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.177+0000 7f7ed77fe700 1 -- 172.21.15.44:0/2285058154 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7ef0070bf0 con 0x7f7ef8130040 2020-11-20T12:03:06.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.177+0000 7f7ef77fe700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ed0056d20 0x7f7ed00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.177+0000 7f7ef77fe700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ed0056d20 0x7f7ed00591e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7ee8004560 tx=0x7f7ef80a68c0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.180+0000 7f7ed77fe700 1 -- 172.21.15.44:0/2285058154 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7ef0045ee0 con 0x7f7ef8130040 2020-11-20T12:03:06.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.393+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7ef8061650 con 0x7f7ef8130040 2020-11-20T12:03:06.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.395+0000 7f7ed77fe700 1 -- 172.21.15.44:0/2285058154 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+135023 (secure 0 0 0) 0x7f7ef0049500 con 0x7f7ef8130040 2020-11-20T12:03:06.401 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:06.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ed0056d20 msgr2=0x7f7ed00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ed0056d20 0x7f7ed00591e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7ee8004560 tx=0x7f7ef80a68c0).stop 2020-11-20T12:03:06.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ef8130040 msgr2=0x7f7ef81304a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ef8130040 0x7f7ef81304a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f7ef00090e0 tx=0x7f7ef00091d0).stop 2020-11-20T12:03:06.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 shutdown_connections 2020-11-20T12:03:06.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ed0056d20 0x7f7ed00591e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ef8130040 0x7f7ef81304a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ee4003c40 0x7f7ef8135730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 --2- 172.21.15.44:0/2285058154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ef813e820 0x7f7ef80ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 >> 172.21.15.44:0/2285058154 conn(0x7f7ef8062690 msgr2=0x7f7ef80608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:06.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 shutdown_connections 2020-11-20T12:03:06.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.398+0000 7f7efdb4e700 1 -- 172.21.15.44:0/2285058154 wait complete. 2020-11-20T12:03:06.412 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":24383},{"type":"v1","addr":"172.21.15.79:6833","nonce":24383}]},"active_addr":"172.21.15.79:6833/24383","active_change":"2020-11-20T12:02:44.565275+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":1971875058}]},{"addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":2698868411}]},{"addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":4283710902}]}]}} 2020-11-20T12:03:06.416 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-20T12:03:06.417 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-20T12:03:06.417 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:06.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.540+0000 7fbea68d5700 1 Processor -- start 2020-11-20T12:03:06.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbea68d5700 1 -- start start 2020-11-20T12:03:06.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbea68d5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea00aa400 0x7fbea00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbea68d5700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea00a9770 con 0x7fbea013ac60 2020-11-20T12:03:06.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbea68d5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea00a6da0 con 0x7fbea00aa020 2020-11-20T12:03:06.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbea68d5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00a56c0 con 0x7fbea00aa400 2020-11-20T12:03:06.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbea4e72700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbea013ac60 0x7fbea00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39902/0 (socket says 172.21.15.44:39902) 2020-11-20T12:03:06.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbea4e72700 1 -- 172.21.15.44:0/252734268 learned_addr learned my addr 172.21.15.44:0/252734268 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:06.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.541+0000 7fbe97fff700 1 --2- 172.21.15.44:0/252734268 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea00aa400 0x7fbea00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe9f7fe700 1 -- 172.21.15.44:0/252734268 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809562884 0 0) 0x7fbe98001450 con 0x7fbea013ac60 2020-11-20T12:03:06.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe9f7fe700 1 -- 172.21.15.44:0/252734268 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe80003240 con 0x7fbea013ac60 2020-11-20T12:03:06.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe9f7fe700 1 -- 172.21.15.44:0/252734268 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084079303 0 0) 0x7fbea00a6da0 con 0x7fbea00aa020 2020-11-20T12:03:06.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe9f7fe700 1 -- 172.21.15.44:0/252734268 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe98001450 con 0x7fbea00aa020 2020-11-20T12:03:06.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe97fff700 1 -- 172.21.15.44:0/252734268 >> v1:172.21.15.79:6789/0 conn(0x7fbea00aa020 legacy=0x7fbea00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe97fff700 1 -- 172.21.15.44:0/252734268 >> v1:172.21.15.44:6789/0 conn(0x7fbea013ac60 legacy=0x7fbea00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe97fff700 1 -- 172.21.15.44:0/252734268 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00a7040 con 0x7fbea00aa400 2020-11-20T12:03:06.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe97fff700 1 --2- 172.21.15.44:0/252734268 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea00aa400 0x7fbea00a9280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbe8800b9d0 tx=0x7fbe88006df0).ready entity=mon.2 client_cookie=d1c4f72eed6fe83f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe9f7fe700 1 -- 172.21.15.44:0/252734268 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88009680 con 0x7fbea00aa400 2020-11-20T12:03:06.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe9f7fe700 1 -- 172.21.15.44:0/252734268 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe880097e0 con 0x7fbea00aa400 2020-11-20T12:03:06.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.542+0000 7fbe9f7fe700 1 -- 172.21.15.44:0/252734268 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88009b60 con 0x7fbea00aa400 2020-11-20T12:03:06.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.543+0000 7fbea68d5700 1 -- 172.21.15.44:0/252734268 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea00aa400 msgr2=0x7fbea00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.543+0000 7fbea68d5700 1 --2- 172.21.15.44:0/252734268 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea00aa400 0x7fbea00a9280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbe8800b9d0 tx=0x7fbe88006df0).stop 2020-11-20T12:03:06.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.543+0000 7fbea68d5700 1 -- 172.21.15.44:0/252734268 shutdown_connections 2020-11-20T12:03:06.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.543+0000 7fbea68d5700 1 --2- 172.21.15.44:0/252734268 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea00aa400 0x7fbea00a9280 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.543+0000 7fbea68d5700 1 -- 172.21.15.44:0/252734268 >> 172.21.15.44:0/252734268 conn(0x7fbea0062690 msgr2=0x7fbea00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:06.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.543+0000 7fbea68d5700 1 -- 172.21.15.44:0/252734268 shutdown_connections 2020-11-20T12:03:06.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.543+0000 7fbea68d5700 1 -- 172.21.15.44:0/252734268 wait complete. 2020-11-20T12:03:06.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 Processor -- start 2020-11-20T12:03:06.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 -- start start 2020-11-20T12:03:06.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbea00aa020 0x7fbea00ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea00aa400 0x7fbea00aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea013ac60 0x7fbea00678f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0145940 con 0x7fbea00aa400 2020-11-20T12:03:06.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00abba0 con 0x7fbea00aa020 2020-11-20T12:03:06.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea68d5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0067500 con 0x7fbea013ac60 2020-11-20T12:03:06.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe97fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea00aa400 0x7fbea00aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe97fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea00aa400 0x7fbea00aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36158/0 (socket says 172.21.15.44:36158) 2020-11-20T12:03:06.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe97fff700 1 -- 172.21.15.44:0/1053674088 learned_addr learned my addr 172.21.15.44:0/1053674088 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:06.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbea4e72700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea013ac60 0x7fbea00678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe9ffff700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbea00aa020 0x7fbea00ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe97fff700 1 -- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea013ac60 msgr2=0x7fbea00678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe97fff700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea013ac60 0x7fbea00678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe97fff700 1 -- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbea00aa020 msgr2=0x7fbea00ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.544+0000 7fbe97fff700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbea00aa020 0x7fbea00ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbe97fff700 1 -- 172.21.15.44:0/1053674088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00ab3d0 con 0x7fbea00aa400 2020-11-20T12:03:06.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbe9ffff700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbea00aa020 0x7fbea00ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:06.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbe97fff700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea00aa400 0x7fbea00aaee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fbe88006e30 tx=0x7fbe88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbe9cff9700 1 -- 172.21.15.44:0/1053674088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88011e20 con 0x7fbea00aa400 2020-11-20T12:03:06.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbe9cff9700 1 -- 172.21.15.44:0/1053674088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8801e460 con 0x7fbea00aa400 2020-11-20T12:03:06.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea00ab660 con 0x7fbea00aa400 2020-11-20T12:03:06.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbe9cff9700 1 -- 172.21.15.44:0/1053674088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8801e7e0 con 0x7fbea00aa400 2020-11-20T12:03:06.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.545+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea0066940 con 0x7fbea00aa400 2020-11-20T12:03:06.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.546+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea0061650 con 0x7fbea00aa400 2020-11-20T12:03:06.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.546+0000 7fbe9cff9700 1 -- 172.21.15.44:0/1053674088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbe88025030 con 0x7fbea00aa400 2020-11-20T12:03:06.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.546+0000 7fbe9cff9700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe7c056d20 0x7fbe7c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.546+0000 7fbe9cff9700 1 -- 172.21.15.44:0/1053674088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbe88071810 con 0x7fbea00aa400 2020-11-20T12:03:06.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.546+0000 7fbe9ffff700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe7c056d20 0x7fbe7c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.547+0000 7fbe9ffff700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe7c056d20 0x7fbe7c0591e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbe90000bc0 tx=0x7fbe90009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.550+0000 7fbe9cff9700 1 -- 172.21.15.44:0/1053674088 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbe88046ae0 con 0x7fbea00aa400 2020-11-20T12:03:06.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.725+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbea00a6a90 con 0x7fbea00aa400 2020-11-20T12:03:06.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.726+0000 7fbe9cff9700 1 -- 172.21.15.44:0/1053674088 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7fbe88023020 con 0x7fbea00aa400 2020-11-20T12:03:06.730 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:06.731 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:06.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe7c056d20 msgr2=0x7fbe7c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe7c056d20 0x7fbe7c0591e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbe90000bc0 tx=0x7fbe90009040).stop 2020-11-20T12:03:06.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea00aa400 msgr2=0x7fbea00aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea00aa400 0x7fbea00aaee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fbe88006e30 tx=0x7fbe88015000).stop 2020-11-20T12:03:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 shutdown_connections 2020-11-20T12:03:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe7c056d20 0x7fbe7c0591e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbea013ac60 0x7fbea00678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbea00aa020 0x7fbea00ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 --2- 172.21.15.44:0/1053674088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbea00aa400 0x7fbea00aaee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 >> 172.21.15.44:0/1053674088 conn(0x7fbea0062690 msgr2=0x7fbea0139be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 shutdown_connections 2020-11-20T12:03:06.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.728+0000 7fbea68d5700 1 -- 172.21.15.44:0/1053674088 wait complete. 2020-11-20T12:03:06.743 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-20T12:03:06.743 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:06.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.864+0000 7f075f44b700 1 Processor -- start 2020-11-20T12:03:06.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.865+0000 7f075f44b700 1 -- start start 2020-11-20T12:03:06.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.865+0000 7f075f44b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f07580aa020 0x7f07580a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.865+0000 7f075f44b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f075813ac60 0x7f07580a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.865+0000 7f075f44b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07580a9770 con 0x7f07580aa400 2020-11-20T12:03:06.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075f44b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07580a56c0 con 0x7f07580aa020 2020-11-20T12:03:06.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075f44b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07580a6da0 con 0x7f075813ac60 2020-11-20T12:03:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f07580aa020 0x7f07580a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f07580aa020 0x7f07580a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36164/0 (socket says 172.21.15.44:36164) 2020-11-20T12:03:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 -- 172.21.15.44:0/4028329897 learned_addr learned my addr 172.21.15.44:0/4028329897 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:06.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d9e8700 1 --2- 172.21.15.44:0/4028329897 >> v2:172.21.15.79:3300/0 conn(0x7f075813ac60 0x7f07580a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 -- 172.21.15.44:0/4028329897 >> v2:172.21.15.79:3300/0 conn(0x7f075813ac60 msgr2=0x7f07580a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:06.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 --2- 172.21.15.44:0/4028329897 >> v2:172.21.15.79:3300/0 conn(0x7f075813ac60 0x7f07580a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 -- 172.21.15.44:0/4028329897 >> v1:172.21.15.79:6789/0 conn(0x7f07580aa400 legacy=0x7f07580a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 -- 172.21.15.44:0/4028329897 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580a7040 con 0x7f07580aa020 2020-11-20T12:03:06.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d9e8700 1 --2- 172.21.15.44:0/4028329897 >> v2:172.21.15.79:3300/0 conn(0x7f075813ac60 0x7f07580a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:06.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f075d1e7700 1 --2- 172.21.15.44:0/4028329897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa020 0x7f07580a9280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f07580a8f70 tx=0x7f074c006eb0).ready entity=mon.0 client_cookie=a5b15feda95efa4d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f074bfff700 1 -- 172.21.15.44:0/4028329897 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0098d0 con 0x7f07580aa020 2020-11-20T12:03:06.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f074bfff700 1 -- 172.21.15.44:0/4028329897 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c009a30 con 0x7f07580aa020 2020-11-20T12:03:06.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.866+0000 7f074bfff700 1 -- 172.21.15.44:0/4028329897 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c009db0 con 0x7f07580aa020 2020-11-20T12:03:06.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 -- 172.21.15.44:0/4028329897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa020 msgr2=0x7f07580a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 --2- 172.21.15.44:0/4028329897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa020 0x7f07580a9280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f07580a8f70 tx=0x7f074c006eb0).stop 2020-11-20T12:03:06.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 -- 172.21.15.44:0/4028329897 shutdown_connections 2020-11-20T12:03:06.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 --2- 172.21.15.44:0/4028329897 >> v2:172.21.15.79:3300/0 conn(0x7f075813ac60 0x7f07580a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 --2- 172.21.15.44:0/4028329897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa020 0x7f07580a9280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 -- 172.21.15.44:0/4028329897 >> 172.21.15.44:0/4028329897 conn(0x7f0758062690 msgr2=0x7f07580a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:06.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 -- 172.21.15.44:0/4028329897 shutdown_connections 2020-11-20T12:03:06.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.867+0000 7f075f44b700 1 -- 172.21.15.44:0/4028329897 wait complete. 2020-11-20T12:03:06.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 Processor -- start 2020-11-20T12:03:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 -- start start 2020-11-20T12:03:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07580aa020 0x7f0758067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa400 0x7f07580ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075813ac60 0x7f0758131a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07580669a0 con 0x7f07580aa400 2020-11-20T12:03:06.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07580ab110 con 0x7f075813ac60 2020-11-20T12:03:06.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.868+0000 7f075f44b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07580ab3b0 con 0x7f07580aa020 2020-11-20T12:03:06.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa400 0x7f07580ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa400 0x7f07580ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36172/0 (socket says 172.21.15.44:36172) 2020-11-20T12:03:06.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 -- 172.21.15.44:0/1424439111 learned_addr learned my addr 172.21.15.44:0/1424439111 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:06.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075d1e7700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07580aa020 0x7f0758067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075d9e8700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075813ac60 0x7f0758131a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 -- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07580aa020 msgr2=0x7f0758067560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07580aa020 0x7f0758067560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 -- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075813ac60 msgr2=0x7f0758131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:06.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075813ac60 0x7f0758131a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:06.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 -- 172.21.15.44:0/1424439111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580add50 con 0x7f07580aa400 2020-11-20T12:03:06.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075c9e6700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa400 0x7f07580ad0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0754001e70 tx=0x7f0754007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f07497fa700 1 -- 172.21.15.44:0/1424439111 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754007020 con 0x7f07580aa400 2020-11-20T12:03:06.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f07497fa700 1 -- 172.21.15.44:0/1424439111 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0754007180 con 0x7f07580aa400 2020-11-20T12:03:06.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0758067a50 con 0x7f07580aa400 2020-11-20T12:03:06.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f07497fa700 1 -- 172.21.15.44:0/1424439111 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075400e560 con 0x7f07580aa400 2020-11-20T12:03:06.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.869+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0758066580 con 0x7f07580aa400 2020-11-20T12:03:06.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.870+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0758135d40 con 0x7f07580aa400 2020-11-20T12:03:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.871+0000 7f07497fa700 1 -- 172.21.15.44:0/1424439111 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0754010070 con 0x7f07580aa400 2020-11-20T12:03:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.871+0000 7f07497fa700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0730056d20 0x7f07300591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.871+0000 7f07497fa700 1 -- 172.21.15.44:0/1424439111 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f075406fa90 con 0x7f07580aa400 2020-11-20T12:03:06.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.871+0000 7f075d1e7700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0730056d20 0x7f07300591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:06.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.872+0000 7f075d1e7700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0730056d20 0x7f07300591e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f074c006ef0 tx=0x7f074c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:06.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:06.875+0000 7f07497fa700 1 -- 172.21.15.44:0/1424439111 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0754044e00 con 0x7f07580aa400 2020-11-20T12:03:07.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.048+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07580a9770 con 0x7f07580aa400 2020-11-20T12:03:07.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.049+0000 7f07497fa700 1 -- 172.21.15.44:0/1424439111 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7f07580a9770 con 0x7f07580aa400 2020-11-20T12:03:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:07.053 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:07.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0730056d20 msgr2=0x7f07300591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0730056d20 0x7f07300591e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f074c006ef0 tx=0x7f074c015000).stop 2020-11-20T12:03:07.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa400 msgr2=0x7f07580ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa400 0x7f07580ad0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f0754001e70 tx=0x7f0754007c50).stop 2020-11-20T12:03:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 shutdown_connections 2020-11-20T12:03:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0730056d20 0x7f07300591e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f075813ac60 0x7f0758131a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07580aa020 0x7f0758067560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 --2- 172.21.15.44:0/1424439111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07580aa400 0x7f07580ad0e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 >> 172.21.15.44:0/1424439111 conn(0x7f0758062690 msgr2=0x7f0758060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 shutdown_connections 2020-11-20T12:03:07.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.051+0000 7f075f44b700 1 -- 172.21.15.44:0/1424439111 wait complete. 2020-11-20T12:03:07.066 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2020-11-20T12:03:07.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.190+0000 7fa67ff94700 1 Processor -- start 2020-11-20T12:03:07.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.190+0000 7fa67ff94700 1 -- start start 2020-11-20T12:03:07.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67ff94700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa6780a3e10 0x7fa6780a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67ff94700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6780a9770 con 0x7fa6780aa020 2020-11-20T12:03:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67ff94700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6780a6da0 con 0x7fa6780aa400 2020-11-20T12:03:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67ff94700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780a56c0 con 0x7fa6780a3e10 2020-11-20T12:03:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67dd30700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa6780aa020 0x7fa6780a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39926/0 (socket says 172.21.15.44:39926) 2020-11-20T12:03:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67dd30700 1 -- 172.21.15.44:0/3036399667 learned_addr learned my addr 172.21.15.44:0/3036399667 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:07.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67e531700 1 --2- 172.21.15.44:0/3036399667 >> v2:172.21.15.79:3300/0 conn(0x7fa6780a3e10 0x7fa6780a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396442560 0 0) 0x7fa6680010d0 con 0x7fa6780aa020 2020-11-20T12:03:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa660003240 con 0x7fa6780aa020 2020-11-20T12:03:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2935260219 0 0) 0x7fa660003240 con 0x7fa6780aa020 2020-11-20T12:03:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 >> v2:172.21.15.79:3300/0 conn(0x7fa6780a3e10 msgr2=0x7fa6780a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 --2- 172.21.15.44:0/3036399667 >> v2:172.21.15.79:3300/0 conn(0x7fa6780a3e10 0x7fa6780a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 >> v1:172.21.15.79:6789/0 conn(0x7fa6780aa400 legacy=0x7fa6780a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780a7040 con 0x7fa6780aa020 2020-11-20T12:03:07.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fa668001670 con 0x7fa6780aa020 2020-11-20T12:03:07.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 >> v1:172.21.15.44:6789/0 conn(0x7fa6780aa020 legacy=0x7fa6780a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660003730 0x7fa660003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6600044e0 0x7fa660004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa660004e50 0x7fa66000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780a56c0 con 0x7fa660004e50 2020-11-20T12:03:07.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600041d0 con 0x7fa6600044e0 2020-11-20T12:03:07.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66000bc20 con 0x7fa660003730 2020-11-20T12:03:07.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.191+0000 7fa67e531700 1 --2- 172.21.15.44:0/3036399667 >> v2:172.21.15.79:3300/0 conn(0x7fa6780a3e10 0x7fa6780a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:07.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa660004e50 0x7fa66000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67d52f700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6600044e0 0x7fa660004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67dd30700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660003730 0x7fa660003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 -- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660003730 msgr2=0x7fa660003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660003730 0x7fa660003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 -- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6600044e0 msgr2=0x7fa660004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6600044e0 0x7fa660004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 -- 172.21.15.44:0/3036399667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa660003240 con 0x7fa660004e50 2020-11-20T12:03:07.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 -- 172.21.15.44:0/3036399667 reap_dead start 2020-11-20T12:03:07.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67e531700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa660004e50 0x7fa66000b730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa67000e2d0 tx=0x7fa6700091d0).ready entity=mon.0 client_cookie=8cc980a94f184974 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67000eea0 con 0x7fa660004e50 2020-11-20T12:03:07.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670009800 con 0x7fa660004e50 2020-11-20T12:03:07.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.192+0000 7fa67cd2e700 1 -- 172.21.15.44:0/3036399667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670012600 con 0x7fa660004e50 2020-11-20T12:03:07.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.193+0000 7fa67ff94700 1 -- 172.21.15.44:0/3036399667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa660004e50 msgr2=0x7fa66000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.193+0000 7fa67ff94700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa660004e50 0x7fa66000b730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa67000e2d0 tx=0x7fa6700091d0).stop 2020-11-20T12:03:07.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.193+0000 7fa67ff94700 1 -- 172.21.15.44:0/3036399667 shutdown_connections 2020-11-20T12:03:07.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.193+0000 7fa67ff94700 1 --2- 172.21.15.44:0/3036399667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa660004e50 0x7fa66000b730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.193+0000 7fa67ff94700 1 -- 172.21.15.44:0/3036399667 >> 172.21.15.44:0/3036399667 conn(0x7fa678062690 msgr2=0x7fa6780a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.193+0000 7fa67ff94700 1 -- 172.21.15.44:0/3036399667 shutdown_connections 2020-11-20T12:03:07.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.193+0000 7fa67ff94700 1 -- 172.21.15.44:0/3036399667 wait complete. 2020-11-20T12:03:07.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 Processor -- start 2020-11-20T12:03:07.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 -- start start 2020-11-20T12:03:07.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660004e50 0x7fa6780ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678067840 0x7fa678147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa678147d50 0x7fa67814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa678134d10 con 0x7fa678147d50 2020-11-20T12:03:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa678067fc0 con 0x7fa678067840 2020-11-20T12:03:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67ff94700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6781473c0 con 0x7fa660004e50 2020-11-20T12:03:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67e531700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa678147d50 0x7fa67814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67e531700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa678147d50 0x7fa67814bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36194/0 (socket says 172.21.15.44:36194) 2020-11-20T12:03:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67e531700 1 -- 172.21.15.44:0/2545231887 learned_addr learned my addr 172.21.15.44:0/2545231887 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67dd30700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660004e50 0x7fa6780ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.194+0000 7fa67d52f700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678067840 0x7fa678147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67e531700 1 -- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660004e50 msgr2=0x7fa6780ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67e531700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa660004e50 0x7fa6780ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67e531700 1 -- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678067840 msgr2=0x7fa678147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67e531700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa678067840 0x7fa678147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67e531700 1 -- 172.21.15.44:0/2545231887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678067cc0 con 0x7fa678147d50 2020-11-20T12:03:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67e531700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa678147d50 0x7fa67814bb20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fa6780a6570 tx=0x7fa67001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67001b420 con 0x7fa678147d50 2020-11-20T12:03:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67001b580 con 0x7fa678147d50 2020-11-20T12:03:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67001b920 con 0x7fa678147d50 2020-11-20T12:03:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa678066b50 con 0x7fa678147d50 2020-11-20T12:03:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.195+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67814c1d0 con 0x7fa678147d50 2020-11-20T12:03:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.196+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa678147240 con 0x7fa678147d50 2020-11-20T12:03:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.198+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa67002c070 con 0x7fa678147d50 2020-11-20T12:03:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa665ffb700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa650056d50 0x7fa650059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa670071f40 con 0x7fa678147d50 2020-11-20T12:03:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa665ffb700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:6809/24728,v1:172.21.15.44:6811/24728] conn(0x7fa65005e930 0x7fa650060d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 --> [v2:172.21.15.44:6809/24728,v1:172.21.15.44:6811/24728] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6500613b0 con 0x7fa65005e930 2020-11-20T12:03:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa670072230 con 0x7fa678147d50 2020-11-20T12:03:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa67dd30700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa650056d50 0x7fa650059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa67d52f700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:6809/24728,v1:172.21.15.44:6811/24728] conn(0x7fa65005e930 0x7fa650060d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa67dd30700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa650056d50 0x7fa650059210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fa668007990 tx=0x7fa668001e50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.199+0000 7fa67d52f700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:6809/24728,v1:172.21.15.44:6811/24728] conn(0x7fa65005e930 0x7fa650060d50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.200+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== osd.0 v2:172.21.15.44:6809/24728 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fa6500613b0 con 0x7fa65005e930 2020-11-20T12:03:07.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 --> [v2:172.21.15.44:6809/24728,v1:172.21.15.44:6811/24728] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa678147580 con 0x7fa65005e930 2020-11-20T12:03:07.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa665ffb700 1 -- 172.21.15.44:0/2545231887 <== osd.0 v2:172.21.15.44:6809/24728 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa678147580 con 0x7fa65005e930 2020-11-20T12:03:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:6809/24728,v1:172.21.15.44:6811/24728] conn(0x7fa65005e930 msgr2=0x7fa650060d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa67ff94700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:6809/24728,v1:172.21.15.44:6811/24728] conn(0x7fa65005e930 0x7fa650060d50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa650056d50 msgr2=0x7fa650059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa67ff94700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa650056d50 0x7fa650059210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fa668007990 tx=0x7fa668001e50).stop 2020-11-20T12:03:07.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa678147d50 msgr2=0x7fa67814bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.217+0000 7fa67ff94700 1 --2- 172.21.15.44:0/2545231887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa678147d50 0x7fa67814bb20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fa6780a6570 tx=0x7fa67001a000).stop 2020-11-20T12:03:07.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.218+0000 7fa67e531700 1 -- 172.21.15.44:0/2545231887 reap_dead start 2020-11-20T12:03:07.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.218+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 shutdown_connections 2020-11-20T12:03:07.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.218+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 >> 172.21.15.44:0/2545231887 conn(0x7fa678062690 msgr2=0x7fa678136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.218+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 shutdown_connections 2020-11-20T12:03:07.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.218+0000 7fa67ff94700 1 -- 172.21.15.44:0/2545231887 wait complete. 2020-11-20T12:03:07.232 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:07.232 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2020-11-20T12:03:07.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.356+0000 7fd8fe772700 1 Processor -- start 2020-11-20T12:03:07.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.356+0000 7fd8fe772700 1 -- start start 2020-11-20T12:03:07.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.356+0000 7fd8fe772700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8f813a4a0 0x7fd8f80a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.356+0000 7fd8fe772700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80a5140 con 0x7fd8f8061180 2020-11-20T12:03:07.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.356+0000 7fd8fe772700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80a7ad0 con 0x7fd8f8061560 2020-11-20T12:03:07.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.356+0000 7fd8fe772700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f80a5fd0 con 0x7fd8f813a4a0 2020-11-20T12:03:07.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f7fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd8f8061180 0x7fd8f80a73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39948/0 (socket says 172.21.15.44:39948) 2020-11-20T12:03:07.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f7fff700 1 -- 172.21.15.44:0/2084361738 learned_addr learned my addr 172.21.15.44:0/2084361738 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:07.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8fcd0f700 1 --2- 172.21.15.44:0/2084361738 >> v2:172.21.15.79:3300/0 conn(0x7fd8f813a4a0 0x7fd8f80a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015776104 0 0) 0x7fd8f80a5140 con 0x7fd8f8061180 2020-11-20T12:03:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e4003240 con 0x7fd8f8061180 2020-11-20T12:03:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534079378 0 0) 0x7fd8f80a7ad0 con 0x7fd8f8061560 2020-11-20T12:03:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f80a5140 con 0x7fd8f8061560 2020-11-20T12:03:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 62285361 0 0) 0x7fd8e4003240 con 0x7fd8f8061180 2020-11-20T12:03:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 >> v2:172.21.15.79:3300/0 conn(0x7fd8f813a4a0 msgr2=0x7fd8f80a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 --2- 172.21.15.44:0/2084361738 >> v2:172.21.15.79:3300/0 conn(0x7fd8f813a4a0 0x7fd8f80a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 >> v1:172.21.15.79:6789/0 conn(0x7fd8f8061560 legacy=0x7fd8f813aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80a53a0 con 0x7fd8f8061180 2020-11-20T12:03:07.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fd8e80014e0 con 0x7fd8f8061180 2020-11-20T12:03:07.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 >> v1:172.21.15.44:6789/0 conn(0x7fd8f8061180 legacy=0x7fd8f80a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40038c0 0x7fd8e4003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40044e0 0x7fd8e4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8e4004e50 0x7fd8e400c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80a5fd0 con 0x7fd8e40038c0 2020-11-20T12:03:07.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e40053b0 con 0x7fd8e40044e0 2020-11-20T12:03:07.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400cdb0 con 0x7fd8e4004e50 2020-11-20T12:03:07.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f7fff700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40038c0 0x7fd8e4003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8f77fe700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40044e0 0x7fd8e4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.357+0000 7fd8fcd0f700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8e4004e50 0x7fd8e400c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f7fff700 1 -- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8e4004e50 msgr2=0x7fd8e400c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f7fff700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8e4004e50 0x7fd8e400c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f7fff700 1 -- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40044e0 msgr2=0x7fd8e4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f7fff700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40044e0 0x7fd8e4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fcd0f700 1 -- 172.21.15.44:0/2084361738 reap_dead start 2020-11-20T12:03:07.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f7fff700 1 -- 172.21.15.44:0/2084361738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4003240 con 0x7fd8e40038c0 2020-11-20T12:03:07.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f7fff700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40038c0 0x7fd8e4003cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fd8e800c440 tx=0x7fd8f8137f20).ready entity=mon.0 client_cookie=a7794d70fbb3eea4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80070d0 con 0x7fd8e40038c0 2020-11-20T12:03:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8007230 con 0x7fd8e40038c0 2020-11-20T12:03:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8f6ffd700 1 -- 172.21.15.44:0/2084361738 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80055f0 con 0x7fd8e40038c0 2020-11-20T12:03:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fe772700 1 -- 172.21.15.44:0/2084361738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40038c0 msgr2=0x7fd8e4003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fe772700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40038c0 0x7fd8e4003cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fd8e800c440 tx=0x7fd8f8137f20).stop 2020-11-20T12:03:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fe772700 1 -- 172.21.15.44:0/2084361738 shutdown_connections 2020-11-20T12:03:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fe772700 1 --2- 172.21.15.44:0/2084361738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8e40038c0 0x7fd8e4003cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fe772700 1 -- 172.21.15.44:0/2084361738 >> 172.21.15.44:0/2084361738 conn(0x7fd8f80ad6e0 msgr2=0x7fd8f80adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fe772700 1 -- 172.21.15.44:0/2084361738 shutdown_connections 2020-11-20T12:03:07.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.358+0000 7fd8fe772700 1 -- 172.21.15.44:0/2084361738 wait complete. 2020-11-20T12:03:07.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.359+0000 7fd8fe772700 1 Processor -- start 2020-11-20T12:03:07.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fe772700 1 -- start start 2020-11-20T12:03:07.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fe772700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40038c0 0x7fd8f8135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fe772700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8f814afd0 0x7fd8f81372d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fe772700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8f80a23e0 0x7fd8f80a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fe772700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f814a5f0 con 0x7fd8f814afd0 2020-11-20T12:03:07.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fe772700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f814a890 con 0x7fd8e40038c0 2020-11-20T12:03:07.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fe772700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f814ab30 con 0x7fd8f80a23e0 2020-11-20T12:03:07.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8f814afd0 0x7fd8f81372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8f814afd0 0x7fd8f81372d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36214/0 (socket says 172.21.15.44:36214) 2020-11-20T12:03:07.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f77fe700 1 -- 172.21.15.44:0/2316828849 learned_addr learned my addr 172.21.15.44:0/2316828849 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:07.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f7fff700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40038c0 0x7fd8f8135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8fcd0f700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8f80a23e0 0x7fd8f80a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f77fe700 1 -- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8f80a23e0 msgr2=0x7fd8f80a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f77fe700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8f80a23e0 0x7fd8f80a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f77fe700 1 -- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40038c0 msgr2=0x7fd8f8135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.360+0000 7fd8f77fe700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e40038c0 0x7fd8f8135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8f77fe700 1 -- 172.21.15.44:0/2316828849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f81377c0 con 0x7fd8f814afd0 2020-11-20T12:03:07.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8f77fe700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8f814afd0 0x7fd8f81372d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd8ec002cb0 tx=0x7fd8ec007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec007020 con 0x7fd8f814afd0 2020-11-20T12:03:07.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec007180 con 0x7fd8f814afd0 2020-11-20T12:03:07.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f8134f50 con 0x7fd8f814afd0 2020-11-20T12:03:07.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00fa30 con 0x7fd8f814afd0 2020-11-20T12:03:07.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f80a2a20 con 0x7fd8f814afd0 2020-11-20T12:03:07.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.361+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd8f8061c60 con 0x7fd8f814afd0 2020-11-20T12:03:07.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.362+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd8ec024070 con 0x7fd8f814afd0 2020-11-20T12:03:07.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.362+0000 7fd8d7fff700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd8d0056d20 0x7fd8d00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd8ec070ac0 con 0x7fd8f814afd0 2020-11-20T12:03:07.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8d7fff700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:6808/24729,v1:172.21.15.44:6810/24729] conn(0x7fd8d005e880 0x7fd8d0060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 --> [v2:172.21.15.44:6808/24729,v1:172.21.15.44:6810/24729] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8d0061300 con 0x7fd8d005e880 2020-11-20T12:03:07.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8ec070ec0 con 0x7fd8f814afd0 2020-11-20T12:03:07.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8f7fff700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd8d0056d20 0x7fd8d00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8fcd0f700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:6808/24729,v1:172.21.15.44:6810/24729] conn(0x7fd8d005e880 0x7fd8d0060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8f7fff700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd8d0056d20 0x7fd8d00591e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fd8e80078a0 tx=0x7fd8e8013000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.363+0000 7fd8fcd0f700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:6808/24729,v1:172.21.15.44:6810/24729] conn(0x7fd8d005e880 0x7fd8d0060ca0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.364+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== osd.1 v2:172.21.15.44:6808/24729 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fd8d0061300 con 0x7fd8d005e880 2020-11-20T12:03:07.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.380+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 --> [v2:172.21.15.44:6808/24729,v1:172.21.15.44:6810/24729] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd8f814a470 con 0x7fd8d005e880 2020-11-20T12:03:07.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8d7fff700 1 -- 172.21.15.44:0/2316828849 <== osd.1 v2:172.21.15.44:6808/24729 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd8f0009960 con 0x7fd8d005e880 2020-11-20T12:03:07.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:6808/24729,v1:172.21.15.44:6810/24729] conn(0x7fd8d005e880 msgr2=0x7fd8d0060ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:6808/24729,v1:172.21.15.44:6810/24729] conn(0x7fd8d005e880 0x7fd8d0060ca0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd8d0056d20 msgr2=0x7fd8d00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd8d0056d20 0x7fd8d00591e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fd8e80078a0 tx=0x7fd8e8013000).stop 2020-11-20T12:03:07.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8f814afd0 msgr2=0x7fd8f81372d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 --2- 172.21.15.44:0/2316828849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8f814afd0 0x7fd8f81372d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd8ec002cb0 tx=0x7fd8ec007b90).stop 2020-11-20T12:03:07.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fcd0f700 1 -- 172.21.15.44:0/2316828849 reap_dead start 2020-11-20T12:03:07.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 shutdown_connections 2020-11-20T12:03:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 >> 172.21.15.44:0/2316828849 conn(0x7fd8f80ad6e0 msgr2=0x7fd8f81383a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 shutdown_connections 2020-11-20T12:03:07.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.381+0000 7fd8fe772700 1 -- 172.21.15.44:0/2316828849 wait complete. 2020-11-20T12:03:07.395 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:07.396 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2020-11-20T12:03:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.519+0000 7fbfffdac700 1 Processor -- start 2020-11-20T12:03:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.519+0000 7fbfffdac700 1 -- start start 2020-11-20T12:03:07.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbfffdac700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff813a990 0x7fbff813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbfffdac700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbff813f800 0x7fbff8141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbfffdac700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff80a6260 con 0x7fbff8060330 2020-11-20T12:03:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbfffdac700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff80a52e0 con 0x7fbff813f800 2020-11-20T12:03:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbfffdac700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff80a6490 con 0x7fbff813a990 2020-11-20T12:03:07.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffd347700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff813a990 0x7fbff813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffe349700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbff813f800 0x7fbff8141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffd347700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff813a990 0x7fbff813adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:47660/0 (socket says 172.21.15.44:47660) 2020-11-20T12:03:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffd347700 1 -- 172.21.15.44:0/2009746558 learned_addr learned my addr 172.21.15.44:0/2009746558 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:07.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffdb48700 1 --1- 172.21.15.44:0/2009746558 >> v1:172.21.15.79:6789/0 conn(0x7fbff8060330 0x7fbff80a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:42846/0 (socket says 172.21.15.44:42846) 2020-11-20T12:03:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffd347700 1 -- 172.21.15.44:0/2009746558 >> v2:172.21.15.79:3300/0 conn(0x7fbff813f800 msgr2=0x7fbff8141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffd347700 1 --2- 172.21.15.44:0/2009746558 >> v2:172.21.15.79:3300/0 conn(0x7fbff813f800 0x7fbff8141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffd347700 1 -- 172.21.15.44:0/2009746558 >> v1:172.21.15.79:6789/0 conn(0x7fbff8060330 legacy=0x7fbff80a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.520+0000 7fbffd347700 1 -- 172.21.15.44:0/2009746558 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff80a6730 con 0x7fbff813a990 2020-11-20T12:03:07.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbffd347700 1 --2- 172.21.15.44:0/2009746558 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff813a990 0x7fbff813adf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbff4006fd0 tx=0x7fbff4006eb0).ready entity=mon.2 client_cookie=a4103cf1bbe3e7c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbffcb46700 1 -- 172.21.15.44:0/2009746558 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4009810 con 0x7fbff813a990 2020-11-20T12:03:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbffcb46700 1 -- 172.21.15.44:0/2009746558 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4009970 con 0x7fbff813a990 2020-11-20T12:03:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbffcb46700 1 -- 172.21.15.44:0/2009746558 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4009cf0 con 0x7fbff813a990 2020-11-20T12:03:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 -- 172.21.15.44:0/2009746558 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff813a990 msgr2=0x7fbff813adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 --2- 172.21.15.44:0/2009746558 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff813a990 0x7fbff813adf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbff4006fd0 tx=0x7fbff4006eb0).stop 2020-11-20T12:03:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 -- 172.21.15.44:0/2009746558 shutdown_connections 2020-11-20T12:03:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 --2- 172.21.15.44:0/2009746558 >> v2:172.21.15.79:3300/0 conn(0x7fbff813f800 0x7fbff8141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 --2- 172.21.15.44:0/2009746558 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff813a990 0x7fbff813adf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 -- 172.21.15.44:0/2009746558 >> 172.21.15.44:0/2009746558 conn(0x7fbff8062690 msgr2=0x7fbff80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 -- 172.21.15.44:0/2009746558 shutdown_connections 2020-11-20T12:03:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.521+0000 7fbfffdac700 1 -- 172.21.15.44:0/2009746558 wait complete. 2020-11-20T12:03:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.522+0000 7fbfffdac700 1 Processor -- start 2020-11-20T12:03:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.522+0000 7fbfffdac700 1 -- start start 2020-11-20T12:03:07.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfffdac700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff8060330 0x7fbff8137910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfffdac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff813a990 0x7fbff813b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfffdac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff813f800 0x7fbff812f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfffdac700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff813d620 con 0x7fbff813f800 2020-11-20T12:03:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfffdac700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff813e450 con 0x7fbff813a990 2020-11-20T12:03:07.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfffdac700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff813d810 con 0x7fbff8060330 2020-11-20T12:03:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff813f800 0x7fbff812f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff813f800 0x7fbff812f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36232/0 (socket says 172.21.15.44:36232) 2020-11-20T12:03:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 -- 172.21.15.44:0/378580947 learned_addr learned my addr 172.21.15.44:0/378580947 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffdb48700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff8060330 0x7fbff8137910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffd347700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff813a990 0x7fbff813b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 -- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff8060330 msgr2=0x7fbff8137910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff8060330 0x7fbff8137910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 -- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff813a990 msgr2=0x7fbff813b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff813a990 0x7fbff813b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 -- 172.21.15.44:0/378580947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff80ad420 con 0x7fbff813f800 2020-11-20T12:03:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffdb48700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff8060330 0x7fbff8137910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbffe349700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff813f800 0x7fbff812f0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fbff000ed20 tx=0x7fbff00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000eee0 con 0x7fbff813f800 2020-11-20T12:03:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff000cb00 con 0x7fbff813f800 2020-11-20T12:03:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.523+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff813be00 con 0x7fbff813f800 2020-11-20T12:03:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.524+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000cea0 con 0x7fbff813f800 2020-11-20T12:03:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.524+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff80adc90 con 0x7fbff813f800 2020-11-20T12:03:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.524+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbff813e650 con 0x7fbff813f800 2020-11-20T12:03:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbff0022070 con 0x7fbff813f800 2020-11-20T12:03:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbfe5ffb700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfd0056d20 0x7fbfd00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbff0070660 con 0x7fbff813f800 2020-11-20T12:03:07.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbfe5ffb700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:6800/24727,v1:172.21.15.44:6801/24727] conn(0x7fbfd005e900 0x7fbfd0060d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 --> [v2:172.21.15.44:6800/24727,v1:172.21.15.44:6801/24727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfd0061380 con 0x7fbfd005e900 2020-11-20T12:03:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbff0072cd0 con 0x7fbff813f800 2020-11-20T12:03:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbffdb48700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfd0056d20 0x7fbfd00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.525+0000 7fbffd347700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:6800/24727,v1:172.21.15.44:6801/24727] conn(0x7fbfd005e900 0x7fbfd0060d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.526+0000 7fbffdb48700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfd0056d20 0x7fbfd00591e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fbfe8000bc0 tx=0x7fbfe8007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.526+0000 7fbffd347700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:6800/24727,v1:172.21.15.44:6801/24727] conn(0x7fbfd005e900 0x7fbfd0060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.526+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== osd.2 v2:172.21.15.44:6800/24727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fbfd0061380 con 0x7fbfd005e900 2020-11-20T12:03:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.544+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 --> [v2:172.21.15.44:6800/24727,v1:172.21.15.44:6801/24727] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbff8134200 con 0x7fbfd005e900 2020-11-20T12:03:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.544+0000 7fbfe5ffb700 1 -- 172.21.15.44:0/378580947 <== osd.2 v2:172.21.15.44:6800/24727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbff8134200 con 0x7fbfd005e900 2020-11-20T12:03:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:6800/24727,v1:172.21.15.44:6801/24727] conn(0x7fbfd005e900 msgr2=0x7fbfd0060d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:6800/24727,v1:172.21.15.44:6801/24727] conn(0x7fbfd005e900 0x7fbfd0060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfd0056d20 msgr2=0x7fbfd00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfd0056d20 0x7fbfd00591e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fbfe8000bc0 tx=0x7fbfe8007b40).stop 2020-11-20T12:03:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff813f800 msgr2=0x7fbff812f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 --2- 172.21.15.44:0/378580947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff813f800 0x7fbff812f0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fbff000ed20 tx=0x7fbff00091d0).stop 2020-11-20T12:03:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbffe349700 1 -- 172.21.15.44:0/378580947 reap_dead start 2020-11-20T12:03:07.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 shutdown_connections 2020-11-20T12:03:07.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 >> 172.21.15.44:0/378580947 conn(0x7fbff8062690 msgr2=0x7fbff8139910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 shutdown_connections 2020-11-20T12:03:07.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.545+0000 7fbfffdac700 1 -- 172.21.15.44:0/378580947 wait complete. 2020-11-20T12:03:07.559 INFO:teuthology.orchestra.run.smithi044.stdout:55834574854 2020-11-20T12:03:07.560 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2020-11-20T12:03:07.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.683+0000 7f92223a1700 1 Processor -- start 2020-11-20T12:03:07.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.683+0000 7f92223a1700 1 -- start start 2020-11-20T12:03:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f92223a1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f921c13a990 0x7f921c13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f92223a1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f921c13f800 0x7f921c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f92223a1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c0a6260 con 0x7f921c060330 2020-11-20T12:03:07.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f92223a1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921c0a52e0 con 0x7f921c13a990 2020-11-20T12:03:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f92223a1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921c0a6490 con 0x7f921c13f800 2020-11-20T12:03:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f921b7fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f921c13a990 0x7f921c13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f921bfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f921c060330 0x7f921c0a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:39986/0 (socket says 172.21.15.44:39986) 2020-11-20T12:03:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f921bfff700 1 -- 172.21.15.44:0/3726064759 learned_addr learned my addr 172.21.15.44:0/3726064759 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:07.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f921b7fe700 1 -- 172.21.15.44:0/3726064759 >> v2:172.21.15.79:3300/0 conn(0x7f921c13f800 msgr2=0x7f921c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f922093e700 1 --2- 172.21.15.44:0/3726064759 >> v2:172.21.15.79:3300/0 conn(0x7f921c13f800 0x7f921c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f921b7fe700 1 --2- 172.21.15.44:0/3726064759 >> v2:172.21.15.79:3300/0 conn(0x7f921c13f800 0x7f921c141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.684+0000 7f921affd700 1 -- 172.21.15.44:0/3726064759 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019509916 0 0) 0x7f921c0a6260 con 0x7f921c060330 2020-11-20T12:03:07.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f921b7fe700 1 -- 172.21.15.44:0/3726064759 >> v1:172.21.15.44:6789/0 conn(0x7f921c060330 legacy=0x7f921c0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f921b7fe700 1 -- 172.21.15.44:0/3726064759 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0a6730 con 0x7f921c13a990 2020-11-20T12:03:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f921b7fe700 1 --2- 172.21.15.44:0/3726064759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c13ae20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f921c0a5a90 tx=0x7f9210006df0).ready entity=mon.0 client_cookie=2c23b7338912fff6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f921affd700 1 -- 172.21.15.44:0/3726064759 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100097b0 con 0x7f921c13a990 2020-11-20T12:03:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f921affd700 1 -- 172.21.15.44:0/3726064759 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210009910 con 0x7f921c13a990 2020-11-20T12:03:07.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f921affd700 1 -- 172.21.15.44:0/3726064759 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210009c90 con 0x7f921c13a990 2020-11-20T12:03:07.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 -- 172.21.15.44:0/3726064759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 msgr2=0x7f921c13ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 --2- 172.21.15.44:0/3726064759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c13ae20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f921c0a5a90 tx=0x7f9210006df0).stop 2020-11-20T12:03:07.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 -- 172.21.15.44:0/3726064759 shutdown_connections 2020-11-20T12:03:07.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 --2- 172.21.15.44:0/3726064759 >> v2:172.21.15.79:3300/0 conn(0x7f921c13f800 0x7f921c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 --2- 172.21.15.44:0/3726064759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c13ae20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 -- 172.21.15.44:0/3726064759 >> 172.21.15.44:0/3726064759 conn(0x7f921c062690 msgr2=0x7f921c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 -- 172.21.15.44:0/3726064759 shutdown_connections 2020-11-20T12:03:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.685+0000 7f92223a1700 1 -- 172.21.15.44:0/3726064759 wait complete. 2020-11-20T12:03:07.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.686+0000 7f92223a1700 1 Processor -- start 2020-11-20T12:03:07.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f92223a1700 1 -- start start 2020-11-20T12:03:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f92223a1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921c060330 0x7f921c0aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f92223a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c135160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f92223a1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921c13f800 0x7f921c14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f92223a1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c149f00 con 0x7f921c13a990 2020-11-20T12:03:07.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f92223a1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c135650 con 0x7f921c060330 2020-11-20T12:03:07.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f92223a1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0a2cd0 con 0x7f921c13f800 2020-11-20T12:03:07.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f921b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c135160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36246/0 (socket says 172.21.15.44:36246) 2020-11-20T12:03:07.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f921b7fe700 1 -- 172.21.15.44:0/1549049845 learned_addr learned my addr 172.21.15.44:0/1549049845 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:07.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f921bfff700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921c060330 0x7f921c0aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.687+0000 7f922093e700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921c13f800 0x7f921c14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f921b7fe700 1 -- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921c13f800 msgr2=0x7f921c14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f921b7fe700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f921c13f800 0x7f921c14a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f921b7fe700 1 -- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921c060330 msgr2=0x7f921c0aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f921b7fe700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f921c060330 0x7f921c0aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f921b7fe700 1 -- 172.21.15.44:0/1549049845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0a2f70 con 0x7f921c13a990 2020-11-20T12:03:07.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f921b7fe700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c135160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f921c0aa8b0 tx=0x7f9210015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210011e50 con 0x7f921c13a990 2020-11-20T12:03:07.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921001e460 con 0x7f921c13a990 2020-11-20T12:03:07.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921001e7e0 con 0x7f921c13a990 2020-11-20T12:03:07.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c0a3210 con 0x7f921c13a990 2020-11-20T12:03:07.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.688+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c0a28a0 con 0x7f921c13a990 2020-11-20T12:03:07.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.689+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f921c061650 con 0x7f921c13a990 2020-11-20T12:03:07.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.689+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9210025030 con 0x7f921c13a990 2020-11-20T12:03:07.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.689+0000 7f91fbfff700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f4056d20 0x7f91f40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.689+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f92100718f0 con 0x7f921c13a990 2020-11-20T12:03:07.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.690+0000 7f91fbfff700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:6824/24740,v1:172.21.15.44:6825/24740] conn(0x7f91f405e900 0x7f91f4060d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.690+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 --> [v2:172.21.15.44:6824/24740,v1:172.21.15.44:6825/24740] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91f4061380 con 0x7f91f405e900 2020-11-20T12:03:07.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.690+0000 7f921bfff700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f4056d20 0x7f91f40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.690+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9210076220 con 0x7f921c13a990 2020-11-20T12:03:07.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.690+0000 7f922093e700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:6824/24740,v1:172.21.15.44:6825/24740] conn(0x7f91f405e900 0x7f91f4060d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.690+0000 7f921bfff700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f4056d20 0x7f91f40591e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f920c0011a0 tx=0x7f920c0079e0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.690+0000 7f922093e700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:6824/24740,v1:172.21.15.44:6825/24740] conn(0x7f91f405e900 0x7f91f4060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.691+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== osd.3 v2:172.21.15.44:6824/24740 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f91f4061380 con 0x7f91f405e900 2020-11-20T12:03:07.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.707+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 --> [v2:172.21.15.44:6824/24740,v1:172.21.15.44:6825/24740] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f921c0446d0 con 0x7f91f405e900 2020-11-20T12:03:07.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f91fbfff700 1 -- 172.21.15.44:0/1549049845 <== osd.3 v2:172.21.15.44:6824/24740 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f921c0446d0 con 0x7f91f405e900 2020-11-20T12:03:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:6824/24740,v1:172.21.15.44:6825/24740] conn(0x7f91f405e900 msgr2=0x7f91f4060d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:6824/24740,v1:172.21.15.44:6825/24740] conn(0x7f91f405e900 0x7f91f4060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f4056d20 msgr2=0x7f91f40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f4056d20 0x7f91f40591e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f920c0011a0 tx=0x7f920c0079e0).stop 2020-11-20T12:03:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 msgr2=0x7f921c135160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 --2- 172.21.15.44:0/1549049845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921c13a990 0x7f921c135160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f921c0aa8b0 tx=0x7f9210015000).stop 2020-11-20T12:03:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f922093e700 1 -- 172.21.15.44:0/1549049845 reap_dead start 2020-11-20T12:03:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 shutdown_connections 2020-11-20T12:03:07.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 >> 172.21.15.44:0/1549049845 conn(0x7f921c062690 msgr2=0x7f921c1394c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 shutdown_connections 2020-11-20T12:03:07.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.708+0000 7f92223a1700 1 -- 172.21.15.44:0/1549049845 wait complete. 2020-11-20T12:03:07.723 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:07.723 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2020-11-20T12:03:07.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.847+0000 7f143c61f700 1 Processor -- start 2020-11-20T12:03:07.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.847+0000 7f143c61f700 1 -- start start 2020-11-20T12:03:07.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143c61f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f14340aa020 0x7f14340a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143c61f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f14340aa400 0x7f14340a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143c61f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f143413ac60 0x7f14340a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143c61f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340a9770 con 0x7f14340aa020 2020-11-20T12:03:07.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143c61f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340a6da0 con 0x7f14340aa400 2020-11-20T12:03:07.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143c61f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340a7040 con 0x7f143413ac60 2020-11-20T12:03:07.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f14340aa020 0x7f14340a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f14340aa020 0x7f14340a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36254/0 (socket says 172.21.15.44:36254) 2020-11-20T12:03:07.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 -- 172.21.15.44:0/1105567828 learned_addr learned my addr 172.21.15.44:0/1105567828 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:07.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143abbc700 1 --2- 172.21.15.44:0/1105567828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f143413ac60 0x7f14340a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f1439bba700 1 --2- 172.21.15.44:0/1105567828 >> v2:172.21.15.79:3300/0 conn(0x7f14340aa400 0x7f14340a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 -- 172.21.15.44:0/1105567828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f143413ac60 msgr2=0x7f14340a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 --2- 172.21.15.44:0/1105567828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f143413ac60 0x7f14340a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 -- 172.21.15.44:0/1105567828 >> v2:172.21.15.79:3300/0 conn(0x7f14340aa400 msgr2=0x7f14340a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:07.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 --2- 172.21.15.44:0/1105567828 >> v2:172.21.15.79:3300/0 conn(0x7f14340aa400 0x7f14340a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f143a3bb700 1 -- 172.21.15.44:0/1105567828 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340a6a90 con 0x7f14340aa020 2020-11-20T12:03:07.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.848+0000 7f1439bba700 1 --2- 172.21.15.44:0/1105567828 >> v2:172.21.15.79:3300/0 conn(0x7f14340aa400 0x7f14340a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:07.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143a3bb700 1 --2- 172.21.15.44:0/1105567828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14340aa020 0x7f14340a9280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f14340a4990 tx=0x7f1424006eb0).ready entity=mon.0 client_cookie=1c223af1a60f3e77 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f14393b9700 1 -- 172.21.15.44:0/1105567828 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424009810 con 0x7f14340aa020 2020-11-20T12:03:07.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f14393b9700 1 -- 172.21.15.44:0/1105567828 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424009970 con 0x7f14340aa020 2020-11-20T12:03:07.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f14393b9700 1 -- 172.21.15.44:0/1105567828 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424009cf0 con 0x7f14340aa020 2020-11-20T12:03:07.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 -- 172.21.15.44:0/1105567828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14340aa020 msgr2=0x7f14340a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 --2- 172.21.15.44:0/1105567828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14340aa020 0x7f14340a9280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f14340a4990 tx=0x7f1424006eb0).stop 2020-11-20T12:03:07.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 -- 172.21.15.44:0/1105567828 shutdown_connections 2020-11-20T12:03:07.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 --2- 172.21.15.44:0/1105567828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14340aa020 0x7f14340a9280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 --2- 172.21.15.44:0/1105567828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f143413ac60 0x7f14340a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 --2- 172.21.15.44:0/1105567828 >> v2:172.21.15.79:3300/0 conn(0x7f14340aa400 0x7f14340a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 -- 172.21.15.44:0/1105567828 >> 172.21.15.44:0/1105567828 conn(0x7f1434062690 msgr2=0x7f14340a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 -- 172.21.15.44:0/1105567828 shutdown_connections 2020-11-20T12:03:07.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.849+0000 7f143c61f700 1 -- 172.21.15.44:0/1105567828 wait complete. 2020-11-20T12:03:07.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.850+0000 7f143c61f700 1 Processor -- start 2020-11-20T12:03:07.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.850+0000 7f143c61f700 1 -- start start 2020-11-20T12:03:07.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143c61f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f14340aa020 0x7f14340a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143c61f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14340aa400 0x7f143409e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143c61f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f143413ac60 0x7f143409da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143c61f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340a0b40 con 0x7f143413ac60 2020-11-20T12:03:07.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143c61f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143409df10 con 0x7f14340aa400 2020-11-20T12:03:07.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143c61f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340adc70 con 0x7f14340aa020 2020-11-20T12:03:07.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f143413ac60 0x7f143409da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f143413ac60 0x7f143409da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36262/0 (socket says 172.21.15.44:36262) 2020-11-20T12:03:07.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 -- 172.21.15.44:0/2900758945 learned_addr learned my addr 172.21.15.44:0/2900758945 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:07.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143a3bb700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f14340aa020 0x7f14340a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f1439bba700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14340aa400 0x7f143409e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 -- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f14340aa020 msgr2=0x7f14340a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f14340aa020 0x7f14340a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 -- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14340aa400 msgr2=0x7f143409e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14340aa400 0x7f143409e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 -- 172.21.15.44:0/2900758945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340a08a0 con 0x7f143413ac60 2020-11-20T12:03:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f143abbc700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f143413ac60 0x7f143409da20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f142c00ee50 tx=0x7f142c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.851+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00cc50 con 0x7f143413ac60 2020-11-20T12:03:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.852+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c00cdb0 con 0x7f143413ac60 2020-11-20T12:03:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.852+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143409d580 con 0x7f143413ac60 2020-11-20T12:03:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.852+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c013880 con 0x7f143413ac60 2020-11-20T12:03:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.852+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1434148e40 con 0x7f143413ac60 2020-11-20T12:03:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.852+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1434061650 con 0x7f143413ac60 2020-11-20T12:03:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.853+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f142c013bf0 con 0x7f143413ac60 2020-11-20T12:03:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.853+0000 7f14227fc700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f140c056d20 0x7f140c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.853+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f142c074940 con 0x7f143413ac60 2020-11-20T12:03:07.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.853+0000 7f143a3bb700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f140c056d20 0x7f140c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.853+0000 7f14227fc700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6816/24937,v1:172.21.15.79:6817/24937] conn(0x7f140c05e900 0x7f140c060d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:07.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.853+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 --> [v2:172.21.15.79:6816/24937,v1:172.21.15.79:6817/24937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f140c061380 con 0x7f140c05e900 2020-11-20T12:03:07.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.853+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f142c079220 con 0x7f143413ac60 2020-11-20T12:03:07.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.854+0000 7f1439bba700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6816/24937,v1:172.21.15.79:6817/24937] conn(0x7f140c05e900 0x7f140c060d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:07.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.854+0000 7f143a3bb700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f140c056d20 0x7f140c0591e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f1424006ef0 tx=0x7f1424015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.854+0000 7f1439bba700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6816/24937,v1:172.21.15.79:6817/24937] conn(0x7f140c05e900 0x7f140c060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:07.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.855+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== osd.4 v2:172.21.15.79:6816/24937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f140c061380 con 0x7f140c05e900 2020-11-20T12:03:07.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.873+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 --> [v2:172.21.15.79:6816/24937,v1:172.21.15.79:6817/24937] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f14340acf30 con 0x7f140c05e900 2020-11-20T12:03:07.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f14227fc700 1 -- 172.21.15.44:0/2900758945 <== osd.4 v2:172.21.15.79:6816/24937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f14340acf30 con 0x7f140c05e900 2020-11-20T12:03:07.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6816/24937,v1:172.21.15.79:6817/24937] conn(0x7f140c05e900 msgr2=0x7f140c060d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6816/24937,v1:172.21.15.79:6817/24937] conn(0x7f140c05e900 0x7f140c060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:07.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f140c056d20 msgr2=0x7f140c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f140c056d20 0x7f140c0591e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f1424006ef0 tx=0x7f1424015000).stop 2020-11-20T12:03:07.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f143413ac60 msgr2=0x7f143409da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:07.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 --2- 172.21.15.44:0/2900758945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f143413ac60 0x7f143409da20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f142c00ee50 tx=0x7f142c0091d0).stop 2020-11-20T12:03:07.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143abbc700 1 -- 172.21.15.44:0/2900758945 reap_dead start 2020-11-20T12:03:07.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 shutdown_connections 2020-11-20T12:03:07.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 >> 172.21.15.44:0/2900758945 conn(0x7f1434062690 msgr2=0x7f14340a3910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:07.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 shutdown_connections 2020-11-20T12:03:07.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:07.874+0000 7f143c61f700 1 -- 172.21.15.44:0/2900758945 wait complete. 2020-11-20T12:03:07.888 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:07.889 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2020-11-20T12:03:08.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.013+0000 7f9687f6c700 1 Processor -- start 2020-11-20T12:03:08.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.013+0000 7f9687f6c700 1 -- start start 2020-11-20T12:03:08.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.013+0000 7f9687f6c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96800aa400 0x7f96800a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.013+0000 7f9687f6c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96800a9770 con 0x7f96800aa020 2020-11-20T12:03:08.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.013+0000 7f9687f6c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96800a6da0 con 0x7f968013ac60 2020-11-20T12:03:08.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.013+0000 7f9687f6c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96800a56c0 con 0x7f96800aa400 2020-11-20T12:03:08.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9685d08700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f96800aa020 0x7f96800a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40016/0 (socket says 172.21.15.44:40016) 2020-11-20T12:03:08.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9685d08700 1 -- 172.21.15.44:0/1007587470 learned_addr learned my addr 172.21.15.44:0/1007587470 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:08.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9685507700 1 --2- 172.21.15.44:0/1007587470 >> v2:172.21.15.79:3300/0 conn(0x7f96800aa400 0x7f96800a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547526619 0 0) 0x7f9678001450 con 0x7f96800aa020 2020-11-20T12:03:08.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9668003240 con 0x7f96800aa020 2020-11-20T12:03:08.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716225061 0 0) 0x7f96800a6da0 con 0x7f968013ac60 2020-11-20T12:03:08.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9678001450 con 0x7f968013ac60 2020-11-20T12:03:08.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2489413251 0 0) 0x7f9668003240 con 0x7f96800aa020 2020-11-20T12:03:08.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9685507700 1 -- 172.21.15.44:0/1007587470 >> v1:172.21.15.79:6789/0 conn(0x7f968013ac60 legacy=0x7f96800a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9685507700 1 -- 172.21.15.44:0/1007587470 >> v1:172.21.15.44:6789/0 conn(0x7f96800aa020 legacy=0x7f96800a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9685507700 1 -- 172.21.15.44:0/1007587470 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800a7040 con 0x7f96800aa400 2020-11-20T12:03:08.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9685507700 1 --2- 172.21.15.44:0/1007587470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96800aa400 0x7f96800a9280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f967000b9a0 tx=0x7f9670006df0).ready entity=mon.1 client_cookie=abc18820ae9cb843 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780019f0 con 0x7f96800aa400 2020-11-20T12:03:08.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9670011400 con 0x7f96800aa400 2020-11-20T12:03:08.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.014+0000 7f9684d06700 1 -- 172.21.15.44:0/1007587470 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670011780 con 0x7f96800aa400 2020-11-20T12:03:08.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.015+0000 7f9687f6c700 1 -- 172.21.15.44:0/1007587470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96800aa400 msgr2=0x7f96800a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.015+0000 7f9687f6c700 1 --2- 172.21.15.44:0/1007587470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96800aa400 0x7f96800a9280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f967000b9a0 tx=0x7f9670006df0).stop 2020-11-20T12:03:08.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.015+0000 7f9687f6c700 1 -- 172.21.15.44:0/1007587470 shutdown_connections 2020-11-20T12:03:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.015+0000 7f9687f6c700 1 --2- 172.21.15.44:0/1007587470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96800aa400 0x7f96800a9280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.015+0000 7f9687f6c700 1 -- 172.21.15.44:0/1007587470 >> 172.21.15.44:0/1007587470 conn(0x7f9680062690 msgr2=0x7f96800a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.015+0000 7f9687f6c700 1 -- 172.21.15.44:0/1007587470 shutdown_connections 2020-11-20T12:03:08.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.015+0000 7f9687f6c700 1 -- 172.21.15.44:0/1007587470 wait complete. 2020-11-20T12:03:08.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 Processor -- start 2020-11-20T12:03:08.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 -- start start 2020-11-20T12:03:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f96800aa020 0x7f9680060360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96800aa400 0x7f96800a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f968013ac60 0x7f96800a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800a2f80 con 0x7f96800aa400 2020-11-20T12:03:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800a3220 con 0x7f968013ac60 2020-11-20T12:03:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.016+0000 7f9687f6c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800a1880 con 0x7f96800aa020 2020-11-20T12:03:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96800aa400 0x7f96800a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96800aa400 0x7f96800a27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36276/0 (socket says 172.21.15.44:36276) 2020-11-20T12:03:08.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 -- 172.21.15.44:0/1969334521 learned_addr learned my addr 172.21.15.44:0/1969334521 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685d08700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f96800aa020 0x7f9680060360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9686509700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f968013ac60 0x7f96800a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 -- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f96800aa020 msgr2=0x7f9680060360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f96800aa020 0x7f9680060360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 -- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f968013ac60 msgr2=0x7f96800a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f968013ac60 0x7f96800a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 -- 172.21.15.44:0/1969334521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800a2c90 con 0x7f96800aa400 2020-11-20T12:03:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9686509700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f968013ac60 0x7f96800a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9685507700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96800aa400 0x7f96800a27a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f9680061280 tx=0x7f9670013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670010b10 con 0x7f96800aa400 2020-11-20T12:03:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9670010c70 con 0x7f96800aa400 2020-11-20T12:03:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968014a0f0 con 0x7f96800aa400 2020-11-20T12:03:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.017+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967001e460 con 0x7f96800aa400 2020-11-20T12:03:08.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.018+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96800a2450 con 0x7f96800aa400 2020-11-20T12:03:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.018+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9680061650 con 0x7f96800aa400 2020-11-20T12:03:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9670025030 con 0x7f96800aa400 2020-11-20T12:03:08.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f966dffb700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9658056d20 0x7f96580591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9670071420 con 0x7f96800aa400 2020-11-20T12:03:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f966dffb700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6800/24927,v1:172.21.15.79:6801/24927] conn(0x7f965805e900 0x7f9658060d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 --> [v2:172.21.15.79:6800/24927,v1:172.21.15.79:6801/24927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9658061380 con 0x7f965805e900 2020-11-20T12:03:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9670073a90 con 0x7f96800aa400 2020-11-20T12:03:08.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f9685d08700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9658056d20 0x7f96580591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f9686509700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6800/24927,v1:172.21.15.79:6801/24927] conn(0x7f965805e900 0x7f9658060d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.019+0000 7f9685d08700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9658056d20 0x7f96580591e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f9678001b20 tx=0x7f9678001840).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.020+0000 7f9686509700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6800/24927,v1:172.21.15.79:6801/24927] conn(0x7f965805e900 0x7f9658060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.021+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== osd.5 v2:172.21.15.79:6800/24927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f9658061380 con 0x7f965805e900 2020-11-20T12:03:08.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.038+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 --> [v2:172.21.15.79:6800/24927,v1:172.21.15.79:6801/24927] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96800a1a40 con 0x7f965805e900 2020-11-20T12:03:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f966dffb700 1 -- 172.21.15.44:0/1969334521 <== osd.5 v2:172.21.15.79:6800/24927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96800a1a40 con 0x7f965805e900 2020-11-20T12:03:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6800/24927,v1:172.21.15.79:6801/24927] conn(0x7f965805e900 msgr2=0x7f9658060d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6800/24927,v1:172.21.15.79:6801/24927] conn(0x7f965805e900 0x7f9658060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9658056d20 msgr2=0x7f96580591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9658056d20 0x7f96580591e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f9678001b20 tx=0x7f9678001840).stop 2020-11-20T12:03:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96800aa400 msgr2=0x7f96800a27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 --2- 172.21.15.44:0/1969334521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96800aa400 0x7f96800a27a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f9680061280 tx=0x7f9670013000).stop 2020-11-20T12:03:08.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9686509700 1 -- 172.21.15.44:0/1969334521 reap_dead start 2020-11-20T12:03:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 shutdown_connections 2020-11-20T12:03:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 >> 172.21.15.44:0/1969334521 conn(0x7f9680062690 msgr2=0x7f968013a040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 shutdown_connections 2020-11-20T12:03:08.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.039+0000 7f9687f6c700 1 -- 172.21.15.44:0/1969334521 wait complete. 2020-11-20T12:03:08.053 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:08.054 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2020-11-20T12:03:08.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.176+0000 7f1382952700 1 Processor -- start 2020-11-20T12:03:08.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.176+0000 7f1382952700 1 -- start start 2020-11-20T12:03:08.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f1382952700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f137c0aa020 0x7f137c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f1382952700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 0x7f137c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f1382952700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137c0a9770 con 0x7f137c13ac60 2020-11-20T12:03:08.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f1382952700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0a6da0 con 0x7f137c0aa020 2020-11-20T12:03:08.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f1382952700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0a7040 con 0x7f137c0aa400 2020-11-20T12:03:08.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f137bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f137c0aa020 0x7f137c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f1380eef700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f137c13ac60 0x7f137c0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40036/0 (socket says 172.21.15.44:40036) 2020-11-20T12:03:08.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.177+0000 7f1380eef700 1 -- 172.21.15.44:0/224052303 learned_addr learned my addr 172.21.15.44:0/224052303 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:08.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137b7fe700 1 --2- 172.21.15.44:0/224052303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 0x7f137c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137bfff700 1 -- 172.21.15.44:0/224052303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 msgr2=0x7f137c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:08.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137bfff700 1 --2- 172.21.15.44:0/224052303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 0x7f137c0a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137bfff700 1 -- 172.21.15.44:0/224052303 >> v1:172.21.15.44:6789/0 conn(0x7f137c13ac60 legacy=0x7f137c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137affd700 1 -- 172.21.15.44:0/224052303 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672452252 0 0) 0x7f1374001450 con 0x7f137c13ac60 2020-11-20T12:03:08.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137bfff700 1 -- 172.21.15.44:0/224052303 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0a6a90 con 0x7f137c0aa020 2020-11-20T12:03:08.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137bfff700 1 --2- 172.21.15.44:0/224052303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aa020 0x7f137c0a9280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f136c00b9a0 tx=0x7f136c006df0).ready entity=mon.0 client_cookie=85a3f8e409c4f288 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137affd700 1 -- 172.21.15.44:0/224052303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0097b0 con 0x7f137c0aa020 2020-11-20T12:03:08.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137affd700 1 -- 172.21.15.44:0/224052303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c009910 con 0x7f137c0aa020 2020-11-20T12:03:08.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f137affd700 1 -- 172.21.15.44:0/224052303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c009c90 con 0x7f137c0aa020 2020-11-20T12:03:08.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 -- 172.21.15.44:0/224052303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aa020 msgr2=0x7f137c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 --2- 172.21.15.44:0/224052303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aa020 0x7f137c0a9280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f136c00b9a0 tx=0x7f136c006df0).stop 2020-11-20T12:03:08.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 -- 172.21.15.44:0/224052303 shutdown_connections 2020-11-20T12:03:08.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 --2- 172.21.15.44:0/224052303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aa020 0x7f137c0a9280 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 --2- 172.21.15.44:0/224052303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 0x7f137c0a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 -- 172.21.15.44:0/224052303 >> 172.21.15.44:0/224052303 conn(0x7f137c062690 msgr2=0x7f137c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 -- 172.21.15.44:0/224052303 shutdown_connections 2020-11-20T12:03:08.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.178+0000 7f1382952700 1 -- 172.21.15.44:0/224052303 wait complete. 2020-11-20T12:03:08.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.179+0000 7f1382952700 1 Processor -- start 2020-11-20T12:03:08.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1382952700 1 -- start start 2020-11-20T12:03:08.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1382952700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa020 0x7f137c0ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1382952700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 0x7f137c0ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1382952700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c13ac60 0x7f137c14a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1382952700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c14a020 con 0x7f137c13ac60 2020-11-20T12:03:08.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1382952700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0ac050 con 0x7f137c0aa020 2020-11-20T12:03:08.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1382952700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c060ed0 con 0x7f137c0aa400 2020-11-20T12:03:08.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1380eef700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c13ac60 0x7f137c14a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1380eef700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c13ac60 0x7f137c14a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36294/0 (socket says 172.21.15.44:36294) 2020-11-20T12:03:08.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f1380eef700 1 -- 172.21.15.44:0/2088163794 learned_addr learned my addr 172.21.15.44:0/2088163794 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f137b7fe700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 0x7f137c0ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.180+0000 7f137bfff700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa020 0x7f137c0ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1380eef700 1 -- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 msgr2=0x7f137c0ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1380eef700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0aa400 0x7f137c0ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1380eef700 1 -- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa020 msgr2=0x7f137c0ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1380eef700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa020 0x7f137c0ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1380eef700 1 -- 172.21.15.44:0/2088163794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0abdb0 con 0x7f137c13ac60 2020-11-20T12:03:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f137bfff700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa020 0x7f137c0ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1380eef700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c13ac60 0x7f137c14a480 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f137400cae0 tx=0x7f1374001320).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374005b80 con 0x7f137c13ac60 2020-11-20T12:03:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374005ce0 con 0x7f137c13ac60 2020-11-20T12:03:08.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c0ab870 con 0x7f137c13ac60 2020-11-20T12:03:08.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374007340 con 0x7f137c13ac60 2020-11-20T12:03:08.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.181+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c14ab30 con 0x7f137c13ac60 2020-11-20T12:03:08.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.182+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f137c149ea0 con 0x7f137c13ac60 2020-11-20T12:03:08.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.182+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1374010030 con 0x7f137c13ac60 2020-11-20T12:03:08.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f135bfff700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1354056d20 0x7f13540591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f137406fe80 con 0x7f137c13ac60 2020-11-20T12:03:08.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f135bfff700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6808/24928,v1:172.21.15.79:6809/24928] conn(0x7f135405e900 0x7f1354060d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 --> [v2:172.21.15.79:6808/24928,v1:172.21.15.79:6809/24928] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1354061380 con 0x7f135405e900 2020-11-20T12:03:08.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f137bfff700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1354056d20 0x7f13540591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13740724f0 con 0x7f137c13ac60 2020-11-20T12:03:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f137b7fe700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6808/24928,v1:172.21.15.79:6809/24928] conn(0x7f135405e900 0x7f1354060d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f137bfff700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1354056d20 0x7f13540591e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f136c00b150 tx=0x7f136c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.183+0000 7f137b7fe700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6808/24928,v1:172.21.15.79:6809/24928] conn(0x7f135405e900 0x7f1354060d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.184+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== osd.6 v2:172.21.15.79:6808/24928 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f1354061380 con 0x7f135405e900 2020-11-20T12:03:08.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.201+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 --> [v2:172.21.15.79:6808/24928,v1:172.21.15.79:6809/24928] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f137c060430 con 0x7f135405e900 2020-11-20T12:03:08.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f135bfff700 1 -- 172.21.15.44:0/2088163794 <== osd.6 v2:172.21.15.79:6808/24928 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f137c060430 con 0x7f135405e900 2020-11-20T12:03:08.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6808/24928,v1:172.21.15.79:6809/24928] conn(0x7f135405e900 msgr2=0x7f1354060d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6808/24928,v1:172.21.15.79:6809/24928] conn(0x7f135405e900 0x7f1354060d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1354056d20 msgr2=0x7f13540591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1354056d20 0x7f13540591e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f136c00b150 tx=0x7f136c015000).stop 2020-11-20T12:03:08.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c13ac60 msgr2=0x7f137c14a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 --2- 172.21.15.44:0/2088163794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c13ac60 0x7f137c14a480 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f137400cae0 tx=0x7f1374001320).stop 2020-11-20T12:03:08.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1380eef700 1 -- 172.21.15.44:0/2088163794 reap_dead start 2020-11-20T12:03:08.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 shutdown_connections 2020-11-20T12:03:08.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 >> 172.21.15.44:0/2088163794 conn(0x7f137c062690 msgr2=0x7f137c1347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 shutdown_connections 2020-11-20T12:03:08.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.202+0000 7f1382952700 1 -- 172.21.15.44:0/2088163794 wait complete. 2020-11-20T12:03:08.217 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:08.218 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2020-11-20T12:03:08.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.340+0000 7f4d6413f700 1 Processor -- start 2020-11-20T12:03:08.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.341+0000 7f4d6413f700 1 -- start start 2020-11-20T12:03:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.341+0000 7f4d6413f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4d5c0aa020 0x7f4d5c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.341+0000 7f4d6413f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d5c13ac60 0x7f4d5c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.341+0000 7f4d6413f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5c0a9770 con 0x7f4d5c0aa400 2020-11-20T12:03:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.341+0000 7f4d6413f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c0a56c0 con 0x7f4d5c0aa020 2020-11-20T12:03:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.341+0000 7f4d6413f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c0a6da0 con 0x7f4d5c13ac60 2020-11-20T12:03:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.341+0000 7f4d61edb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4d5c0aa020 0x7f4d5c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d61edb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4d5c0aa020 0x7f4d5c0a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36300/0 (socket says 172.21.15.44:36300) 2020-11-20T12:03:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d61edb700 1 -- 172.21.15.44:0/1608185384 learned_addr learned my addr 172.21.15.44:0/1608185384 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d626dc700 1 --2- 172.21.15.44:0/1608185384 >> v2:172.21.15.79:3300/0 conn(0x7f4d5c13ac60 0x7f4d5c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d61edb700 1 -- 172.21.15.44:0/1608185384 >> v2:172.21.15.79:3300/0 conn(0x7f4d5c13ac60 msgr2=0x7f4d5c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:08.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d61edb700 1 --2- 172.21.15.44:0/1608185384 >> v2:172.21.15.79:3300/0 conn(0x7f4d5c13ac60 0x7f4d5c0a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d61edb700 1 -- 172.21.15.44:0/1608185384 >> v1:172.21.15.79:6789/0 conn(0x7f4d5c0aa400 legacy=0x7f4d5c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d61edb700 1 -- 172.21.15.44:0/1608185384 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c0a7040 con 0x7f4d5c0aa020 2020-11-20T12:03:08.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d61edb700 1 --2- 172.21.15.44:0/1608185384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa020 0x7f4d5c0a9280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f4d5c0a8f70 tx=0x7f4d4c006eb0).ready entity=mon.0 client_cookie=43c6adc2425bd97 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d60ed9700 1 -- 172.21.15.44:0/1608185384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c009810 con 0x7f4d5c0aa020 2020-11-20T12:03:08.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d60ed9700 1 -- 172.21.15.44:0/1608185384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c009970 con 0x7f4d5c0aa020 2020-11-20T12:03:08.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.342+0000 7f4d60ed9700 1 -- 172.21.15.44:0/1608185384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c009cf0 con 0x7f4d5c0aa020 2020-11-20T12:03:08.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 -- 172.21.15.44:0/1608185384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa020 msgr2=0x7f4d5c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 --2- 172.21.15.44:0/1608185384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa020 0x7f4d5c0a9280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f4d5c0a8f70 tx=0x7f4d4c006eb0).stop 2020-11-20T12:03:08.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 -- 172.21.15.44:0/1608185384 shutdown_connections 2020-11-20T12:03:08.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 --2- 172.21.15.44:0/1608185384 >> v2:172.21.15.79:3300/0 conn(0x7f4d5c13ac60 0x7f4d5c0a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 --2- 172.21.15.44:0/1608185384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa020 0x7f4d5c0a9280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 -- 172.21.15.44:0/1608185384 >> 172.21.15.44:0/1608185384 conn(0x7f4d5c062690 msgr2=0x7f4d5c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 -- 172.21.15.44:0/1608185384 shutdown_connections 2020-11-20T12:03:08.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.343+0000 7f4d6413f700 1 -- 172.21.15.44:0/1608185384 wait complete. 2020-11-20T12:03:08.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 Processor -- start 2020-11-20T12:03:08.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 -- start start 2020-11-20T12:03:08.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d5c0aa020 0x7f4d5c0677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa400 0x7f4d5c09d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d5c13ac60 0x7f4d5c147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c14c420 con 0x7f4d5c0aa400 2020-11-20T12:03:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c0a32c0 con 0x7f4d5c0aa020 2020-11-20T12:03:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d6413f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c14ba40 con 0x7f4d5c13ac60 2020-11-20T12:03:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d616da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa400 0x7f4d5c09d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d616da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa400 0x7f4d5c09d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36310/0 (socket says 172.21.15.44:36310) 2020-11-20T12:03:08.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d616da700 1 -- 172.21.15.44:0/1440124047 learned_addr learned my addr 172.21.15.44:0/1440124047 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d61edb700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d5c0aa020 0x7f4d5c0677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.344+0000 7f4d626dc700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d5c13ac60 0x7f4d5c147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d616da700 1 -- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d5c13ac60 msgr2=0x7f4d5c147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d616da700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d5c13ac60 0x7f4d5c147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d616da700 1 -- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d5c0aa020 msgr2=0x7f4d5c0677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d616da700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d5c0aa020 0x7f4d5c0677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d616da700 1 -- 172.21.15.44:0/1440124047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c14bce0 con 0x7f4d5c0aa400 2020-11-20T12:03:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d616da700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa400 0x7f4d5c09d630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4d5800ca90 tx=0x7f4d58001460).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58006870 con 0x7f4d5c0aa400 2020-11-20T12:03:08.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d580069d0 con 0x7f4d5c0aa400 2020-11-20T12:03:08.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58006d50 con 0x7f4d5c0aa400 2020-11-20T12:03:08.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c14bf80 con 0x7f4d5c0aa400 2020-11-20T12:03:08.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.345+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c14b6f0 con 0x7f4d5c0aa400 2020-11-20T12:03:08.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.346+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d5c1370e0 con 0x7f4d5c0aa400 2020-11-20T12:03:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.346+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4d58018020 con 0x7f4d5c0aa400 2020-11-20T12:03:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.346+0000 7f4d49ffb700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4d34056d20 0x7f4d340591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.346+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4d5806f510 con 0x7f4d5c0aa400 2020-11-20T12:03:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.347+0000 7f4d49ffb700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6824/24960,v1:172.21.15.79:6825/24960] conn(0x7f4d3405e900 0x7f4d34060d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.347+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 --> [v2:172.21.15.79:6824/24960,v1:172.21.15.79:6825/24960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d34061380 con 0x7f4d3405e900 2020-11-20T12:03:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.347+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d58071b80 con 0x7f4d5c0aa400 2020-11-20T12:03:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.347+0000 7f4d61edb700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4d34056d20 0x7f4d340591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.347+0000 7f4d626dc700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6824/24960,v1:172.21.15.79:6825/24960] conn(0x7f4d3405e900 0x7f4d34060d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.347+0000 7f4d61edb700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4d34056d20 0x7f4d340591e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4d4c006ef0 tx=0x7f4d4c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.347+0000 7f4d626dc700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6824/24960,v1:172.21.15.79:6825/24960] conn(0x7f4d3405e900 0x7f4d34060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.348+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== osd.7 v2:172.21.15.79:6824/24960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f4d34061380 con 0x7f4d3405e900 2020-11-20T12:03:08.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.364+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 --> [v2:172.21.15.79:6824/24960,v1:172.21.15.79:6825/24960] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4d5c148770 con 0x7f4d3405e900 2020-11-20T12:03:08.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.365+0000 7f4d49ffb700 1 -- 172.21.15.44:0/1440124047 <== osd.7 v2:172.21.15.79:6824/24960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4d5c148770 con 0x7f4d3405e900 2020-11-20T12:03:08.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6824/24960,v1:172.21.15.79:6825/24960] conn(0x7f4d3405e900 msgr2=0x7f4d34060d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6824/24960,v1:172.21.15.79:6825/24960] conn(0x7f4d3405e900 0x7f4d34060d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4d34056d20 msgr2=0x7f4d340591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4d34056d20 0x7f4d340591e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4d4c006ef0 tx=0x7f4d4c015000).stop 2020-11-20T12:03:08.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa400 msgr2=0x7f4d5c09d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 --2- 172.21.15.44:0/1440124047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d5c0aa400 0x7f4d5c09d630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4d5800ca90 tx=0x7f4d58001460).stop 2020-11-20T12:03:08.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d626dc700 1 -- 172.21.15.44:0/1440124047 reap_dead start 2020-11-20T12:03:08.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 shutdown_connections 2020-11-20T12:03:08.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 >> 172.21.15.44:0/1440124047 conn(0x7f4d5c062690 msgr2=0x7f4d5c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 shutdown_connections 2020-11-20T12:03:08.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.366+0000 7f4d6413f700 1 -- 172.21.15.44:0/1440124047 wait complete. 2020-11-20T12:03:08.381 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:08.381 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-20T12:03:08.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.504+0000 7fea328d4700 1 Processor -- start 2020-11-20T12:03:08.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.504+0000 7fea328d4700 1 -- start start 2020-11-20T12:03:08.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.504+0000 7fea328d4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fea2c0aa020 0x7fea2c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.504+0000 7fea328d4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fea2c0aa400 0x7fea2c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.504+0000 7fea328d4700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c0a9770 con 0x7fea2c13ac60 2020-11-20T12:03:08.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.504+0000 7fea328d4700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c0a6da0 con 0x7fea2c0aa400 2020-11-20T12:03:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.504+0000 7fea328d4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c0a7040 con 0x7fea2c0aa020 2020-11-20T12:03:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fea2c0aa400 0x7fea2c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fea2c0aa400 0x7fea2c0a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36320/0 (socket says 172.21.15.44:36320) 2020-11-20T12:03:08.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea30e71700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fea2c13ac60 0x7fea2c0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40064/0 (socket says 172.21.15.44:40064) 2020-11-20T12:03:08.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 -- 172.21.15.44:0/3437726959 learned_addr learned my addr 172.21.15.44:0/3437726959 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2bfff700 1 --2- 172.21.15.44:0/3437726959 >> v2:172.21.15.79:3300/0 conn(0x7fea2c0aa020 0x7fea2c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 -- 172.21.15.44:0/3437726959 >> v2:172.21.15.79:3300/0 conn(0x7fea2c0aa020 msgr2=0x7fea2c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:08.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 --2- 172.21.15.44:0/3437726959 >> v2:172.21.15.79:3300/0 conn(0x7fea2c0aa020 0x7fea2c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 -- 172.21.15.44:0/3437726959 >> v1:172.21.15.44:6789/0 conn(0x7fea2c13ac60 legacy=0x7fea2c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2affd700 1 -- 172.21.15.44:0/3437726959 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51625739 0 0) 0x7fea24001450 con 0x7fea2c13ac60 2020-11-20T12:03:08.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 -- 172.21.15.44:0/3437726959 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0a6a90 con 0x7fea2c0aa400 2020-11-20T12:03:08.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2b7fe700 1 --2- 172.21.15.44:0/3437726959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c0aa400 0x7fea2c0a4fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fea2c0adec0 tx=0x7fea20006eb0).ready entity=mon.0 client_cookie=79d5263e4a30cec8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2affd700 1 -- 172.21.15.44:0/3437726959 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20009810 con 0x7fea2c0aa400 2020-11-20T12:03:08.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2affd700 1 -- 172.21.15.44:0/3437726959 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20009970 con 0x7fea2c0aa400 2020-11-20T12:03:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.505+0000 7fea2affd700 1 -- 172.21.15.44:0/3437726959 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20009cf0 con 0x7fea2c0aa400 2020-11-20T12:03:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 -- 172.21.15.44:0/3437726959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c0aa400 msgr2=0x7fea2c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 --2- 172.21.15.44:0/3437726959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c0aa400 0x7fea2c0a4fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fea2c0adec0 tx=0x7fea20006eb0).stop 2020-11-20T12:03:08.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 -- 172.21.15.44:0/3437726959 shutdown_connections 2020-11-20T12:03:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 --2- 172.21.15.44:0/3437726959 >> v2:172.21.15.79:3300/0 conn(0x7fea2c0aa020 0x7fea2c0a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 --2- 172.21.15.44:0/3437726959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c0aa400 0x7fea2c0a4fb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 -- 172.21.15.44:0/3437726959 >> 172.21.15.44:0/3437726959 conn(0x7fea2c062690 msgr2=0x7fea2c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 -- 172.21.15.44:0/3437726959 shutdown_connections 2020-11-20T12:03:08.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.506+0000 7fea328d4700 1 -- 172.21.15.44:0/3437726959 wait complete. 2020-11-20T12:03:08.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 Processor -- start 2020-11-20T12:03:08.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 -- start start 2020-11-20T12:03:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea2c0aa020 0x7fea2c0a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea2c0aa400 0x7fea2c09da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c13ac60 0x7fea2c134080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c09f5b0 con 0x7fea2c13ac60 2020-11-20T12:03:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0a29a0 con 0x7fea2c0aa020 2020-11-20T12:03:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.507+0000 7fea328d4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0a2c40 con 0x7fea2c0aa400 2020-11-20T12:03:08.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c13ac60 0x7fea2c134080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c13ac60 0x7fea2c134080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36326/0 (socket says 172.21.15.44:36326) 2020-11-20T12:03:08.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 -- 172.21.15.44:0/299932496 learned_addr learned my addr 172.21.15.44:0/299932496 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea2bfff700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea2c0aa020 0x7fea2c0a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea2b7fe700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea2c0aa400 0x7fea2c09da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 -- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea2c0aa400 msgr2=0x7fea2c09da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea2c0aa400 0x7fea2c09da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 -- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea2c0aa020 msgr2=0x7fea2c0a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea2c0aa020 0x7fea2c0a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 -- 172.21.15.44:0/299932496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0a30e0 con 0x7fea2c13ac60 2020-11-20T12:03:08.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea2bfff700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea2c0aa020 0x7fea2c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:08.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea30e71700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c13ac60 0x7fea2c134080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fea2400cae0 tx=0x7fea24007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea0bfff700 1 -- 172.21.15.44:0/299932496 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24005ac0 con 0x7fea2c13ac60 2020-11-20T12:03:08.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea0bfff700 1 -- 172.21.15.44:0/299932496 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea24005c20 con 0x7fea2c13ac60 2020-11-20T12:03:08.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea0bfff700 1 -- 172.21.15.44:0/299932496 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24007110 con 0x7fea2c13ac60 2020-11-20T12:03:08.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c0ad840 con 0x7fea2c13ac60 2020-11-20T12:03:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.508+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c09e0b0 con 0x7fea2c13ac60 2020-11-20T12:03:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.509+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2c0a9a20 con 0x7fea2c13ac60 2020-11-20T12:03:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.509+0000 7fea0bfff700 1 -- 172.21.15.44:0/299932496 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fea24010030 con 0x7fea2c13ac60 2020-11-20T12:03:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.510+0000 7fea0bfff700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea04056d20 0x7fea040591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.510+0000 7fea0bfff700 1 -- 172.21.15.44:0/299932496 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fea2406fd70 con 0x7fea2c13ac60 2020-11-20T12:03:08.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.510+0000 7fea2bfff700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea04056d20 0x7fea040591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.510+0000 7fea2bfff700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea04056d20 0x7fea040591e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fea1c006bd0 tx=0x7fea1c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.513+0000 7fea0bfff700 1 -- 172.21.15.44:0/299932496 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fea24045060 con 0x7fea2c13ac60 2020-11-20T12:03:08.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.688+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fea2c0a9770 con 0x7fea2c13ac60 2020-11-20T12:03:08.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.689+0000 7fea0bfff700 1 -- 172.21.15.44:0/299932496 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea2c0a9770 con 0x7fea2c13ac60 2020-11-20T12:03:08.693 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea04056d20 msgr2=0x7fea040591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea04056d20 0x7fea040591e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fea1c006bd0 tx=0x7fea1c008040).stop 2020-11-20T12:03:08.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c13ac60 msgr2=0x7fea2c134080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c13ac60 0x7fea2c134080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fea2400cae0 tx=0x7fea24007f90).stop 2020-11-20T12:03:08.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 shutdown_connections 2020-11-20T12:03:08.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea04056d20 0x7fea040591e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea2c13ac60 0x7fea2c134080 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea2c0aa020 0x7fea2c0a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 --2- 172.21.15.44:0/299932496 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea2c0aa400 0x7fea2c09da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.690+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 >> 172.21.15.44:0/299932496 conn(0x7fea2c062690 msgr2=0x7fea2c0a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.691+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 shutdown_connections 2020-11-20T12:03:08.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.691+0000 7fea328d4700 1 -- 172.21.15.44:0/299932496 wait complete. 2020-11-20T12:03:08.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2020-11-20T12:03:08.706 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2020-11-20T12:03:08.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.829+0000 7fc7fad9b700 1 Processor -- start 2020-11-20T12:03:08.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.829+0000 7fc7fad9b700 1 -- start start 2020-11-20T12:03:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7fad9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc7f40a3450 0x7fc7f40a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7fad9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc7f413eae0 0x7fc7f413a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7fad9b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f413aea0 con 0x7fc7f413b4a0 2020-11-20T12:03:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7fad9b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f40a3af0 con 0x7fc7f40a3450 2020-11-20T12:03:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7fad9b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f413e100 con 0x7fc7f413eae0 2020-11-20T12:03:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc7f40a3450 0x7fc7f40a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc7f40a3450 0x7fc7f40a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36334/0 (socket says 172.21.15.44:36334) 2020-11-20T12:03:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 -- 172.21.15.44:0/4030842611 learned_addr learned my addr 172.21.15.44:0/4030842611 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f9338700 1 --2- 172.21.15.44:0/4030842611 >> v2:172.21.15.79:3300/0 conn(0x7fc7f413eae0 0x7fc7f413a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 -- 172.21.15.44:0/4030842611 >> v2:172.21.15.79:3300/0 conn(0x7fc7f413eae0 msgr2=0x7fc7f413a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:08.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 --2- 172.21.15.44:0/4030842611 >> v2:172.21.15.79:3300/0 conn(0x7fc7f413eae0 0x7fc7f413a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 -- 172.21.15.44:0/4030842611 >> v1:172.21.15.79:6789/0 conn(0x7fc7f413b4a0 legacy=0x7fc7f40aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 -- 172.21.15.44:0/4030842611 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f413e3a0 con 0x7fc7f40a3450 2020-11-20T12:03:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f3fff700 1 --2- 172.21.15.44:0/4030842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a3450 0x7fc7f40a1880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fc7f40641e0 tx=0x7fc7e8006eb0).ready entity=mon.0 client_cookie=3b1d8cefb5aa4d50 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f37fe700 1 -- 172.21.15.44:0/4030842611 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8009890 con 0x7fc7f40a3450 2020-11-20T12:03:08.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f37fe700 1 -- 172.21.15.44:0/4030842611 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e80099f0 con 0x7fc7f40a3450 2020-11-20T12:03:08.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.830+0000 7fc7f37fe700 1 -- 172.21.15.44:0/4030842611 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8009d70 con 0x7fc7f40a3450 2020-11-20T12:03:08.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 -- 172.21.15.44:0/4030842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a3450 msgr2=0x7fc7f40a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/4030842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a3450 0x7fc7f40a1880 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fc7f40641e0 tx=0x7fc7e8006eb0).stop 2020-11-20T12:03:08.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 -- 172.21.15.44:0/4030842611 shutdown_connections 2020-11-20T12:03:08.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/4030842611 >> v2:172.21.15.79:3300/0 conn(0x7fc7f413eae0 0x7fc7f413a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/4030842611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a3450 0x7fc7f40a1880 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 -- 172.21.15.44:0/4030842611 >> 172.21.15.44:0/4030842611 conn(0x7fc7f4062690 msgr2=0x7fc7f40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 -- 172.21.15.44:0/4030842611 shutdown_connections 2020-11-20T12:03:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.831+0000 7fc7fad9b700 1 -- 172.21.15.44:0/4030842611 wait complete. 2020-11-20T12:03:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.832+0000 7fc7fad9b700 1 Processor -- start 2020-11-20T12:03:08.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.832+0000 7fc7fad9b700 1 -- start start 2020-11-20T12:03:08.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7f413b4a0 0x7fc7f4067b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc7f413eae0 0x7fc7f40ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a42d0 0x7fc7f4065f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40657f0 con 0x7fc7f40a42d0 2020-11-20T12:03:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4065b10 con 0x7fc7f413b4a0 2020-11-20T12:03:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f414a670 con 0x7fc7f413eae0 2020-11-20T12:03:08.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a42d0 0x7fc7f4065f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a42d0 0x7fc7f4065f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36340/0 (socket says 172.21.15.44:36340) 2020-11-20T12:03:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 -- 172.21.15.44:0/3677434391 learned_addr learned my addr 172.21.15.44:0/3677434391 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f3fff700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc7f413eae0 0x7fc7f40ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f8b37700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7f413b4a0 0x7fc7f4067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 -- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc7f413eae0 msgr2=0x7fc7f40ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc7f413eae0 0x7fc7f40ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 -- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7f413b4a0 msgr2=0x7fc7f4067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:08.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7f413b4a0 0x7fc7f4067b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 -- 172.21.15.44:0/3677434391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f4065550 con 0x7fc7f40a42d0 2020-11-20T12:03:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f9338700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a42d0 0x7fc7f4065f70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc7e400d7c0 tx=0x7fc7e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f0ff9700 1 -- 172.21.15.44:0/3677434391 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400f5e0 con 0x7fc7f40a42d0 2020-11-20T12:03:08.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f0ff9700 1 -- 172.21.15.44:0/3677434391 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e400f740 con 0x7fc7f40a42d0 2020-11-20T12:03:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f40651d0 con 0x7fc7f40a42d0 2020-11-20T12:03:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7f0ff9700 1 -- 172.21.15.44:0/3677434391 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400fae0 con 0x7fc7f40a42d0 2020-11-20T12:03:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.833+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f40666a0 con 0x7fc7f40a42d0 2020-11-20T12:03:08.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.834+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f4065c80 con 0x7fc7f40a42d0 2020-11-20T12:03:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.835+0000 7fc7f0ff9700 1 -- 172.21.15.44:0/3677434391 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc7e401c460 con 0x7fc7f40a42d0 2020-11-20T12:03:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.835+0000 7fc7f0ff9700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc7cc056d20 0x7fc7cc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.835+0000 7fc7f0ff9700 1 -- 172.21.15.44:0/3677434391 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc7e406ffc0 con 0x7fc7f40a42d0 2020-11-20T12:03:08.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.835+0000 7fc7f8b37700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc7cc056d20 0x7fc7cc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.835+0000 7fc7f8b37700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc7cc056d20 0x7fc7cc0591e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fc7ec003dc0 tx=0x7fc7ec007ef0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:08.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:08.838+0000 7fc7f0ff9700 1 -- 172.21.15.44:0/3677434391 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc7e4045250 con 0x7fc7f40a42d0 2020-11-20T12:03:09.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.013+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc7f4065360 con 0x7fc7f40a42d0 2020-11-20T12:03:09.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.014+0000 7fc7f0ff9700 1 -- 172.21.15.44:0/3677434391 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7e4048870 con 0x7fc7f40a42d0 2020-11-20T12:03:09.018 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:09.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc7cc056d20 msgr2=0x7fc7cc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc7cc056d20 0x7fc7cc0591e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fc7ec003dc0 tx=0x7fc7ec007ef0).stop 2020-11-20T12:03:09.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a42d0 msgr2=0x7fc7f4065f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a42d0 0x7fc7f4065f70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc7e400d7c0 tx=0x7fc7e4008040).stop 2020-11-20T12:03:09.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 shutdown_connections 2020-11-20T12:03:09.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc7cc056d20 0x7fc7cc0591e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7f40a42d0 0x7fc7f4065f70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7f413b4a0 0x7fc7f4067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 --2- 172.21.15.44:0/3677434391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc7f413eae0 0x7fc7f40ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.015+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 >> 172.21.15.44:0/3677434391 conn(0x7fc7f4062690 msgr2=0x7fc7f413bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:09.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.016+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 shutdown_connections 2020-11-20T12:03:09.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.016+0000 7fc7fad9b700 1 -- 172.21.15.44:0/3677434391 wait complete. 2020-11-20T12:03:09.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2020-11-20T12:03:09.031 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-20T12:03:09.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.154+0000 7f93ccbeb700 1 Processor -- start 2020-11-20T12:03:09.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.154+0000 7f93ccbeb700 1 -- start start 2020-11-20T12:03:09.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93ccbeb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93c8060330 0x7f93c80a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93ccbeb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c80a4c30 0x7f93c813a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93ccbeb700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c80a6260 con 0x7f93c813ae80 2020-11-20T12:03:09.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93ccbeb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c80a52e0 con 0x7f93c8060330 2020-11-20T12:03:09.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93ccbeb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80a6490 con 0x7f93c80a4c30 2020-11-20T12:03:09.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c6d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f93c813ae80 0x7f93c8136f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40096/0 (socket says 172.21.15.44:40096) 2020-11-20T12:03:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c6d9d700 1 -- 172.21.15.44:0/65547801 learned_addr learned my addr 172.21.15.44:0/65547801 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c659c700 1 --2- 172.21.15.44:0/65547801 >> v2:172.21.15.79:3300/0 conn(0x7f93c8060330 0x7f93c80a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c5d9b700 1 --2- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c80a4c30 0x7f93c813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c559a700 1 -- 172.21.15.44:0/65547801 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450231890 0 0) 0x7f93c80a6260 con 0x7f93c813ae80 2020-11-20T12:03:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c559a700 1 -- 172.21.15.44:0/65547801 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ac003240 con 0x7f93c813ae80 2020-11-20T12:03:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c659c700 1 -- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c80a4c30 msgr2=0x7f93c813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:09.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c659c700 1 --2- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c80a4c30 0x7f93c813a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c559a700 1 -- 172.21.15.44:0/65547801 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3576981598 0 0) 0x7f93ac003240 con 0x7f93c813ae80 2020-11-20T12:03:09.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c659c700 1 -- 172.21.15.44:0/65547801 >> v1:172.21.15.44:6789/0 conn(0x7f93c813ae80 legacy=0x7f93c8136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.155+0000 7f93c659c700 1 -- 172.21.15.44:0/65547801 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80a6730 con 0x7f93c8060330 2020-11-20T12:03:09.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93c659c700 1 --2- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c8060330 0x7f93c80a4740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f93bc00b9a0 tx=0x7f93bc006df0).ready entity=mon.1 client_cookie=3df53e5fe7cfea9a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93c559a700 1 -- 172.21.15.44:0/65547801 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc014070 con 0x7f93c8060330 2020-11-20T12:03:09.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93c559a700 1 -- 172.21.15.44:0/65547801 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc00e460 con 0x7f93c8060330 2020-11-20T12:03:09.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93c559a700 1 -- 172.21.15.44:0/65547801 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0001860 con 0x7f93c8060330 2020-11-20T12:03:09.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 -- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c8060330 msgr2=0x7f93c80a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c8060330 0x7f93c80a4740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f93bc00b9a0 tx=0x7f93bc006df0).stop 2020-11-20T12:03:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 -- 172.21.15.44:0/65547801 shutdown_connections 2020-11-20T12:03:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c8060330 0x7f93c80a4740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/65547801 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c80a4c30 0x7f93c813a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 -- 172.21.15.44:0/65547801 >> 172.21.15.44:0/65547801 conn(0x7f93c8062690 msgr2=0x7f93c80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:09.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 -- 172.21.15.44:0/65547801 shutdown_connections 2020-11-20T12:03:09.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.156+0000 7f93ccbeb700 1 -- 172.21.15.44:0/65547801 wait complete. 2020-11-20T12:03:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.157+0000 7f93ccbeb700 1 Processor -- start 2020-11-20T12:03:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.157+0000 7f93ccbeb700 1 -- start start 2020-11-20T12:03:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c8060330 0x7f93c80abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93c80a4c30 0x7f93c80ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c813ae80 0x7f93c8135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80a3140 con 0x7f93c80a4c30 2020-11-20T12:03:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8149ee0 con 0x7f93c813ae80 2020-11-20T12:03:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c81353e0 con 0x7f93c8060330 2020-11-20T12:03:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93c80a4c30 0x7f93c80ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93c80a4c30 0x7f93c80ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36352/0 (socket says 172.21.15.44:36352) 2020-11-20T12:03:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 -- 172.21.15.44:0/3158758895 learned_addr learned my addr 172.21.15.44:0/3158758895 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c6d9d700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c813ae80 0x7f93c8135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c659c700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c8060330 0x7f93c80abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 -- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c8060330 msgr2=0x7f93c80abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c8060330 0x7f93c80abea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 -- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c813ae80 msgr2=0x7f93c8135b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c813ae80 0x7f93c8135b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 -- 172.21.15.44:0/3158758895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8135680 con 0x7f93c80a4c30 2020-11-20T12:03:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93c5d9b700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93c80a4c30 0x7f93c80ac390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f93b800eba0 tx=0x7f93b8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93b67fc700 1 -- 172.21.15.44:0/3158758895 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800ca60 con 0x7f93c80a4c30 2020-11-20T12:03:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93b67fc700 1 -- 172.21.15.44:0/3158758895 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b800cbc0 con 0x7f93c80a4c30 2020-11-20T12:03:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93b67fc700 1 -- 172.21.15.44:0/3158758895 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b80123f0 con 0x7f93c80a4c30 2020-11-20T12:03:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c81372b0 con 0x7f93c80a4c30 2020-11-20T12:03:09.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.158+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c8136010 con 0x7f93c80a4c30 2020-11-20T12:03:09.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.159+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c8061650 con 0x7f93c80a4c30 2020-11-20T12:03:09.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.160+0000 7f93b67fc700 1 -- 172.21.15.44:0/3158758895 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f93b80127a0 con 0x7f93c80a4c30 2020-11-20T12:03:09.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.160+0000 7f93b67fc700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f939c056d20 0x7f939c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.160+0000 7f93b67fc700 1 -- 172.21.15.44:0/3158758895 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f93b80705b0 con 0x7f93c80a4c30 2020-11-20T12:03:09.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.160+0000 7f93c659c700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f939c056d20 0x7f939c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.160+0000 7f93c659c700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f939c056d20 0x7f939c0591e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f93bc00e370 tx=0x7f93bc014000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.163+0000 7f93b67fc700 1 -- 172.21.15.44:0/3158758895 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f93b80458a0 con 0x7f93c80a4c30 2020-11-20T12:03:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.338+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93c80446d0 con 0x7f93c80a4c30 2020-11-20T12:03:09.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.338+0000 7f93b67fc700 1 -- 172.21.15.44:0/3158758895 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93b8048ec0 con 0x7f93c80a4c30 2020-11-20T12:03:09.342 INFO:teuthology.orchestra.run.smithi044.stdout:55834574854 2020-11-20T12:03:09.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f939c056d20 msgr2=0x7f939c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f939c056d20 0x7f939c0591e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f93bc00e370 tx=0x7f93bc014000).stop 2020-11-20T12:03:09.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93c80a4c30 msgr2=0x7f93c80ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93c80a4c30 0x7f93c80ac390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f93b800eba0 tx=0x7f93b8006df0).stop 2020-11-20T12:03:09.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 shutdown_connections 2020-11-20T12:03:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f939c056d20 0x7f939c0591e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93c813ae80 0x7f93c8135b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93c8060330 0x7f93c80abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 --2- 172.21.15.44:0/3158758895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93c80a4c30 0x7f93c80ac390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 >> 172.21.15.44:0/3158758895 conn(0x7f93c8062690 msgr2=0x7f93c8060900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:09.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 shutdown_connections 2020-11-20T12:03:09.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.340+0000 7f93ccbeb700 1 -- 172.21.15.44:0/3158758895 wait complete. 2020-11-20T12:03:09.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2020-11-20T12:03:09.355 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2020-11-20T12:03:09.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.478+0000 7fe4bad80700 1 Processor -- start 2020-11-20T12:03:09.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.478+0000 7fe4bad80700 1 -- start start 2020-11-20T12:03:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.478+0000 7fe4bad80700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe4b4060bc0 0x7fe4b40a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4bad80700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b4134470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4bad80700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b40a9010 con 0x7fe4b4134960 2020-11-20T12:03:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4bad80700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b40a92a0 con 0x7fe4b4060bc0 2020-11-20T12:03:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4bad80700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40a9540 con 0x7fe4b40a6a70 2020-11-20T12:03:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b8b1c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe4b4060bc0 0x7fe4b40a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b931d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe4b4134960 0x7fe4b40a46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40108/0 (socket says 172.21.15.44:40108) 2020-11-20T12:03:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b931d700 1 -- 172.21.15.44:0/3439421434 learned_addr learned my addr 172.21.15.44:0/3439421434 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b3fff700 1 --2- 172.21.15.44:0/3439421434 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b4134470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b8b1c700 1 -- 172.21.15.44:0/3439421434 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 msgr2=0x7fe4b4134470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b8b1c700 1 --2- 172.21.15.44:0/3439421434 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b4134470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b8b1c700 1 -- 172.21.15.44:0/3439421434 >> v1:172.21.15.44:6789/0 conn(0x7fe4b4134960 legacy=0x7fe4b40a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b8b1c700 1 -- 172.21.15.44:0/3439421434 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4060fa0 con 0x7fe4b4060bc0 2020-11-20T12:03:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b37fe700 1 -- 172.21.15.44:0/3439421434 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910811155 0 0) 0x7fe4b40a9010 con 0x7fe4b4134960 2020-11-20T12:03:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b3fff700 1 --2- 172.21.15.44:0/3439421434 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b4134470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b8b1c700 1 --2- 172.21.15.44:0/3439421434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4060bc0 0x7fe4b40a8b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fe49c00cb30 tx=0x7fe49c006e80).ready entity=mon.0 client_cookie=9701e08a9c742df9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b37fe700 1 -- 172.21.15.44:0/3439421434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c014070 con 0x7fe4b4060bc0 2020-11-20T12:03:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b37fe700 1 -- 172.21.15.44:0/3439421434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c00a840 con 0x7fe4b4060bc0 2020-11-20T12:03:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.479+0000 7fe4b37fe700 1 -- 172.21.15.44:0/3439421434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00abc0 con 0x7fe4b4060bc0 2020-11-20T12:03:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 -- 172.21.15.44:0/3439421434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4060bc0 msgr2=0x7fe4b40a8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 --2- 172.21.15.44:0/3439421434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4060bc0 0x7fe4b40a8b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fe49c00cb30 tx=0x7fe49c006e80).stop 2020-11-20T12:03:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 -- 172.21.15.44:0/3439421434 shutdown_connections 2020-11-20T12:03:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 --2- 172.21.15.44:0/3439421434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4060bc0 0x7fe4b40a8b20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 --2- 172.21.15.44:0/3439421434 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b4134470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 -- 172.21.15.44:0/3439421434 >> 172.21.15.44:0/3439421434 conn(0x7fe4b40aceb0 msgr2=0x7fe4b40ad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 -- 172.21.15.44:0/3439421434 shutdown_connections 2020-11-20T12:03:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.480+0000 7fe4bad80700 1 -- 172.21.15.44:0/3439421434 wait complete. 2020-11-20T12:03:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 Processor -- start 2020-11-20T12:03:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 -- start start 2020-11-20T12:03:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b4060bc0 0x7fe4b41335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b41462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4134960 0x7fe4b4149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4067f70 con 0x7fe4b4134960 2020-11-20T12:03:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4146ae0 con 0x7fe4b4060bc0 2020-11-20T12:03:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.481+0000 7fe4bad80700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4146d80 con 0x7fe4b40a6a70 2020-11-20T12:03:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4134960 0x7fe4b4149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4134960 0x7fe4b4149980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36368/0 (socket says 172.21.15.44:36368) 2020-11-20T12:03:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 -- 172.21.15.44:0/1674289758 learned_addr learned my addr 172.21.15.44:0/1674289758 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b3fff700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b41462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b8b1c700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b4060bc0 0x7fe4b41335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 -- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 msgr2=0x7fe4b41462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b41462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 -- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b4060bc0 msgr2=0x7fe4b41335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b4060bc0 0x7fe4b41335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 -- 172.21.15.44:0/1674289758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4147020 con 0x7fe4b4134960 2020-11-20T12:03:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b3fff700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b41462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b8b1c700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b4060bc0 0x7fe4b41335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b931d700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4134960 0x7fe4b4149980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe4ac00cae0 tx=0x7fe4ac001320).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b0ff9700 1 -- 172.21.15.44:0/1674289758 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac005b80 con 0x7fe4b4134960 2020-11-20T12:03:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b0ff9700 1 -- 172.21.15.44:0/1674289758 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac005ce0 con 0x7fe4b4134960 2020-11-20T12:03:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4b0ff9700 1 -- 172.21.15.44:0/1674289758 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac007340 con 0x7fe4b4134960 2020-11-20T12:03:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b41467e0 con 0x7fe4b4134960 2020-11-20T12:03:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.482+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b40665a0 con 0x7fe4b4134960 2020-11-20T12:03:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.483+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b40a8e20 con 0x7fe4b4134960 2020-11-20T12:03:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.483+0000 7fe4b0ff9700 1 -- 172.21.15.44:0/1674289758 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe4ac010030 con 0x7fe4b4134960 2020-11-20T12:03:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.484+0000 7fe4b0ff9700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe490056d20 0x7fe4900591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.484+0000 7fe4b0ff9700 1 -- 172.21.15.44:0/1674289758 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe4ac070120 con 0x7fe4b4134960 2020-11-20T12:03:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.484+0000 7fe4b8b1c700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe490056d20 0x7fe4900591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.484+0000 7fe4b8b1c700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe490056d20 0x7fe4900591e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe49c00a750 tx=0x7fe49c006e80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.487+0000 7fe4b0ff9700 1 -- 172.21.15.44:0/1674289758 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe4ac045410 con 0x7fe4b4134960 2020-11-20T12:03:09.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.661+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe4b40a6630 con 0x7fe4b4134960 2020-11-20T12:03:09.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.661+0000 7fe4b0ff9700 1 -- 172.21.15.44:0/1674289758 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4ac048a30 con 0x7fe4b4134960 2020-11-20T12:03:09.666 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:09.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe490056d20 msgr2=0x7fe4900591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe490056d20 0x7fe4900591e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fe49c00a750 tx=0x7fe49c006e80).stop 2020-11-20T12:03:09.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4134960 msgr2=0x7fe4b4149980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4134960 0x7fe4b4149980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe4ac00cae0 tx=0x7fe4ac001320).stop 2020-11-20T12:03:09.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 shutdown_connections 2020-11-20T12:03:09.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe490056d20 0x7fe4900591e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4b4134960 0x7fe4b4149980 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b4060bc0 0x7fe4b41335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 --2- 172.21.15.44:0/1674289758 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe4b40a6a70 0x7fe4b41462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 >> 172.21.15.44:0/1674289758 conn(0x7fe4b40aceb0 msgr2=0x7fe4b4060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 shutdown_connections 2020-11-20T12:03:09.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.663+0000 7fe4bad80700 1 -- 172.21.15.44:0/1674289758 wait complete. 2020-11-20T12:03:09.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2020-11-20T12:03:09.679 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2020-11-20T12:03:09.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.802+0000 7f2658a28700 1 Processor -- start 2020-11-20T12:03:09.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.802+0000 7f2658a28700 1 -- start start 2020-11-20T12:03:09.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f2658a28700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f26540aa020 0x7f26540a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f2658a28700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26540aa400 0x7f26540a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f2658a28700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26540a9770 con 0x7f265413ac60 2020-11-20T12:03:09.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f2658a28700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26540a6da0 con 0x7f26540aa020 2020-11-20T12:03:09.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f2658a28700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26540a7040 con 0x7f26540aa400 2020-11-20T12:03:09.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f26540aa020 0x7f26540a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f26540aa020 0x7f26540a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36372/0 (socket says 172.21.15.44:36372) 2020-11-20T12:03:09.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 -- 172.21.15.44:0/2127196897 learned_addr learned my addr 172.21.15.44:0/2127196897 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:09.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f2651d9b700 1 --2- 172.21.15.44:0/2127196897 >> v2:172.21.15.79:3300/0 conn(0x7f26540aa400 0x7f26540a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 -- 172.21.15.44:0/2127196897 >> v2:172.21.15.79:3300/0 conn(0x7f26540aa400 msgr2=0x7f26540a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:09.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 --2- 172.21.15.44:0/2127196897 >> v2:172.21.15.79:3300/0 conn(0x7f26540aa400 0x7f26540a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 -- 172.21.15.44:0/2127196897 >> v1:172.21.15.79:6789/0 conn(0x7f265413ac60 legacy=0x7f26540a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:09.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 -- 172.21.15.44:0/2127196897 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540a6a90 con 0x7f26540aa020 2020-11-20T12:03:09.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f2651d9b700 1 --2- 172.21.15.44:0/2127196897 >> v2:172.21.15.79:3300/0 conn(0x7f26540aa400 0x7f26540a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:09.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.803+0000 7f265259c700 1 --2- 172.21.15.44:0/2127196897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa020 0x7f26540a9280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f26540a4990 tx=0x7f2648006eb0).ready entity=mon.0 client_cookie=534e5cadae14eeb3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f265159a700 1 -- 172.21.15.44:0/2127196897 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648009890 con 0x7f26540aa020 2020-11-20T12:03:09.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f265159a700 1 -- 172.21.15.44:0/2127196897 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26480099f0 con 0x7f26540aa020 2020-11-20T12:03:09.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f265159a700 1 -- 172.21.15.44:0/2127196897 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648009d70 con 0x7f26540aa020 2020-11-20T12:03:09.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 -- 172.21.15.44:0/2127196897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa020 msgr2=0x7f26540a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 --2- 172.21.15.44:0/2127196897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa020 0x7f26540a9280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f26540a4990 tx=0x7f2648006eb0).stop 2020-11-20T12:03:09.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 -- 172.21.15.44:0/2127196897 shutdown_connections 2020-11-20T12:03:09.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 --2- 172.21.15.44:0/2127196897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa020 0x7f26540a9280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 --2- 172.21.15.44:0/2127196897 >> v2:172.21.15.79:3300/0 conn(0x7f26540aa400 0x7f26540a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 -- 172.21.15.44:0/2127196897 >> 172.21.15.44:0/2127196897 conn(0x7f2654062690 msgr2=0x7f26540a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 -- 172.21.15.44:0/2127196897 shutdown_connections 2020-11-20T12:03:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.804+0000 7f2658a28700 1 -- 172.21.15.44:0/2127196897 wait complete. 2020-11-20T12:03:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.805+0000 7f2658a28700 1 Processor -- start 2020-11-20T12:03:09.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.805+0000 7f2658a28700 1 -- start start 2020-11-20T12:03:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26540aa020 0x7f2654116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa400 0x7f26540ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265413ac60 0x7f26540acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265414d510 con 0x7f26540aa400 2020-11-20T12:03:09.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265414d770 con 0x7f265413ac60 2020-11-20T12:03:09.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265414da10 con 0x7f26540aa020 2020-11-20T12:03:09.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa400 0x7f26540ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa400 0x7f26540ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36378/0 (socket says 172.21.15.44:36378) 2020-11-20T12:03:09.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 -- 172.21.15.44:0/421611120 learned_addr learned my addr 172.21.15.44:0/421611120 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f265259c700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26540aa020 0x7f2654116440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2652d9d700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265413ac60 0x7f26540acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 -- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26540aa020 msgr2=0x7f2654116440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26540aa020 0x7f2654116440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 -- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265413ac60 msgr2=0x7f26540acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265413ac60 0x7f26540acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 -- 172.21.15.44:0/421611120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265414dcb0 con 0x7f26540aa400 2020-11-20T12:03:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2652d9d700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265413ac60 0x7f26540acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2651d9b700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa400 0x7f26540ac8c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f264400cc80 tx=0x7f2644008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f26427fc700 1 -- 172.21.15.44:0/421611120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400a9e0 con 0x7f26540aa400 2020-11-20T12:03:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f26427fc700 1 -- 172.21.15.44:0/421611120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264400ab40 con 0x7f26540aa400 2020-11-20T12:03:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2654114fd0 con 0x7f26540aa400 2020-11-20T12:03:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f26427fc700 1 -- 172.21.15.44:0/421611120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644011410 con 0x7f26540aa400 2020-11-20T12:03:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.806+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2654116930 con 0x7f26540aa400 2020-11-20T12:03:09.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.807+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265414de40 con 0x7f26540aa400 2020-11-20T12:03:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.807+0000 7f26427fc700 1 -- 172.21.15.44:0/421611120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2644011800 con 0x7f26540aa400 2020-11-20T12:03:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.808+0000 7f26427fc700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2628056d20 0x7f26280591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.808+0000 7f26427fc700 1 -- 172.21.15.44:0/421611120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2644070600 con 0x7f26540aa400 2020-11-20T12:03:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.808+0000 7f265259c700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2628056d20 0x7f26280591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:09.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.808+0000 7f265259c700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2628056d20 0x7f26280591e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f2648006ef0 tx=0x7f2648015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:09.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.811+0000 7f26427fc700 1 -- 172.21.15.44:0/421611120 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f26440458f0 con 0x7f26540aa400 2020-11-20T12:03:09.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.984+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2654061650 con 0x7f26540aa400 2020-11-20T12:03:09.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.984+0000 7f26427fc700 1 -- 172.21.15.44:0/421611120 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2644048f10 con 0x7f26540aa400 2020-11-20T12:03:09.989 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:09.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2628056d20 msgr2=0x7f26280591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2628056d20 0x7f26280591e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f2648006ef0 tx=0x7f2648015000).stop 2020-11-20T12:03:09.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa400 msgr2=0x7f26540ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:09.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa400 0x7f26540ac8c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f264400cc80 tx=0x7f2644008040).stop 2020-11-20T12:03:09.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 shutdown_connections 2020-11-20T12:03:09.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2628056d20 0x7f26280591e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f265413ac60 0x7f26540acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26540aa020 0x7f2654116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 --2- 172.21.15.44:0/421611120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26540aa400 0x7f26540ac8c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 >> 172.21.15.44:0/421611120 conn(0x7f2654062690 msgr2=0x7f2654135fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 shutdown_connections 2020-11-20T12:03:09.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:09.986+0000 7f2658a28700 1 -- 172.21.15.44:0/421611120 wait complete. 2020-11-20T12:03:10.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2020-11-20T12:03:10.002 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2020-11-20T12:03:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.125+0000 7fb5f8876700 1 Processor -- start 2020-11-20T12:03:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.125+0000 7fb5f8876700 1 -- start start 2020-11-20T12:03:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.125+0000 7fb5f8876700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb5f40aa400 0x7fb5f40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.125+0000 7fb5f8876700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f413ac60 0x7fb5f40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.125+0000 7fb5f8876700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40a9770 con 0x7fb5f40aa020 2020-11-20T12:03:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.125+0000 7fb5f8876700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f40a56c0 con 0x7fb5f40aa400 2020-11-20T12:03:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.125+0000 7fb5f8876700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40a6da0 con 0x7fb5f413ac60 2020-11-20T12:03:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f259c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb5f40aa020 0x7fb5f40a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40132/0 (socket says 172.21.15.44:40132) 2020-11-20T12:03:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f259c700 1 -- 172.21.15.44:0/3974934136 learned_addr learned my addr 172.21.15.44:0/3974934136 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f2d9d700 1 --2- 172.21.15.44:0/3974934136 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f413ac60 0x7fb5f40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f1d9b700 1 --2- 172.21.15.44:0/3974934136 >> v2:172.21.15.79:3300/0 conn(0x7fb5f40aa400 0x7fb5f40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f159a700 1 -- 172.21.15.44:0/3974934136 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918277701 0 0) 0x7fb5ec001450 con 0x7fb5f40aa020 2020-11-20T12:03:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f159a700 1 -- 172.21.15.44:0/3974934136 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d4003240 con 0x7fb5f40aa020 2020-11-20T12:03:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f2d9d700 1 -- 172.21.15.44:0/3974934136 >> v2:172.21.15.79:3300/0 conn(0x7fb5f40aa400 msgr2=0x7fb5f40a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f2d9d700 1 --2- 172.21.15.44:0/3974934136 >> v2:172.21.15.79:3300/0 conn(0x7fb5f40aa400 0x7fb5f40a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f2d9d700 1 -- 172.21.15.44:0/3974934136 >> v1:172.21.15.44:6789/0 conn(0x7fb5f40aa020 legacy=0x7fb5f40a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f2d9d700 1 -- 172.21.15.44:0/3974934136 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40a7040 con 0x7fb5f413ac60 2020-11-20T12:03:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f1d9b700 1 --2- 172.21.15.44:0/3974934136 >> v2:172.21.15.79:3300/0 conn(0x7fb5f40aa400 0x7fb5f40a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f2d9d700 1 --2- 172.21.15.44:0/3974934136 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f413ac60 0x7fb5f40a3700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb5e400b9a0 tx=0x7fb5e4006df0).ready entity=mon.2 client_cookie=589b8ab3b4f19585 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f159a700 1 -- 172.21.15.44:0/3974934136 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4009680 con 0x7fb5f413ac60 2020-11-20T12:03:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f159a700 1 -- 172.21.15.44:0/3974934136 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e40097e0 con 0x7fb5f413ac60 2020-11-20T12:03:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.126+0000 7fb5f159a700 1 -- 172.21.15.44:0/3974934136 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4009b60 con 0x7fb5f413ac60 2020-11-20T12:03:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 -- 172.21.15.44:0/3974934136 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f413ac60 msgr2=0x7fb5f40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 --2- 172.21.15.44:0/3974934136 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f413ac60 0x7fb5f40a3700 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb5e400b9a0 tx=0x7fb5e4006df0).stop 2020-11-20T12:03:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 -- 172.21.15.44:0/3974934136 shutdown_connections 2020-11-20T12:03:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 --2- 172.21.15.44:0/3974934136 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f413ac60 0x7fb5f40a3700 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 --2- 172.21.15.44:0/3974934136 >> v2:172.21.15.79:3300/0 conn(0x7fb5f40aa400 0x7fb5f40a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 -- 172.21.15.44:0/3974934136 >> 172.21.15.44:0/3974934136 conn(0x7fb5f4062690 msgr2=0x7fb5f40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 -- 172.21.15.44:0/3974934136 shutdown_connections 2020-11-20T12:03:10.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.127+0000 7fb5f8876700 1 -- 172.21.15.44:0/3974934136 wait complete. 2020-11-20T12:03:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.128+0000 7fb5f8876700 1 Processor -- start 2020-11-20T12:03:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.128+0000 7fb5f8876700 1 -- start start 2020-11-20T12:03:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.128+0000 7fb5f8876700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f40aa020 0x7fb5f4147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.128+0000 7fb5f8876700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40aa400 0x7fb5f40ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.128+0000 7fb5f8876700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5f413ac60 0x7fb5f40ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.128+0000 7fb5f8876700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4138a90 con 0x7fb5f40aa400 2020-11-20T12:03:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.128+0000 7fb5f8876700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40acb30 con 0x7fb5f413ac60 2020-11-20T12:03:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f8876700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4146bc0 con 0x7fb5f40aa020 2020-11-20T12:03:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40aa400 0x7fb5f40ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40aa400 0x7fb5f40ab470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36390/0 (socket says 172.21.15.44:36390) 2020-11-20T12:03:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 -- 172.21.15.44:0/2581782564 learned_addr learned my addr 172.21.15.44:0/2581782564 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f259c700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f40aa020 0x7fb5f4147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f2d9d700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5f413ac60 0x7fb5f40ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 -- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f40aa020 msgr2=0x7fb5f4147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f40aa020 0x7fb5f4147300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 -- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5f413ac60 msgr2=0x7fb5f40ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5f413ac60 0x7fb5f40ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 -- 172.21.15.44:0/2581782564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f413a510 con 0x7fb5f40aa400 2020-11-20T12:03:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f259c700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f40aa020 0x7fb5f4147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f1d9b700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40aa400 0x7fb5f40ab470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb5dc00ef50 tx=0x7fb5dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5ea7fc700 1 -- 172.21.15.44:0/2581782564 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00cbf0 con 0x7fb5f40aa400 2020-11-20T12:03:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5ea7fc700 1 -- 172.21.15.44:0/2581782564 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc00cd50 con 0x7fb5f40aa400 2020-11-20T12:03:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f40ac620 con 0x7fb5f40aa400 2020-11-20T12:03:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5ea7fc700 1 -- 172.21.15.44:0/2581782564 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc012600 con 0x7fb5f40aa400 2020-11-20T12:03:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.129+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f41479b0 con 0x7fb5f40aa400 2020-11-20T12:03:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.130+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f40ac970 con 0x7fb5f40aa400 2020-11-20T12:03:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.130+0000 7fb5ea7fc700 1 -- 172.21.15.44:0/2581782564 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb5dc012970 con 0x7fb5f40aa400 2020-11-20T12:03:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.131+0000 7fb5ea7fc700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5d0056d20 0x7fb5d00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.131+0000 7fb5ea7fc700 1 -- 172.21.15.44:0/2581782564 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb5dc070830 con 0x7fb5f40aa400 2020-11-20T12:03:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.131+0000 7fb5f259c700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5d0056d20 0x7fb5d00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.131+0000 7fb5f259c700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5d0056d20 0x7fb5d00591e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb5ec007fd0 tx=0x7fb5ec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.134+0000 7fb5ea7fc700 1 -- 172.21.15.44:0/2581782564 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb5dc045b20 con 0x7fb5f40aa400 2020-11-20T12:03:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.310+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb5f4061650 con 0x7fb5f40aa400 2020-11-20T12:03:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.311+0000 7fb5ea7fc700 1 -- 172.21.15.44:0/2581782564 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb5dc049140 con 0x7fb5f40aa400 2020-11-20T12:03:10.315 INFO:teuthology.orchestra.run.smithi044.stdout:60129542150 2020-11-20T12:03:10.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5d0056d20 msgr2=0x7fb5d00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5d0056d20 0x7fb5d00591e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb5ec007fd0 tx=0x7fb5ec009040).stop 2020-11-20T12:03:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40aa400 msgr2=0x7fb5f40ab470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40aa400 0x7fb5f40ab470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fb5dc00ef50 tx=0x7fb5dc006df0).stop 2020-11-20T12:03:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 shutdown_connections 2020-11-20T12:03:10.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5d0056d20 0x7fb5d00591e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5f413ac60 0x7fb5f40ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb5f40aa020 0x7fb5f4147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 --2- 172.21.15.44:0/2581782564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f40aa400 0x7fb5f40ab470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 >> 172.21.15.44:0/2581782564 conn(0x7fb5f4062690 msgr2=0x7fb5f40a3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 shutdown_connections 2020-11-20T12:03:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.313+0000 7fb5f8876700 1 -- 172.21.15.44:0/2581782564 wait complete. 2020-11-20T12:03:10.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2020-11-20T12:03:10.328 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2020-11-20T12:03:10.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.486+0000 7f1f9098e700 1 Processor -- start 2020-11-20T12:03:10.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.486+0000 7f1f9098e700 1 -- start start 2020-11-20T12:03:10.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f9098e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1f8c0a3e10 0x7f1f8c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f9098e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c0a9770 con 0x7f1f8c0aa400 2020-11-20T12:03:10.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f9098e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c0a6da0 con 0x7f1f8c0aa020 2020-11-20T12:03:10.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f9098e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c0a56c0 con 0x7f1f8c0a3e10 2020-11-20T12:03:10.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f89d9b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1f8c0aa400 0x7f1f8c0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40148/0 (socket says 172.21.15.44:40148) 2020-11-20T12:03:10.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f89d9b700 1 -- 172.21.15.44:0/2000778546 learned_addr learned my addr 172.21.15.44:0/2000778546 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:10.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8ad9d700 1 --2- 172.21.15.44:0/2000778546 >> v2:172.21.15.79:3300/0 conn(0x7f1f8c0a3e10 0x7f1f8c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8959a700 1 -- 172.21.15.44:0/2000778546 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328316886 0 0) 0x7f1f740010d0 con 0x7f1f8c0aa400 2020-11-20T12:03:10.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8959a700 1 -- 172.21.15.44:0/2000778546 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f6c003240 con 0x7f1f8c0aa400 2020-11-20T12:03:10.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8ad9d700 1 -- 172.21.15.44:0/2000778546 >> v1:172.21.15.79:6789/0 conn(0x7f1f8c0aa020 legacy=0x7f1f8c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8ad9d700 1 -- 172.21.15.44:0/2000778546 >> v1:172.21.15.44:6789/0 conn(0x7f1f8c0aa400 legacy=0x7f1f8c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8ad9d700 1 -- 172.21.15.44:0/2000778546 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0a7040 con 0x7f1f8c0a3e10 2020-11-20T12:03:10.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8ad9d700 1 --2- 172.21.15.44:0/2000778546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0a3e10 0x7f1f8c0a9280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1f7c00b9a0 tx=0x7f1f7c006df0).ready entity=mon.1 client_cookie=90e1634f36fa81fd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8959a700 1 -- 172.21.15.44:0/2000778546 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0097b0 con 0x7f1f8c0a3e10 2020-11-20T12:03:10.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8959a700 1 -- 172.21.15.44:0/2000778546 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c009910 con 0x7f1f8c0a3e10 2020-11-20T12:03:10.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.487+0000 7f1f8959a700 1 -- 172.21.15.44:0/2000778546 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c009c90 con 0x7f1f8c0a3e10 2020-11-20T12:03:10.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.488+0000 7f1f9098e700 1 -- 172.21.15.44:0/2000778546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0a3e10 msgr2=0x7f1f8c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.488+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2000778546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0a3e10 0x7f1f8c0a9280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1f7c00b9a0 tx=0x7f1f7c006df0).stop 2020-11-20T12:03:10.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.488+0000 7f1f9098e700 1 -- 172.21.15.44:0/2000778546 shutdown_connections 2020-11-20T12:03:10.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.488+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2000778546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0a3e10 0x7f1f8c0a9280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.488+0000 7f1f9098e700 1 -- 172.21.15.44:0/2000778546 >> 172.21.15.44:0/2000778546 conn(0x7f1f8c062690 msgr2=0x7f1f8c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:10.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.488+0000 7f1f9098e700 1 -- 172.21.15.44:0/2000778546 shutdown_connections 2020-11-20T12:03:10.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.488+0000 7f1f9098e700 1 -- 172.21.15.44:0/2000778546 wait complete. 2020-11-20T12:03:10.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 Processor -- start 2020-11-20T12:03:10.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 -- start start 2020-11-20T12:03:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f8c0a3e10 0x7f1f8c067240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0aa020 0x7f1f8c065f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c0aa400 0x7f1f8c066450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c065570 con 0x7f1f8c0aa400 2020-11-20T12:03:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c146920 con 0x7f1f8c0aa020 2020-11-20T12:03:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.489+0000 7f1f9098e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c066d10 con 0x7f1f8c0a3e10 2020-11-20T12:03:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c0aa400 0x7f1f8c066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c0aa400 0x7f1f8c066450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36406/0 (socket says 172.21.15.44:36406) 2020-11-20T12:03:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 -- 172.21.15.44:0/2788044439 learned_addr learned my addr 172.21.15.44:0/2788044439 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8a59c700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f8c0a3e10 0x7f1f8c067240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f89d9b700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0aa020 0x7f1f8c065f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 -- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f8c0a3e10 msgr2=0x7f1f8c067240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f8c0a3e10 0x7f1f8c067240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 -- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0aa020 msgr2=0x7f1f8c065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0aa020 0x7f1f8c065f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 -- 172.21.15.44:0/2788044439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c146b10 con 0x7f1f8c0aa400 2020-11-20T12:03:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f8ad9d700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c0aa400 0x7f1f8c066450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f1f7c006e30 tx=0x7f1f7c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f827fc700 1 -- 172.21.15.44:0/2788044439 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c009e40 con 0x7f1f8c0aa400 2020-11-20T12:03:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f827fc700 1 -- 172.21.15.44:0/2788044439 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c01d5d0 con 0x7f1f8c0aa400 2020-11-20T12:03:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c066a70 con 0x7f1f8c0aa400 2020-11-20T12:03:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f827fc700 1 -- 172.21.15.44:0/2788044439 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c01d970 con 0x7f1f8c0aa400 2020-11-20T12:03:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.490+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c1471b0 con 0x7f1f8c0aa400 2020-11-20T12:03:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.491+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c061650 con 0x7f1f8c0aa400 2020-11-20T12:03:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.491+0000 7f1f827fc700 1 -- 172.21.15.44:0/2788044439 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1f7c01a040 con 0x7f1f8c0aa400 2020-11-20T12:03:10.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.492+0000 7f1f827fc700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f60056d50 0x7f1f60059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.492+0000 7f1f827fc700 1 -- 172.21.15.44:0/2788044439 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1f7c071770 con 0x7f1f8c0aa400 2020-11-20T12:03:10.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.492+0000 7f1f8a59c700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f60056d50 0x7f1f60059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.492+0000 7f1f8a59c700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f60056d50 0x7f1f60059210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f1f84003c00 tx=0x7f1f84009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.495+0000 7f1f827fc700 1 -- 172.21.15.44:0/2788044439 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1f7c046a60 con 0x7f1f8c0aa400 2020-11-20T12:03:10.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.671+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1f8c0a6a90 con 0x7f1f8c0aa400 2020-11-20T12:03:10.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.671+0000 7f1f827fc700 1 -- 172.21.15.44:0/2788044439 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f7c024020 con 0x7f1f8c0aa400 2020-11-20T12:03:10.675 INFO:teuthology.orchestra.run.smithi044.stdout:60129542151 2020-11-20T12:03:10.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.672+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f60056d50 msgr2=0x7f1f60059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.672+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f60056d50 0x7f1f60059210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f1f84003c00 tx=0x7f1f84009040).stop 2020-11-20T12:03:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.672+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c0aa400 msgr2=0x7f1f8c066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.672+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c0aa400 0x7f1f8c066450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f1f7c006e30 tx=0x7f1f7c015000).stop 2020-11-20T12:03:10.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 shutdown_connections 2020-11-20T12:03:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f60056d50 0x7f1f60059210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f8c0aa400 0x7f1f8c066450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f8c0a3e10 0x7f1f8c067240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 --2- 172.21.15.44:0/2788044439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c0aa020 0x7f1f8c065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 >> 172.21.15.44:0/2788044439 conn(0x7f1f8c062690 msgr2=0x7f1f8c139ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 shutdown_connections 2020-11-20T12:03:10.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.673+0000 7f1f9098e700 1 -- 172.21.15.44:0/2788044439 wait complete. 2020-11-20T12:03:10.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.6 2020-11-20T12:03:10.688 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2020-11-20T12:03:10.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.811+0000 7f2686d4d700 1 Processor -- start 2020-11-20T12:03:10.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.811+0000 7f2686d4d700 1 -- start start 2020-11-20T12:03:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2686d4d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800aa1e0 0x7f2680065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2686d4d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26800a7810 0x7f2680066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2686d4d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f268013c170 0x7f268013c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2686d4d700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2680066520 con 0x7f268013c170 2020-11-20T12:03:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2686d4d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26800a8bf0 con 0x7f26800a7810 2020-11-20T12:03:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2686d4d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26800a86b0 con 0x7f26800aa1e0 2020-11-20T12:03:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f268013c170 0x7f268013c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f268013c170 0x7f268013c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36412/0 (socket says 172.21.15.44:36412) 2020-11-20T12:03:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 -- 172.21.15.44:0/3737631210 learned_addr learned my addr 172.21.15.44:0/3737631210 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2684ae9700 1 --2- 172.21.15.44:0/3737631210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800aa1e0 0x7f2680065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 -- 172.21.15.44:0/3737631210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800aa1e0 msgr2=0x7f2680065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 --2- 172.21.15.44:0/3737631210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800aa1e0 0x7f2680065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 -- 172.21.15.44:0/3737631210 >> v2:172.21.15.79:3300/0 conn(0x7f26800a7810 msgr2=0x7f2680066030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f267ffff700 1 --2- 172.21.15.44:0/3737631210 >> v2:172.21.15.79:3300/0 conn(0x7f26800a7810 0x7f2680066030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 --2- 172.21.15.44:0/3737631210 >> v2:172.21.15.79:3300/0 conn(0x7f26800a7810 0x7f2680066030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 -- 172.21.15.44:0/3737631210 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26800a8950 con 0x7f268013c170 2020-11-20T12:03:10.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f2684ae9700 1 --2- 172.21.15.44:0/3737631210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800aa1e0 0x7f2680065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:10.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f267ffff700 1 --2- 172.21.15.44:0/3737631210 >> v2:172.21.15.79:3300/0 conn(0x7f26800a7810 0x7f2680066030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:10.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.812+0000 7f26852ea700 1 --2- 172.21.15.44:0/3737631210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268013c170 0x7f268013c5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f2680062b30 tx=0x7f2678009290).ready entity=mon.0 client_cookie=f776e4c55f730190 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f267f7fe700 1 -- 172.21.15.44:0/3737631210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267800bc40 con 0x7f268013c170 2020-11-20T12:03:10.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f267f7fe700 1 -- 172.21.15.44:0/3737631210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267800bda0 con 0x7f268013c170 2020-11-20T12:03:10.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f267f7fe700 1 -- 172.21.15.44:0/3737631210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678011610 con 0x7f268013c170 2020-11-20T12:03:10.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 -- 172.21.15.44:0/3737631210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268013c170 msgr2=0x7f268013c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 --2- 172.21.15.44:0/3737631210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268013c170 0x7f268013c5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f2680062b30 tx=0x7f2678009290).stop 2020-11-20T12:03:10.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 -- 172.21.15.44:0/3737631210 shutdown_connections 2020-11-20T12:03:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 --2- 172.21.15.44:0/3737631210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268013c170 0x7f268013c5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 --2- 172.21.15.44:0/3737631210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800aa1e0 0x7f2680065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 --2- 172.21.15.44:0/3737631210 >> v2:172.21.15.79:3300/0 conn(0x7f26800a7810 0x7f2680066030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 -- 172.21.15.44:0/3737631210 >> 172.21.15.44:0/3737631210 conn(0x7f2680062690 msgr2=0x7f26800a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 -- 172.21.15.44:0/3737631210 shutdown_connections 2020-11-20T12:03:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.813+0000 7f2686d4d700 1 -- 172.21.15.44:0/3737631210 wait complete. 2020-11-20T12:03:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.814+0000 7f2686d4d700 1 Processor -- start 2020-11-20T12:03:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.814+0000 7f2686d4d700 1 -- start start 2020-11-20T12:03:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800a7810 0x7f26801163d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26800aa1e0 0x7f26800a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f268013c170 0x7f268014ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800a3090 con 0x7f26800aa1e0 2020-11-20T12:03:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800a26b0 con 0x7f268013c170 2020-11-20T12:03:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26800a2950 con 0x7f26800a7810 2020-11-20T12:03:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26800aa1e0 0x7f26800a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26800aa1e0 0x7f26800a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36418/0 (socket says 172.21.15.44:36418) 2020-11-20T12:03:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 -- 172.21.15.44:0/1263958934 learned_addr learned my addr 172.21.15.44:0/1263958934 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2684ae9700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800a7810 0x7f26801163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f26852ea700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f268013c170 0x7f268014ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 -- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800a7810 msgr2=0x7f26801163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800a7810 0x7f26801163d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 -- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f268013c170 msgr2=0x7f268014ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f268013c170 0x7f268014ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 -- 172.21.15.44:0/1263958934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26800a2bf0 con 0x7f26800aa1e0 2020-11-20T12:03:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267ffff700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26800aa1e0 0x7f26800a3420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f266c00c920 tx=0x7f266c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267cff9700 1 -- 172.21.15.44:0/1263958934 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c00e5d0 con 0x7f26800aa1e0 2020-11-20T12:03:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267cff9700 1 -- 172.21.15.44:0/1263958934 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c00e730 con 0x7f26800aa1e0 2020-11-20T12:03:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26800a3910 con 0x7f26800aa1e0 2020-11-20T12:03:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f267cff9700 1 -- 172.21.15.44:0/1263958934 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c00eab0 con 0x7f26800aa1e0 2020-11-20T12:03:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.815+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268014b230 con 0x7f26800aa1e0 2020-11-20T12:03:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.816+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2680061650 con 0x7f26800aa1e0 2020-11-20T12:03:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.817+0000 7f267cff9700 1 -- 172.21.15.44:0/1263958934 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f266c011410 con 0x7f26800aa1e0 2020-11-20T12:03:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.817+0000 7f267cff9700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2658056d50 0x7f2658059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.817+0000 7f267cff9700 1 -- 172.21.15.44:0/1263958934 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f266c06ff00 con 0x7f26800aa1e0 2020-11-20T12:03:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.817+0000 7f2684ae9700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2658056d50 0x7f2658059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.818+0000 7f2684ae9700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2658056d50 0x7f2658059210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f2674006fd0 tx=0x7f2674008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.821+0000 7f267cff9700 1 -- 172.21.15.44:0/1263958934 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f266c045190 con 0x7f26800aa1e0 2020-11-20T12:03:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.994+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f26800446d0 con 0x7f26800aa1e0 2020-11-20T12:03:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.994+0000 7f267cff9700 1 -- 172.21.15.44:0/1263958934 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f266c0487b0 con 0x7f26800aa1e0 2020-11-20T12:03:10.998 INFO:teuthology.orchestra.run.smithi044.stdout:60129542151 2020-11-20T12:03:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2658056d50 msgr2=0x7f2658059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2658056d50 0x7f2658059210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f2674006fd0 tx=0x7f2674008040).stop 2020-11-20T12:03:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26800aa1e0 msgr2=0x7f26800a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26800aa1e0 0x7f26800a3420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f266c00c920 tx=0x7f266c008040).stop 2020-11-20T12:03:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 shutdown_connections 2020-11-20T12:03:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2658056d50 0x7f2658059210 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f268013c170 0x7f268014ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26800a7810 0x7f26801163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 --2- 172.21.15.44:0/1263958934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26800aa1e0 0x7f26800a3420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 >> 172.21.15.44:0/1263958934 conn(0x7f2680062690 msgr2=0x7f26800ab640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:11.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 shutdown_connections 2020-11-20T12:03:11.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:10.995+0000 7f2686d4d700 1 -- 172.21.15.44:0/1263958934 wait complete. 2020-11-20T12:03:11.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.7 2020-11-20T12:03:11.011 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-20T12:03:11.011 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-20T12:03:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.133+0000 7f921e145700 1 Processor -- start 2020-11-20T12:03:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.133+0000 7f921e145700 1 -- start start 2020-11-20T12:03:11.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f921e145700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92180aa400 0x7f92180a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f921e145700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92180a9770 con 0x7f92180aa020 2020-11-20T12:03:11.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f921e145700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92180a6da0 con 0x7f921813ac60 2020-11-20T12:03:11.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f921e145700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92180a56c0 con 0x7f92180aa400 2020-11-20T12:03:11.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92177fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f92180aa020 0x7f92180a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40172/0 (socket says 172.21.15.44:40172) 2020-11-20T12:03:11.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92177fe700 1 -- 172.21.15.44:0/3211244892 learned_addr learned my addr 172.21.15.44:0/3211244892 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:11.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959736321 0 0) 0x7f92080010d0 con 0x7f92180aa020 2020-11-20T12:03:11.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f9216ffd700 1 --2- 172.21.15.44:0/3211244892 >> v2:172.21.15.79:3300/0 conn(0x7f92180aa400 0x7f92180a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9204003240 con 0x7f92180aa020 2020-11-20T12:03:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 798790535 0 0) 0x7f9204003240 con 0x7f92180aa020 2020-11-20T12:03:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 >> v2:172.21.15.79:3300/0 conn(0x7f92180aa400 msgr2=0x7f92180a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 --2- 172.21.15.44:0/3211244892 >> v2:172.21.15.79:3300/0 conn(0x7f92180aa400 0x7f92180a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 >> v1:172.21.15.79:6789/0 conn(0x7f921813ac60 legacy=0x7f92180a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92180a7040 con 0x7f92180aa020 2020-11-20T12:03:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f9208001670 con 0x7f92180aa020 2020-11-20T12:03:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 >> v1:172.21.15.44:6789/0 conn(0x7f92180aa020 legacy=0x7f92180a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9204003730 0x7f9204003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92040044e0 0x7f9204004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204004e50 0x7f920400d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180a56c0 con 0x7f9204003730 2020-11-20T12:03:11.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204004260 con 0x7f92040044e0 2020-11-20T12:03:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920400de30 con 0x7f9204004e50 2020-11-20T12:03:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.134+0000 7f92177fe700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9204003730 0x7f9204003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f9216ffd700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92040044e0 0x7f9204004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f9217fff700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204004e50 0x7f920400d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92177fe700 1 -- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204004e50 msgr2=0x7f920400d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92177fe700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204004e50 0x7f920400d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92177fe700 1 -- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92040044e0 msgr2=0x7f9204004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92177fe700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92040044e0 0x7f9204004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f9217fff700 1 -- 172.21.15.44:0/3211244892 reap_dead start 2020-11-20T12:03:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92177fe700 1 -- 172.21.15.44:0/3211244892 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204003240 con 0x7f9204003730 2020-11-20T12:03:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92177fe700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9204003730 0x7f9204003b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f920800c440 tx=0x7f92180a6f30).ready entity=mon.0 client_cookie=65d46b43224bfb10 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920800ce40 con 0x7f9204003730 2020-11-20T12:03:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92080072e0 con 0x7f9204003730 2020-11-20T12:03:11.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f92167fc700 1 -- 172.21.15.44:0/3211244892 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208005820 con 0x7f9204003730 2020-11-20T12:03:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f921e145700 1 -- 172.21.15.44:0/3211244892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9204003730 msgr2=0x7f9204003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f921e145700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9204003730 0x7f9204003b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f920800c440 tx=0x7f92180a6f30).stop 2020-11-20T12:03:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f921e145700 1 -- 172.21.15.44:0/3211244892 shutdown_connections 2020-11-20T12:03:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f921e145700 1 --2- 172.21.15.44:0/3211244892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9204003730 0x7f9204003b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.135+0000 7f921e145700 1 -- 172.21.15.44:0/3211244892 >> 172.21.15.44:0/3211244892 conn(0x7f9218062690 msgr2=0x7f92180a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.136+0000 7f921e145700 1 -- 172.21.15.44:0/3211244892 shutdown_connections 2020-11-20T12:03:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.136+0000 7f921e145700 1 -- 172.21.15.44:0/3211244892 wait complete. 2020-11-20T12:03:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 Processor -- start 2020-11-20T12:03:11.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 -- start start 2020-11-20T12:03:11.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204003730 0x7f92180ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921814dbf0 0x7f921814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9218133c10 0x7f9218134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180ac4e0 con 0x7f921814dbf0 2020-11-20T12:03:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921814e560 con 0x7f9218133c10 2020-11-20T12:03:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f921e145700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180acf30 con 0x7f9204003730 2020-11-20T12:03:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f9216ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921814dbf0 0x7f921814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f92177fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204003730 0x7f92180ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f9216ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921814dbf0 0x7f921814e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36438/0 (socket says 172.21.15.44:36438) 2020-11-20T12:03:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f9216ffd700 1 -- 172.21.15.44:0/380382764 learned_addr learned my addr 172.21.15.44:0/380382764 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.137+0000 7f9217fff700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9218133c10 0x7f9218134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f92177fe700 1 -- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9218133c10 msgr2=0x7f9218134090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f92177fe700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9218133c10 0x7f9218134090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f92177fe700 1 -- 172.21.15.44:0/380382764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921814dbf0 msgr2=0x7f921814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f92177fe700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921814dbf0 0x7f921814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f92177fe700 1 -- 172.21.15.44:0/380382764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92180ad1d0 con 0x7f9204003730 2020-11-20T12:03:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f9216ffd700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921814dbf0 0x7f921814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:11.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f92177fe700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204003730 0x7f92180ad7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f920800c440 tx=0x7f9208015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f91f77fe700 1 -- 172.21.15.44:0/380382764 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208005bd0 con 0x7f9204003730 2020-11-20T12:03:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f91f77fe700 1 -- 172.21.15.44:0/380382764 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208005d30 con 0x7f9204003730 2020-11-20T12:03:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f91f77fe700 1 -- 172.21.15.44:0/380382764 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208005ea0 con 0x7f9204003730 2020-11-20T12:03:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921814d750 con 0x7f9204003730 2020-11-20T12:03:11.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.138+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9218114aa0 con 0x7f9204003730 2020-11-20T12:03:11.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.139+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92180acda0 con 0x7f9204003730 2020-11-20T12:03:11.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.139+0000 7f91f77fe700 1 -- 172.21.15.44:0/380382764 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f920801d070 con 0x7f9204003730 2020-11-20T12:03:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.140+0000 7f91f77fe700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f0056d20 0x7f91f00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.140+0000 7f91f77fe700 1 -- 172.21.15.44:0/380382764 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9208070ba0 con 0x7f9204003730 2020-11-20T12:03:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.140+0000 7f9216ffd700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f0056d20 0x7f91f00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.140+0000 7f9216ffd700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f0056d20 0x7f91f00591e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f92180a85b0 tx=0x7f920c006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.144+0000 7f91f77fe700 1 -- 172.21.15.44:0/380382764 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9208045e30 con 0x7f9204003730 2020-11-20T12:03:11.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.311+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 --> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92180a9770 con 0x7f91f0056d20 2020-11-20T12:03:11.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.315+0000 7f91f77fe700 1 -- 172.21.15.44:0/380382764 <== mgr.4105 v2:172.21.15.79:6832/24383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82152 (secure 0 0 0) 0x7f92180a9770 con 0x7f91f0056d20 2020-11-20T12:03:11.319 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:11.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f0056d20 msgr2=0x7f91f00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f0056d20 0x7f91f00591e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f92180a85b0 tx=0x7f920c006df0).stop 2020-11-20T12:03:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204003730 msgr2=0x7f92180ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204003730 0x7f92180ad7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f920800c440 tx=0x7f9208015000).stop 2020-11-20T12:03:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 shutdown_connections 2020-11-20T12:03:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f91f0056d20 0x7f91f00591e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9218133c10 0x7f9218134090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9204003730 0x7f92180ad7c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 --2- 172.21.15.44:0/380382764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f921814dbf0 0x7f921814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 >> 172.21.15.44:0/380382764 conn(0x7f9218062690 msgr2=0x7f92180a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 shutdown_connections 2020-11-20T12:03:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.316+0000 7f921e145700 1 -- 172.21.15.44:0/380382764 wait complete. 2020-11-20T12:03:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2020-11-20T12:03:11.331 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2020-11-20T12:03:10.576195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41216,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":39040,"kb_avail":754933504,"statfs":{"total":773094113280,"available":773051908096,"internally_reserved":0,"allocated":2162688,"data_stored":837448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39976960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.395221"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873331+0000","last_change":"2020-11-20T12:02:59.871769+0000","last_active":"2020-11-20T12:02:59.873331+0000","last_peered":"2020-11-20T12:02:59.873331+0000","last_clean":"2020-11-20T12:02:59.873331+0000","last_became_active":"2020-11-20T12:02:59.871151+0000","last_became_peered":"2020-11-20T12:02:59.871151+0000","last_unstale":"2020-11-20T12:02:59.873331+0000","last_undegraded":"2020-11-20T12:02:59.873331+0000","last_fullsized":"2020-11-20T12:02:59.873331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.774547+0000","last_change":"2020-11-20T12:02:53.744074+0000","last_active":"2020-11-20T12:02:59.774547+0000","last_peered":"2020-11-20T12:02:59.774547+0000","last_clean":"2020-11-20T12:02:59.774547+0000","last_became_active":"2020-11-20T12:02:53.743596+0000","last_became_peered":"2020-11-20T12:02:53.743596+0000","last_unstale":"2020-11-20T12:02:59.774547+0000","last_undegraded":"2020-11-20T12:02:59.774547+0000","last_fullsized":"2020-11-20T12:02:59.774547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.195451+0000","last_change":"2020-11-20T12:02:56.751652+0000","last_active":"2020-11-20T12:03:00.195451+0000","last_peered":"2020-11-20T12:03:00.195451+0000","last_clean":"2020-11-20T12:03:00.195451+0000","last_became_active":"2020-11-20T12:02:56.751344+0000","last_became_peered":"2020-11-20T12:02:56.751344+0000","last_unstale":"2020-11-20T12:03:00.195451+0000","last_undegraded":"2020-11-20T12:03:00.195451+0000","last_fullsized":"2020-11-20T12:03:00.195451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.780084+0000","last_change":"2020-11-20T12:02:59.779112+0000","last_active":"2020-11-20T12:02:59.780084+0000","last_peered":"2020-11-20T12:02:59.780084+0000","last_clean":"2020-11-20T12:02:59.780084+0000","last_became_active":"2020-11-20T12:02:59.778434+0000","last_became_peered":"2020-11-20T12:02:59.778434+0000","last_unstale":"2020-11-20T12:02:59.780084+0000","last_undegraded":"2020-11-20T12:02:59.780084+0000","last_fullsized":"2020-11-20T12:02:59.780084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.867987+0000","last_change":"2020-11-20T12:02:53.623419+0000","last_active":"2020-11-20T12:02:59.867987+0000","last_peered":"2020-11-20T12:02:59.867987+0000","last_clean":"2020-11-20T12:02:59.867987+0000","last_became_active":"2020-11-20T12:02:53.623135+0000","last_became_peered":"2020-11-20T12:02:53.623135+0000","last_unstale":"2020-11-20T12:02:59.867987+0000","last_undegraded":"2020-11-20T12:02:59.867987+0000","last_fullsized":"2020-11-20T12:02:59.867987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.191541+0000","last_change":"2020-11-20T12:02:56.750366+0000","last_active":"2020-11-20T12:03:00.191541+0000","last_peered":"2020-11-20T12:03:00.191541+0000","last_clean":"2020-11-20T12:03:00.191541+0000","last_became_active":"2020-11-20T12:02:56.749896+0000","last_became_peered":"2020-11-20T12:02:56.749896+0000","last_unstale":"2020-11-20T12:03:00.191541+0000","last_undegraded":"2020-11-20T12:03:00.191541+0000","last_fullsized":"2020-11-20T12:03:00.191541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873473+0000","last_change":"2020-11-20T12:02:59.871656+0000","last_active":"2020-11-20T12:02:59.873473+0000","last_peered":"2020-11-20T12:02:59.873473+0000","last_clean":"2020-11-20T12:02:59.873473+0000","last_became_active":"2020-11-20T12:02:59.871033+0000","last_became_peered":"2020-11-20T12:02:59.871033+0000","last_unstale":"2020-11-20T12:02:59.873473+0000","last_undegraded":"2020-11-20T12:02:59.873473+0000","last_fullsized":"2020-11-20T12:02:59.873473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769916+0000","last_change":"2020-11-20T12:02:53.620206+0000","last_active":"2020-11-20T12:02:59.769916+0000","last_peered":"2020-11-20T12:02:59.769916+0000","last_clean":"2020-11-20T12:02:59.769916+0000","last_became_active":"2020-11-20T12:02:53.619848+0000","last_became_peered":"2020-11-20T12:02:53.619848+0000","last_unstale":"2020-11-20T12:02:59.769916+0000","last_undegraded":"2020-11-20T12:02:59.769916+0000","last_fullsized":"2020-11-20T12:02:59.769916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.196841+0000","last_change":"2020-11-20T12:02:56.751843+0000","last_active":"2020-11-20T12:03:00.196841+0000","last_peered":"2020-11-20T12:03:00.196841+0000","last_clean":"2020-11-20T12:03:00.196841+0000","last_became_active":"2020-11-20T12:02:56.751444+0000","last_became_peered":"2020-11-20T12:02:56.751444+0000","last_unstale":"2020-11-20T12:03:00.196841+0000","last_undegraded":"2020-11-20T12:03:00.196841+0000","last_fullsized":"2020-11-20T12:03:00.196841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871493+0000","last_change":"2020-11-20T12:02:59.870900+0000","last_active":"2020-11-20T12:02:59.871493+0000","last_peered":"2020-11-20T12:02:59.871493+0000","last_clean":"2020-11-20T12:02:59.871493+0000","last_became_active":"2020-11-20T12:02:59.870194+0000","last_became_peered":"2020-11-20T12:02:59.870194+0000","last_unstale":"2020-11-20T12:02:59.871493+0000","last_undegraded":"2020-11-20T12:02:59.871493+0000","last_fullsized":"2020-11-20T12:02:59.871493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868625+0000","last_change":"2020-11-20T12:02:53.621673+0000","last_active":"2020-11-20T12:02:59.868625+0000","last_peered":"2020-11-20T12:02:59.868625+0000","last_clean":"2020-11-20T12:02:59.868625+0000","last_became_active":"2020-11-20T12:02:53.621317+0000","last_became_peered":"2020-11-20T12:02:53.621317+0000","last_unstale":"2020-11-20T12:02:59.868625+0000","last_undegraded":"2020-11-20T12:02:59.868625+0000","last_fullsized":"2020-11-20T12:02:59.868625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868438+0000","last_change":"2020-11-20T12:02:49.777943+0000","last_active":"2020-11-20T12:02:59.868438+0000","last_peered":"2020-11-20T12:02:59.868438+0000","last_clean":"2020-11-20T12:02:59.868438+0000","last_became_active":"2020-11-20T12:02:49.777287+0000","last_became_peered":"2020-11-20T12:02:49.777287+0000","last_unstale":"2020-11-20T12:02:59.868438+0000","last_undegraded":"2020-11-20T12:02:59.868438+0000","last_fullsized":"2020-11-20T12:02:59.868438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_clean_scrub_stamp":"2020-11-20T12:02:45.568237+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":"13","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.182489+0000","last_change":"2020-11-20T12:02:56.751911+0000","last_active":"2020-11-20T12:03:00.182489+0000","last_peered":"2020-11-20T12:03:00.182489+0000","last_clean":"2020-11-20T12:03:00.182489+0000","last_became_active":"2020-11-20T12:02:56.751558+0000","last_became_peered":"2020-11-20T12:02:56.751558+0000","last_unstale":"2020-11-20T12:03:00.182489+0000","last_undegraded":"2020-11-20T12:03:00.182489+0000","last_fullsized":"2020-11-20T12:03:00.182489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.870381+0000","last_change":"2020-11-20T12:02:59.869783+0000","last_active":"2020-11-20T12:02:59.870381+0000","last_peered":"2020-11-20T12:02:59.870381+0000","last_clean":"2020-11-20T12:02:59.870381+0000","last_became_active":"2020-11-20T12:02:59.869158+0000","last_became_peered":"2020-11-20T12:02:59.869158+0000","last_unstale":"2020-11-20T12:02:59.870381+0000","last_undegraded":"2020-11-20T12:02:59.870381+0000","last_fullsized":"2020-11-20T12:02:59.870381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868367+0000","last_change":"2020-11-20T12:02:53.621849+0000","last_active":"2020-11-20T12:02:59.868367+0000","last_peered":"2020-11-20T12:02:59.868367+0000","last_clean":"2020-11-20T12:02:59.868367+0000","last_became_active":"2020-11-20T12:02:53.621479+0000","last_became_peered":"2020-11-20T12:02:53.621479+0000","last_unstale":"2020-11-20T12:02:59.868367+0000","last_undegraded":"2020-11-20T12:02:59.868367+0000","last_fullsized":"2020-11-20T12:02:59.868367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.770789+0000","last_change":"2020-11-20T12:02:56.750191+0000","last_active":"2020-11-20T12:02:59.770789+0000","last_peered":"2020-11-20T12:02:59.770789+0000","last_clean":"2020-11-20T12:02:59.770789+0000","last_became_active":"2020-11-20T12:02:56.749787+0000","last_became_peered":"2020-11-20T12:02:56.749787+0000","last_unstale":"2020-11-20T12:02:59.770789+0000","last_undegraded":"2020-11-20T12:02:59.770789+0000","last_fullsized":"2020-11-20T12:02:59.770789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873151+0000","last_change":"2020-11-20T12:02:59.871503+0000","last_active":"2020-11-20T12:02:59.873151+0000","last_peered":"2020-11-20T12:02:59.873151+0000","last_clean":"2020-11-20T12:02:59.873151+0000","last_became_active":"2020-11-20T12:02:59.870839+0000","last_became_peered":"2020-11-20T12:02:59.870839+0000","last_unstale":"2020-11-20T12:02:59.873151+0000","last_undegraded":"2020-11-20T12:02:59.873151+0000","last_fullsized":"2020-11-20T12:02:59.873151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868440+0000","last_change":"2020-11-20T12:02:53.746269+0000","last_active":"2020-11-20T12:02:59.868440+0000","last_peered":"2020-11-20T12:02:59.868440+0000","last_clean":"2020-11-20T12:02:59.868440+0000","last_became_active":"2020-11-20T12:02:53.745919+0000","last_became_peered":"2020-11-20T12:02:53.745919+0000","last_unstale":"2020-11-20T12:02:59.868440+0000","last_undegraded":"2020-11-20T12:02:59.868440+0000","last_fullsized":"2020-11-20T12:02:59.868440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.194175+0000","last_change":"2020-11-20T12:02:56.752631+0000","last_active":"2020-11-20T12:03:00.194175+0000","last_peered":"2020-11-20T12:03:00.194175+0000","last_clean":"2020-11-20T12:03:00.194175+0000","last_became_active":"2020-11-20T12:02:56.752287+0000","last_became_peered":"2020-11-20T12:02:56.752287+0000","last_unstale":"2020-11-20T12:03:00.194175+0000","last_undegraded":"2020-11-20T12:03:00.194175+0000","last_fullsized":"2020-11-20T12:03:00.194175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871787+0000","last_change":"2020-11-20T12:02:59.869976+0000","last_active":"2020-11-20T12:02:59.871787+0000","last_peered":"2020-11-20T12:02:59.871787+0000","last_clean":"2020-11-20T12:02:59.871787+0000","last_became_active":"2020-11-20T12:02:59.869214+0000","last_became_peered":"2020-11-20T12:02:59.869214+0000","last_unstale":"2020-11-20T12:02:59.871787+0000","last_undegraded":"2020-11-20T12:02:59.871787+0000","last_fullsized":"2020-11-20T12:02:59.871787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192866+0000","last_change":"2020-11-20T12:02:57.108914+0000","last_active":"2020-11-20T12:03:00.192866+0000","last_peered":"2020-11-20T12:03:00.192866+0000","last_clean":"2020-11-20T12:03:00.192866+0000","last_became_active":"2020-11-20T12:02:57.108538+0000","last_became_peered":"2020-11-20T12:02:57.108538+0000","last_unstale":"2020-11-20T12:03:00.192866+0000","last_undegraded":"2020-11-20T12:03:00.192866+0000","last_fullsized":"2020-11-20T12:03:00.192866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.771858+0000","last_change":"2020-11-20T12:02:53.618862+0000","last_active":"2020-11-20T12:02:59.771858+0000","last_peered":"2020-11-20T12:02:59.771858+0000","last_clean":"2020-11-20T12:02:59.771858+0000","last_became_active":"2020-11-20T12:02:53.618438+0000","last_became_peered":"2020-11-20T12:02:53.618438+0000","last_unstale":"2020-11-20T12:02:59.771858+0000","last_undegraded":"2020-11-20T12:02:59.771858+0000","last_fullsized":"2020-11-20T12:02:59.771858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873226+0000","last_change":"2020-11-20T12:02:59.872290+0000","last_active":"2020-11-20T12:02:59.873226+0000","last_peered":"2020-11-20T12:02:59.873226+0000","last_clean":"2020-11-20T12:02:59.873226+0000","last_became_active":"2020-11-20T12:02:59.870620+0000","last_became_peered":"2020-11-20T12:02:59.870620+0000","last_unstale":"2020-11-20T12:02:59.873226+0000","last_undegraded":"2020-11-20T12:02:59.873226+0000","last_fullsized":"2020-11-20T12:02:59.873226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769189+0000","last_change":"2020-11-20T12:02:53.620514+0000","last_active":"2020-11-20T12:02:59.769189+0000","last_peered":"2020-11-20T12:02:59.769189+0000","last_clean":"2020-11-20T12:02:59.769189+0000","last_became_active":"2020-11-20T12:02:53.620221+0000","last_became_peered":"2020-11-20T12:02:53.620221+0000","last_unstale":"2020-11-20T12:02:59.769189+0000","last_undegraded":"2020-11-20T12:02:59.769189+0000","last_fullsized":"2020-11-20T12:02:59.769189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"16","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192738+0000","last_change":"2020-11-20T12:02:56.752071+0000","last_active":"2020-11-20T12:03:00.192738+0000","last_peered":"2020-11-20T12:03:00.192738+0000","last_clean":"2020-11-20T12:03:00.192738+0000","last_became_active":"2020-11-20T12:02:56.751653+0000","last_became_peered":"2020-11-20T12:02:56.751653+0000","last_unstale":"2020-11-20T12:03:00.192738+0000","last_undegraded":"2020-11-20T12:03:00.192738+0000","last_fullsized":"2020-11-20T12:03:00.192738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":104105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5196,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366644,"statfs":{"total":96636764160,"available":96631443456,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.764}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5140,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366700,"statfs":{"total":96636764160,"available":96631500800,"internally_reserved":0,"allocated":274432,"data_stored":105640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5204,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366636,"statfs":{"total":96636764160,"available":96631435264,"internally_reserved":0,"allocated":274432,"data_stored":105640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":278528,"data_stored":104840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5132,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366708,"statfs":{"total":96636764160,"available":96631508992,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]}]}} 2020-11-20T12:03:11.334 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-20T12:03:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.455+0000 7f2eb43af700 1 Processor -- start 2020-11-20T12:03:11.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.455+0000 7f2eb43af700 1 -- start start 2020-11-20T12:03:11.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb43af700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2eac13a990 0x7f2eac13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb43af700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2eac13f800 0x7f2eac141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb43af700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0a6260 con 0x7f2eac060330 2020-11-20T12:03:11.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb43af700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0a52e0 con 0x7f2eac13a990 2020-11-20T12:03:11.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb43af700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0a6490 con 0x7f2eac13f800 2020-11-20T12:03:11.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb194a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2eac13a990 0x7f2eac13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb214b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2eac060330 0x7f2eac0a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40192/0 (socket says 172.21.15.44:40192) 2020-11-20T12:03:11.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb214b700 1 -- 172.21.15.44:0/1296798713 learned_addr learned my addr 172.21.15.44:0/1296798713 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:11.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb294c700 1 --2- 172.21.15.44:0/1296798713 >> v2:172.21.15.79:3300/0 conn(0x7f2eac13f800 0x7f2eac141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb1149700 1 -- 172.21.15.44:0/1296798713 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519577248 0 0) 0x7f2eac0a6260 con 0x7f2eac060330 2020-11-20T12:03:11.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.456+0000 7f2eb1149700 1 -- 172.21.15.44:0/1296798713 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e98003240 con 0x7f2eac060330 2020-11-20T12:03:11.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb194a700 1 -- 172.21.15.44:0/1296798713 >> v2:172.21.15.79:3300/0 conn(0x7f2eac13f800 msgr2=0x7f2eac141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:11.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb194a700 1 --2- 172.21.15.44:0/1296798713 >> v2:172.21.15.79:3300/0 conn(0x7f2eac13f800 0x7f2eac141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb194a700 1 -- 172.21.15.44:0/1296798713 >> v1:172.21.15.44:6789/0 conn(0x7f2eac060330 legacy=0x7f2eac0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb194a700 1 -- 172.21.15.44:0/1296798713 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac0a6730 con 0x7f2eac13a990 2020-11-20T12:03:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb194a700 1 --2- 172.21.15.44:0/1296798713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac13a990 0x7f2eac13ae20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f2eac0a5a90 tx=0x7f2ea0006df0).ready entity=mon.0 client_cookie=63540e0bb97d9ff4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb1149700 1 -- 172.21.15.44:0/1296798713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0009640 con 0x7f2eac13a990 2020-11-20T12:03:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb1149700 1 -- 172.21.15.44:0/1296798713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea00097a0 con 0x7f2eac13a990 2020-11-20T12:03:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb1149700 1 -- 172.21.15.44:0/1296798713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0009b20 con 0x7f2eac13a990 2020-11-20T12:03:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 -- 172.21.15.44:0/1296798713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac13a990 msgr2=0x7f2eac13ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1296798713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac13a990 0x7f2eac13ae20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f2eac0a5a90 tx=0x7f2ea0006df0).stop 2020-11-20T12:03:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 -- 172.21.15.44:0/1296798713 shutdown_connections 2020-11-20T12:03:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1296798713 >> v2:172.21.15.79:3300/0 conn(0x7f2eac13f800 0x7f2eac141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1296798713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac13a990 0x7f2eac13ae20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 -- 172.21.15.44:0/1296798713 >> 172.21.15.44:0/1296798713 conn(0x7f2eac062690 msgr2=0x7f2eac0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 -- 172.21.15.44:0/1296798713 shutdown_connections 2020-11-20T12:03:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.457+0000 7f2eb43af700 1 -- 172.21.15.44:0/1296798713 wait complete. 2020-11-20T12:03:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.458+0000 7f2eb43af700 1 Processor -- start 2020-11-20T12:03:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.458+0000 7f2eb43af700 1 -- start start 2020-11-20T12:03:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb43af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac060330 0x7f2eac13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb43af700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2eac13a990 0x7f2eac147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb43af700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2eac13f800 0x7f2eac134040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb43af700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac131b80 con 0x7f2eac060330 2020-11-20T12:03:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb43af700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac13db20 con 0x7f2eac13a990 2020-11-20T12:03:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb43af700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac13ddc0 con 0x7f2eac13f800 2020-11-20T12:03:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac060330 0x7f2eac13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac060330 0x7f2eac13ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36452/0 (socket says 172.21.15.44:36452) 2020-11-20T12:03:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 -- 172.21.15.44:0/1819474388 learned_addr learned my addr 172.21.15.44:0/1819474388 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb194a700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2eac13a990 0x7f2eac147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb294c700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2eac13f800 0x7f2eac134040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 -- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2eac13f800 msgr2=0x7f2eac134040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2eac13f800 0x7f2eac134040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 -- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2eac13a990 msgr2=0x7f2eac147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2eac13a990 0x7f2eac147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 -- 172.21.15.44:0/1819474388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac131830 con 0x7f2eac060330 2020-11-20T12:03:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.459+0000 7f2eb214b700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac060330 0x7f2eac13ef70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f2ea400ca60 tx=0x7f2ea4007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.460+0000 7f2eaa7fc700 1 -- 172.21.15.44:0/1819474388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4005b80 con 0x7f2eac060330 2020-11-20T12:03:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.460+0000 7f2eaa7fc700 1 -- 172.21.15.44:0/1819474388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4005ce0 con 0x7f2eac060330 2020-11-20T12:03:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.460+0000 7f2eaa7fc700 1 -- 172.21.15.44:0/1819474388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4007340 con 0x7f2eac060330 2020-11-20T12:03:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.460+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac148290 con 0x7f2eac060330 2020-11-20T12:03:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.460+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac13e260 con 0x7f2eac060330 2020-11-20T12:03:11.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.461+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac13e4b0 con 0x7f2eac060330 2020-11-20T12:03:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.461+0000 7f2eaa7fc700 1 -- 172.21.15.44:0/1819474388 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2ea4010030 con 0x7f2eac060330 2020-11-20T12:03:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.461+0000 7f2eaa7fc700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d20 0x7f2e840591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.461+0000 7f2eaa7fc700 1 -- 172.21.15.44:0/1819474388 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2ea4070090 con 0x7f2eac060330 2020-11-20T12:03:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.461+0000 7f2eb194a700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d20 0x7f2e840591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.462+0000 7f2eb194a700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d20 0x7f2e840591e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2ea0006e30 tx=0x7f2ea0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.465+0000 7f2eaa7fc700 1 -- 172.21.15.44:0/1819474388 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2eac13e4b0 con 0x7f2eac060330 2020-11-20T12:03:11.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.633+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 --> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2eac13e4b0 con 0x7f2e84056d20 2020-11-20T12:03:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.637+0000 7f2eaa7fc700 1 -- 172.21.15.44:0/1819474388 <== mgr.4105 v2:172.21.15.79:6832/24383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82152 (secure 0 0 0) 0x7f2eac13e4b0 con 0x7f2e84056d20 2020-11-20T12:03:11.642 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d20 msgr2=0x7f2e840591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d20 0x7f2e840591e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2ea0006e30 tx=0x7f2ea0015000).stop 2020-11-20T12:03:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac060330 msgr2=0x7f2eac13ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac060330 0x7f2eac13ef70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f2ea400ca60 tx=0x7f2ea4007e40).stop 2020-11-20T12:03:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 shutdown_connections 2020-11-20T12:03:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d20 0x7f2e840591e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2eac13f800 0x7f2eac134040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2eac060330 0x7f2eac13ef70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 --2- 172.21.15.44:0/1819474388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2eac13a990 0x7f2eac147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 >> 172.21.15.44:0/1819474388 conn(0x7f2eac062690 msgr2=0x7f2eac138bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 shutdown_connections 2020-11-20T12:03:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.639+0000 7f2eb43af700 1 -- 172.21.15.44:0/1819474388 wait complete. 2020-11-20T12:03:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2020-11-20T12:03:11.653 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2020-11-20T12:03:10.576195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41216,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":39040,"kb_avail":754933504,"statfs":{"total":773094113280,"available":773051908096,"internally_reserved":0,"allocated":2162688,"data_stored":837448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39976960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.395221"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873331+0000","last_change":"2020-11-20T12:02:59.871769+0000","last_active":"2020-11-20T12:02:59.873331+0000","last_peered":"2020-11-20T12:02:59.873331+0000","last_clean":"2020-11-20T12:02:59.873331+0000","last_became_active":"2020-11-20T12:02:59.871151+0000","last_became_peered":"2020-11-20T12:02:59.871151+0000","last_unstale":"2020-11-20T12:02:59.873331+0000","last_undegraded":"2020-11-20T12:02:59.873331+0000","last_fullsized":"2020-11-20T12:02:59.873331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.774547+0000","last_change":"2020-11-20T12:02:53.744074+0000","last_active":"2020-11-20T12:02:59.774547+0000","last_peered":"2020-11-20T12:02:59.774547+0000","last_clean":"2020-11-20T12:02:59.774547+0000","last_became_active":"2020-11-20T12:02:53.743596+0000","last_became_peered":"2020-11-20T12:02:53.743596+0000","last_unstale":"2020-11-20T12:02:59.774547+0000","last_undegraded":"2020-11-20T12:02:59.774547+0000","last_fullsized":"2020-11-20T12:02:59.774547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.195451+0000","last_change":"2020-11-20T12:02:56.751652+0000","last_active":"2020-11-20T12:03:00.195451+0000","last_peered":"2020-11-20T12:03:00.195451+0000","last_clean":"2020-11-20T12:03:00.195451+0000","last_became_active":"2020-11-20T12:02:56.751344+0000","last_became_peered":"2020-11-20T12:02:56.751344+0000","last_unstale":"2020-11-20T12:03:00.195451+0000","last_undegraded":"2020-11-20T12:03:00.195451+0000","last_fullsized":"2020-11-20T12:03:00.195451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.780084+0000","last_change":"2020-11-20T12:02:59.779112+0000","last_active":"2020-11-20T12:02:59.780084+0000","last_peered":"2020-11-20T12:02:59.780084+0000","last_clean":"2020-11-20T12:02:59.780084+0000","last_became_active":"2020-11-20T12:02:59.778434+0000","last_became_peered":"2020-11-20T12:02:59.778434+0000","last_unstale":"2020-11-20T12:02:59.780084+0000","last_undegraded":"2020-11-20T12:02:59.780084+0000","last_fullsized":"2020-11-20T12:02:59.780084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.867987+0000","last_change":"2020-11-20T12:02:53.623419+0000","last_active":"2020-11-20T12:02:59.867987+0000","last_peered":"2020-11-20T12:02:59.867987+0000","last_clean":"2020-11-20T12:02:59.867987+0000","last_became_active":"2020-11-20T12:02:53.623135+0000","last_became_peered":"2020-11-20T12:02:53.623135+0000","last_unstale":"2020-11-20T12:02:59.867987+0000","last_undegraded":"2020-11-20T12:02:59.867987+0000","last_fullsized":"2020-11-20T12:02:59.867987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.191541+0000","last_change":"2020-11-20T12:02:56.750366+0000","last_active":"2020-11-20T12:03:00.191541+0000","last_peered":"2020-11-20T12:03:00.191541+0000","last_clean":"2020-11-20T12:03:00.191541+0000","last_became_active":"2020-11-20T12:02:56.749896+0000","last_became_peered":"2020-11-20T12:02:56.749896+0000","last_unstale":"2020-11-20T12:03:00.191541+0000","last_undegraded":"2020-11-20T12:03:00.191541+0000","last_fullsized":"2020-11-20T12:03:00.191541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873473+0000","last_change":"2020-11-20T12:02:59.871656+0000","last_active":"2020-11-20T12:02:59.873473+0000","last_peered":"2020-11-20T12:02:59.873473+0000","last_clean":"2020-11-20T12:02:59.873473+0000","last_became_active":"2020-11-20T12:02:59.871033+0000","last_became_peered":"2020-11-20T12:02:59.871033+0000","last_unstale":"2020-11-20T12:02:59.873473+0000","last_undegraded":"2020-11-20T12:02:59.873473+0000","last_fullsized":"2020-11-20T12:02:59.873473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769916+0000","last_change":"2020-11-20T12:02:53.620206+0000","last_active":"2020-11-20T12:02:59.769916+0000","last_peered":"2020-11-20T12:02:59.769916+0000","last_clean":"2020-11-20T12:02:59.769916+0000","last_became_active":"2020-11-20T12:02:53.619848+0000","last_became_peered":"2020-11-20T12:02:53.619848+0000","last_unstale":"2020-11-20T12:02:59.769916+0000","last_undegraded":"2020-11-20T12:02:59.769916+0000","last_fullsized":"2020-11-20T12:02:59.769916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.196841+0000","last_change":"2020-11-20T12:02:56.751843+0000","last_active":"2020-11-20T12:03:00.196841+0000","last_peered":"2020-11-20T12:03:00.196841+0000","last_clean":"2020-11-20T12:03:00.196841+0000","last_became_active":"2020-11-20T12:02:56.751444+0000","last_became_peered":"2020-11-20T12:02:56.751444+0000","last_unstale":"2020-11-20T12:03:00.196841+0000","last_undegraded":"2020-11-20T12:03:00.196841+0000","last_fullsized":"2020-11-20T12:03:00.196841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871493+0000","last_change":"2020-11-20T12:02:59.870900+0000","last_active":"2020-11-20T12:02:59.871493+0000","last_peered":"2020-11-20T12:02:59.871493+0000","last_clean":"2020-11-20T12:02:59.871493+0000","last_became_active":"2020-11-20T12:02:59.870194+0000","last_became_peered":"2020-11-20T12:02:59.870194+0000","last_unstale":"2020-11-20T12:02:59.871493+0000","last_undegraded":"2020-11-20T12:02:59.871493+0000","last_fullsized":"2020-11-20T12:02:59.871493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868625+0000","last_change":"2020-11-20T12:02:53.621673+0000","last_active":"2020-11-20T12:02:59.868625+0000","last_peered":"2020-11-20T12:02:59.868625+0000","last_clean":"2020-11-20T12:02:59.868625+0000","last_became_active":"2020-11-20T12:02:53.621317+0000","last_became_peered":"2020-11-20T12:02:53.621317+0000","last_unstale":"2020-11-20T12:02:59.868625+0000","last_undegraded":"2020-11-20T12:02:59.868625+0000","last_fullsized":"2020-11-20T12:02:59.868625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868438+0000","last_change":"2020-11-20T12:02:49.777943+0000","last_active":"2020-11-20T12:02:59.868438+0000","last_peered":"2020-11-20T12:02:59.868438+0000","last_clean":"2020-11-20T12:02:59.868438+0000","last_became_active":"2020-11-20T12:02:49.777287+0000","last_became_peered":"2020-11-20T12:02:49.777287+0000","last_unstale":"2020-11-20T12:02:59.868438+0000","last_undegraded":"2020-11-20T12:02:59.868438+0000","last_fullsized":"2020-11-20T12:02:59.868438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_clean_scrub_stamp":"2020-11-20T12:02:45.568237+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":"13","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.182489+0000","last_change":"2020-11-20T12:02:56.751911+0000","last_active":"2020-11-20T12:03:00.182489+0000","last_peered":"2020-11-20T12:03:00.182489+0000","last_clean":"2020-11-20T12:03:00.182489+0000","last_became_active":"2020-11-20T12:02:56.751558+0000","last_became_peered":"2020-11-20T12:02:56.751558+0000","last_unstale":"2020-11-20T12:03:00.182489+0000","last_undegraded":"2020-11-20T12:03:00.182489+0000","last_fullsized":"2020-11-20T12:03:00.182489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.870381+0000","last_change":"2020-11-20T12:02:59.869783+0000","last_active":"2020-11-20T12:02:59.870381+0000","last_peered":"2020-11-20T12:02:59.870381+0000","last_clean":"2020-11-20T12:02:59.870381+0000","last_became_active":"2020-11-20T12:02:59.869158+0000","last_became_peered":"2020-11-20T12:02:59.869158+0000","last_unstale":"2020-11-20T12:02:59.870381+0000","last_undegraded":"2020-11-20T12:02:59.870381+0000","last_fullsized":"2020-11-20T12:02:59.870381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868367+0000","last_change":"2020-11-20T12:02:53.621849+0000","last_active":"2020-11-20T12:02:59.868367+0000","last_peered":"2020-11-20T12:02:59.868367+0000","last_clean":"2020-11-20T12:02:59.868367+0000","last_became_active":"2020-11-20T12:02:53.621479+0000","last_became_peered":"2020-11-20T12:02:53.621479+0000","last_unstale":"2020-11-20T12:02:59.868367+0000","last_undegraded":"2020-11-20T12:02:59.868367+0000","last_fullsized":"2020-11-20T12:02:59.868367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.770789+0000","last_change":"2020-11-20T12:02:56.750191+0000","last_active":"2020-11-20T12:02:59.770789+0000","last_peered":"2020-11-20T12:02:59.770789+0000","last_clean":"2020-11-20T12:02:59.770789+0000","last_became_active":"2020-11-20T12:02:56.749787+0000","last_became_peered":"2020-11-20T12:02:56.749787+0000","last_unstale":"2020-11-20T12:02:59.770789+0000","last_undegraded":"2020-11-20T12:02:59.770789+0000","last_fullsized":"2020-11-20T12:02:59.770789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873151+0000","last_change":"2020-11-20T12:02:59.871503+0000","last_active":"2020-11-20T12:02:59.873151+0000","last_peered":"2020-11-20T12:02:59.873151+0000","last_clean":"2020-11-20T12:02:59.873151+0000","last_became_active":"2020-11-20T12:02:59.870839+0000","last_became_peered":"2020-11-20T12:02:59.870839+0000","last_unstale":"2020-11-20T12:02:59.873151+0000","last_undegraded":"2020-11-20T12:02:59.873151+0000","last_fullsized":"2020-11-20T12:02:59.873151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868440+0000","last_change":"2020-11-20T12:02:53.746269+0000","last_active":"2020-11-20T12:02:59.868440+0000","last_peered":"2020-11-20T12:02:59.868440+0000","last_clean":"2020-11-20T12:02:59.868440+0000","last_became_active":"2020-11-20T12:02:53.745919+0000","last_became_peered":"2020-11-20T12:02:53.745919+0000","last_unstale":"2020-11-20T12:02:59.868440+0000","last_undegraded":"2020-11-20T12:02:59.868440+0000","last_fullsized":"2020-11-20T12:02:59.868440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.194175+0000","last_change":"2020-11-20T12:02:56.752631+0000","last_active":"2020-11-20T12:03:00.194175+0000","last_peered":"2020-11-20T12:03:00.194175+0000","last_clean":"2020-11-20T12:03:00.194175+0000","last_became_active":"2020-11-20T12:02:56.752287+0000","last_became_peered":"2020-11-20T12:02:56.752287+0000","last_unstale":"2020-11-20T12:03:00.194175+0000","last_undegraded":"2020-11-20T12:03:00.194175+0000","last_fullsized":"2020-11-20T12:03:00.194175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871787+0000","last_change":"2020-11-20T12:02:59.869976+0000","last_active":"2020-11-20T12:02:59.871787+0000","last_peered":"2020-11-20T12:02:59.871787+0000","last_clean":"2020-11-20T12:02:59.871787+0000","last_became_active":"2020-11-20T12:02:59.869214+0000","last_became_peered":"2020-11-20T12:02:59.869214+0000","last_unstale":"2020-11-20T12:02:59.871787+0000","last_undegraded":"2020-11-20T12:02:59.871787+0000","last_fullsized":"2020-11-20T12:02:59.871787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192866+0000","last_change":"2020-11-20T12:02:57.108914+0000","last_active":"2020-11-20T12:03:00.192866+0000","last_peered":"2020-11-20T12:03:00.192866+0000","last_clean":"2020-11-20T12:03:00.192866+0000","last_became_active":"2020-11-20T12:02:57.108538+0000","last_became_peered":"2020-11-20T12:02:57.108538+0000","last_unstale":"2020-11-20T12:03:00.192866+0000","last_undegraded":"2020-11-20T12:03:00.192866+0000","last_fullsized":"2020-11-20T12:03:00.192866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.771858+0000","last_change":"2020-11-20T12:02:53.618862+0000","last_active":"2020-11-20T12:02:59.771858+0000","last_peered":"2020-11-20T12:02:59.771858+0000","last_clean":"2020-11-20T12:02:59.771858+0000","last_became_active":"2020-11-20T12:02:53.618438+0000","last_became_peered":"2020-11-20T12:02:53.618438+0000","last_unstale":"2020-11-20T12:02:59.771858+0000","last_undegraded":"2020-11-20T12:02:59.771858+0000","last_fullsized":"2020-11-20T12:02:59.771858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873226+0000","last_change":"2020-11-20T12:02:59.872290+0000","last_active":"2020-11-20T12:02:59.873226+0000","last_peered":"2020-11-20T12:02:59.873226+0000","last_clean":"2020-11-20T12:02:59.873226+0000","last_became_active":"2020-11-20T12:02:59.870620+0000","last_became_peered":"2020-11-20T12:02:59.870620+0000","last_unstale":"2020-11-20T12:02:59.873226+0000","last_undegraded":"2020-11-20T12:02:59.873226+0000","last_fullsized":"2020-11-20T12:02:59.873226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769189+0000","last_change":"2020-11-20T12:02:53.620514+0000","last_active":"2020-11-20T12:02:59.769189+0000","last_peered":"2020-11-20T12:02:59.769189+0000","last_clean":"2020-11-20T12:02:59.769189+0000","last_became_active":"2020-11-20T12:02:53.620221+0000","last_became_peered":"2020-11-20T12:02:53.620221+0000","last_unstale":"2020-11-20T12:02:59.769189+0000","last_undegraded":"2020-11-20T12:02:59.769189+0000","last_fullsized":"2020-11-20T12:02:59.769189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"16","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192738+0000","last_change":"2020-11-20T12:02:56.752071+0000","last_active":"2020-11-20T12:03:00.192738+0000","last_peered":"2020-11-20T12:03:00.192738+0000","last_clean":"2020-11-20T12:03:00.192738+0000","last_became_active":"2020-11-20T12:02:56.751653+0000","last_became_peered":"2020-11-20T12:02:56.751653+0000","last_unstale":"2020-11-20T12:03:00.192738+0000","last_undegraded":"2020-11-20T12:03:00.192738+0000","last_fullsized":"2020-11-20T12:03:00.192738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":104105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5196,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366644,"statfs":{"total":96636764160,"available":96631443456,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.764}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5140,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366700,"statfs":{"total":96636764160,"available":96631500800,"internally_reserved":0,"allocated":274432,"data_stored":105640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5204,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366636,"statfs":{"total":96636764160,"available":96631435264,"internally_reserved":0,"allocated":274432,"data_stored":105640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":278528,"data_stored":104840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5132,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366708,"statfs":{"total":96636764160,"available":96631508992,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]}]}} 2020-11-20T12:03:11.656 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-20T12:03:11.656 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-20T12:03:11.656 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-20T12:03:11.657 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2020-11-20T12:03:11.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.778+0000 7f713fe94700 1 Processor -- start 2020-11-20T12:03:11.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713fe94700 1 -- start start 2020-11-20T12:03:11.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713fe94700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71380aa400 0x7f71380a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713fe94700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f713813ac60 0x7f71380a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713fe94700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380a9770 con 0x7f71380aa020 2020-11-20T12:03:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713fe94700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71380a56c0 con 0x7f713813ac60 2020-11-20T12:03:11.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713fe94700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71380a6da0 con 0x7f71380aa400 2020-11-20T12:03:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713e431700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f713813ac60 0x7f71380a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713dc30700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f71380aa020 0x7f71380a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40206/0 (socket says 172.21.15.44:40206) 2020-11-20T12:03:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713dc30700 1 -- 172.21.15.44:0/3744691495 learned_addr learned my addr 172.21.15.44:0/3744691495 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.779+0000 7f713d42f700 1 --2- 172.21.15.44:0/3744691495 >> v2:172.21.15.79:3300/0 conn(0x7f71380aa400 0x7f71380a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713cc2e700 1 -- 172.21.15.44:0/3744691495 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943888725 0 0) 0x7f71280010d0 con 0x7f71380aa020 2020-11-20T12:03:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713cc2e700 1 -- 172.21.15.44:0/3744691495 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7120003240 con 0x7f71380aa020 2020-11-20T12:03:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713e431700 1 -- 172.21.15.44:0/3744691495 >> v2:172.21.15.79:3300/0 conn(0x7f71380aa400 msgr2=0x7f71380a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713e431700 1 --2- 172.21.15.44:0/3744691495 >> v2:172.21.15.79:3300/0 conn(0x7f71380aa400 0x7f71380a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713e431700 1 -- 172.21.15.44:0/3744691495 >> v1:172.21.15.44:6789/0 conn(0x7f71380aa020 legacy=0x7f71380a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713e431700 1 -- 172.21.15.44:0/3744691495 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380a7040 con 0x7f713813ac60 2020-11-20T12:03:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713e431700 1 --2- 172.21.15.44:0/3744691495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f713813ac60 0x7f71380a3700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f71380a4990 tx=0x7f7130006df0).ready entity=mon.0 client_cookie=85ae00542e7968f2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713cc2e700 1 -- 172.21.15.44:0/3744691495 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130009640 con 0x7f713813ac60 2020-11-20T12:03:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713cc2e700 1 -- 172.21.15.44:0/3744691495 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71300097a0 con 0x7f713813ac60 2020-11-20T12:03:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713cc2e700 1 -- 172.21.15.44:0/3744691495 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130009b20 con 0x7f713813ac60 2020-11-20T12:03:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713fe94700 1 -- 172.21.15.44:0/3744691495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f713813ac60 msgr2=0x7f71380a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713fe94700 1 --2- 172.21.15.44:0/3744691495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f713813ac60 0x7f71380a3700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f71380a4990 tx=0x7f7130006df0).stop 2020-11-20T12:03:11.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713fe94700 1 -- 172.21.15.44:0/3744691495 shutdown_connections 2020-11-20T12:03:11.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713fe94700 1 --2- 172.21.15.44:0/3744691495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f713813ac60 0x7f71380a3700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713fe94700 1 --2- 172.21.15.44:0/3744691495 >> v2:172.21.15.79:3300/0 conn(0x7f71380aa400 0x7f71380a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.780+0000 7f713fe94700 1 -- 172.21.15.44:0/3744691495 >> 172.21.15.44:0/3744691495 conn(0x7f7138062690 msgr2=0x7f71380a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:11.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.781+0000 7f713fe94700 1 -- 172.21.15.44:0/3744691495 shutdown_connections 2020-11-20T12:03:11.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.781+0000 7f713fe94700 1 -- 172.21.15.44:0/3744691495 wait complete. 2020-11-20T12:03:11.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.781+0000 7f713fe94700 1 Processor -- start 2020-11-20T12:03:11.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.781+0000 7f713fe94700 1 -- start start 2020-11-20T12:03:11.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713fe94700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71380aa020 0x7f7138131400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713fe94700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71380aa400 0x7f71380677a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713fe94700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f713813ac60 0x7f7138131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713fe94700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138133100 con 0x7f71380aa020 2020-11-20T12:03:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713fe94700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138135370 con 0x7f713813ac60 2020-11-20T12:03:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713fe94700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7138134160 con 0x7f71380aa400 2020-11-20T12:03:11.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71380aa020 0x7f7138131400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71380aa020 0x7f7138131400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36464/0 (socket says 172.21.15.44:36464) 2020-11-20T12:03:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 -- 172.21.15.44:0/668379388 learned_addr learned my addr 172.21.15.44:0/668379388 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713e431700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f713813ac60 0x7f7138131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713d42f700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71380aa400 0x7f71380677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 -- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71380aa400 msgr2=0x7f71380677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71380aa400 0x7f71380677a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 -- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f713813ac60 msgr2=0x7f7138131ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f713813ac60 0x7f7138131ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713dc30700 1 -- 172.21.15.44:0/668379388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7138067c90 con 0x7f71380aa020 2020-11-20T12:03:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.782+0000 7f713e431700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f713813ac60 0x7f7138131ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.783+0000 7f713dc30700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71380aa020 0x7f7138131400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f712800c710 tx=0x7f7128007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.783+0000 7f7125ffb700 1 -- 172.21.15.44:0/668379388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71280072e0 con 0x7f71380aa020 2020-11-20T12:03:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.783+0000 7f7125ffb700 1 -- 172.21.15.44:0/668379388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128007440 con 0x7f71380aa020 2020-11-20T12:03:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.783+0000 7f7125ffb700 1 -- 172.21.15.44:0/668379388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128005820 con 0x7f71380aa020 2020-11-20T12:03:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.783+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71381321d0 con 0x7f71380aa020 2020-11-20T12:03:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.783+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71380ada00 con 0x7f71380aa020 2020-11-20T12:03:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.784+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7138061650 con 0x7f71380aa020 2020-11-20T12:03:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.784+0000 7f7125ffb700 1 -- 172.21.15.44:0/668379388 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7128010070 con 0x7f71380aa020 2020-11-20T12:03:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.784+0000 7f7125ffb700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7110056d20 0x7f71100591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.784+0000 7f7125ffb700 1 -- 172.21.15.44:0/668379388 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f712806f530 con 0x7f71380aa020 2020-11-20T12:03:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.785+0000 7f713d42f700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7110056d20 0x7f71100591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.785+0000 7f713d42f700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7110056d20 0x7f71100591e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7134009440 tx=0x7f7134008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:11.788+0000 7f7125ffb700 1 -- 172.21.15.44:0/668379388 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7128044820 con 0x7f71380aa020 2020-11-20T12:03:12.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.029+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f71380a9770 con 0x7f71380aa020 2020-11-20T12:03:12.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.029+0000 7f7125ffb700 1 -- 172.21.15.44:0/668379388 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f71380a9770 con 0x7f71380aa020 2020-11-20T12:03:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:12.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.030+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7110056d20 msgr2=0x7f71100591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.030+0000 7f713fe94700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7110056d20 0x7f71100591e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7134009440 tx=0x7f7134008040).stop 2020-11-20T12:03:12.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.030+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71380aa020 msgr2=0x7f7138131400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.030+0000 7f713fe94700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71380aa020 0x7f7138131400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f712800c710 tx=0x7f7128007900).stop 2020-11-20T12:03:12.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 shutdown_connections 2020-11-20T12:03:12.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7110056d20 0x7f71100591e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f713813ac60 0x7f7138131ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71380aa020 0x7f7138131400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 --2- 172.21.15.44:0/668379388 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71380aa400 0x7f71380677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 >> 172.21.15.44:0/668379388 conn(0x7f7138062690 msgr2=0x7f71381369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:12.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 shutdown_connections 2020-11-20T12:03:12.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.031+0000 7f713fe94700 1 -- 172.21.15.44:0/668379388 wait complete. 2020-11-20T12:03:12.045 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-20T12:03:12.046 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-20T12:03:12.047 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:12.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.171+0000 7f0f15e1e700 1 Processor -- start 2020-11-20T12:03:12.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.171+0000 7f0f15e1e700 1 -- start start 2020-11-20T12:03:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.171+0000 7f0f15e1e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0f100aa020 0x7f0f100a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.171+0000 7f0f15e1e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa400 0x7f0f100a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.171+0000 7f0f15e1e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f100a9770 con 0x7f0f1013ac60 2020-11-20T12:03:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.171+0000 7f0f15e1e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f100a6da0 con 0x7f0f100aa020 2020-11-20T12:03:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.171+0000 7f0f15e1e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100a7040 con 0x7f0f100aa400 2020-11-20T12:03:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0f100aa020 0x7f0f100a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0effd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa400 0x7f0f100a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0ffff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0f1013ac60 0x7f0f100a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:43096/0 (socket says 172.21.15.44:43096) 2020-11-20T12:03:12.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0f100aa020 0x7f0f100a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:56754/0 (socket says 172.21.15.44:56754) 2020-11-20T12:03:12.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0f7fe700 1 -- 172.21.15.44:0/3871213768 learned_addr learned my addr 172.21.15.44:0/3871213768 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:12.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0effd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa400 0x7f0f100a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:47910/0 (socket says 172.21.15.44:47910) 2020-11-20T12:03:12.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0effd700 1 -- 172.21.15.44:0/3871213768 >> v2:172.21.15.79:3300/0 conn(0x7f0f100aa020 msgr2=0x7f0f100a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:12.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0effd700 1 --2- 172.21.15.44:0/3871213768 >> v2:172.21.15.79:3300/0 conn(0x7f0f100aa020 0x7f0f100a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0effd700 1 -- 172.21.15.44:0/3871213768 >> v1:172.21.15.79:6789/0 conn(0x7f0f1013ac60 legacy=0x7f0f100a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0effd700 1 -- 172.21.15.44:0/3871213768 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100a6a90 con 0x7f0f100aa400 2020-11-20T12:03:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0effd700 1 --2- 172.21.15.44:0/3871213768 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa400 0x7f0f100a4fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0ef800b9a0 tx=0x7f0ef8006df0).ready entity=mon.2 client_cookie=897dbfb21773d807 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0e7fc700 1 -- 172.21.15.44:0/3871213768 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80097b0 con 0x7f0f100aa400 2020-11-20T12:03:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0e7fc700 1 -- 172.21.15.44:0/3871213768 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef8009910 con 0x7f0f100aa400 2020-11-20T12:03:12.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.172+0000 7f0f0e7fc700 1 -- 172.21.15.44:0/3871213768 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8009c90 con 0x7f0f100aa400 2020-11-20T12:03:12.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 -- 172.21.15.44:0/3871213768 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa400 msgr2=0x7f0f100a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/3871213768 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa400 0x7f0f100a4fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f0ef800b9a0 tx=0x7f0ef8006df0).stop 2020-11-20T12:03:12.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 -- 172.21.15.44:0/3871213768 shutdown_connections 2020-11-20T12:03:12.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/3871213768 >> v2:172.21.15.79:3300/0 conn(0x7f0f100aa020 0x7f0f100a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/3871213768 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa400 0x7f0f100a4fb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 -- 172.21.15.44:0/3871213768 >> 172.21.15.44:0/3871213768 conn(0x7f0f10062690 msgr2=0x7f0f100a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 -- 172.21.15.44:0/3871213768 shutdown_connections 2020-11-20T12:03:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.173+0000 7f0f15e1e700 1 -- 172.21.15.44:0/3871213768 wait complete. 2020-11-20T12:03:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.174+0000 7f0f15e1e700 1 Processor -- start 2020-11-20T12:03:12.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.174+0000 7f0f15e1e700 1 -- start start 2020-11-20T12:03:12.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa020 0x7f0f100ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f100aa400 0x7f0f1014a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f1013ac60 0x7f0f10134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f10135300 con 0x7f0f100aa400 2020-11-20T12:03:12.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1014a0b0 con 0x7f0f1013ac60 2020-11-20T12:03:12.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1014a350 con 0x7f0f100aa020 2020-11-20T12:03:12.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f100aa400 0x7f0f1014a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f100aa400 0x7f0f1014a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36480/0 (socket says 172.21.15.44:36480) 2020-11-20T12:03:12.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 -- 172.21.15.44:0/308872728 learned_addr learned my addr 172.21.15.44:0/308872728 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:12.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0f7fe700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa020 0x7f0f100ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0ffff700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f1013ac60 0x7f0f10134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 -- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa020 msgr2=0x7f0f100ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa020 0x7f0f100ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 -- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f1013ac60 msgr2=0x7f0f10134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f1013ac60 0x7f0f10134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 -- 172.21.15.44:0/308872728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100ad670 con 0x7f0f100aa400 2020-11-20T12:03:12.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0ffff700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f1013ac60 0x7f0f10134690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:12.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f0effd700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f100aa400 0x7f0f1014a7f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f0ef8006e30 tx=0x7f0ef8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f077fe700 1 -- 172.21.15.44:0/308872728 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8011d50 con 0x7f0f100aa400 2020-11-20T12:03:12.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f077fe700 1 -- 172.21.15.44:0/308872728 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef801e460 con 0x7f0f100aa400 2020-11-20T12:03:12.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f077fe700 1 -- 172.21.15.44:0/308872728 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef801e7e0 con 0x7f0f100aa400 2020-11-20T12:03:12.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f100a1880 con 0x7f0f100aa400 2020-11-20T12:03:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.175+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f100abda0 con 0x7f0f100aa400 2020-11-20T12:03:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.176+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f100abf30 con 0x7f0f100aa400 2020-11-20T12:03:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.177+0000 7f0f077fe700 1 -- 172.21.15.44:0/308872728 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0ef8025030 con 0x7f0f100aa400 2020-11-20T12:03:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.177+0000 7f0f077fe700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0eec056d50 0x7f0eec059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.177+0000 7f0f077fe700 1 -- 172.21.15.44:0/308872728 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0ef8071790 con 0x7f0f100aa400 2020-11-20T12:03:12.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.177+0000 7f0f0f7fe700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0eec056d50 0x7f0eec059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.177+0000 7f0f0f7fe700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0eec056d50 0x7f0eec059210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f0f08009d90 tx=0x7f0f08008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.180+0000 7f0f077fe700 1 -- 172.21.15.44:0/308872728 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0ef8046b30 con 0x7f0f100aa400 2020-11-20T12:03:12.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.354+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f10136540 con 0x7f0f100aa400 2020-11-20T12:03:12.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.355+0000 7f0f077fe700 1 -- 172.21.15.44:0/308872728 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7f0ef80312a0 con 0x7f0f100aa400 2020-11-20T12:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:12.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0eec056d50 msgr2=0x7f0eec059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0eec056d50 0x7f0eec059210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f0f08009d90 tx=0x7f0f08008040).stop 2020-11-20T12:03:12.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f100aa400 msgr2=0x7f0f1014a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f100aa400 0x7f0f1014a7f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f0ef8006e30 tx=0x7f0ef8015000).stop 2020-11-20T12:03:12.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 shutdown_connections 2020-11-20T12:03:12.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0eec056d50 0x7f0eec059210 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0f1013ac60 0x7f0f10134690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0f100aa020 0x7f0f100ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 --2- 172.21.15.44:0/308872728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0f100aa400 0x7f0f1014a7f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 >> 172.21.15.44:0/308872728 conn(0x7f0f10062690 msgr2=0x7f0f100a3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 shutdown_connections 2020-11-20T12:03:12.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.357+0000 7f0f15e1e700 1 -- 172.21.15.44:0/308872728 wait complete. 2020-11-20T12:03:12.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:02:44.611733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:02:52.195125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:02:55.303534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:02:57.791405+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:03:12.373 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:03:12.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.498+0000 7f39cb208700 1 Processor -- start 2020-11-20T12:03:12.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.498+0000 7f39cb208700 1 -- start start 2020-11-20T12:03:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.498+0000 7f39cb208700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f39c40a4cf0 0x7f39c413a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.498+0000 7f39cb208700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c4067380 0x7f39c4067830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.498+0000 7f39cb208700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c413a800 con 0x7f39c40a9a60 2020-11-20T12:03:12.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.498+0000 7f39cb208700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c40a3ea0 con 0x7f39c40a4cf0 2020-11-20T12:03:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.498+0000 7f39cb208700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40a9320 con 0x7f39c4067380 2020-11-20T12:03:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39ca206700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f39c40a4cf0 0x7f39c413a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39caa07700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c4067380 0x7f39c4067830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39ca206700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f39c40a4cf0 0x7f39c413a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:56768/0 (socket says 172.21.15.44:56768) 2020-11-20T12:03:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39caa07700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c4067380 0x7f39c4067830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:47926/0 (socket says 172.21.15.44:47926) 2020-11-20T12:03:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39ca206700 1 -- 172.21.15.44:0/2906531349 learned_addr learned my addr 172.21.15.44:0/2906531349 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39c9a05700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f39c40a9a60 0x7f39c40a66e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:43114/0 (socket says 172.21.15.44:43114) 2020-11-20T12:03:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39caa07700 1 -- 172.21.15.44:0/2906531349 >> v2:172.21.15.79:3300/0 conn(0x7f39c40a4cf0 msgr2=0x7f39c413a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:12.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39caa07700 1 --2- 172.21.15.44:0/2906531349 >> v2:172.21.15.79:3300/0 conn(0x7f39c40a4cf0 0x7f39c413a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39caa07700 1 -- 172.21.15.44:0/2906531349 >> v1:172.21.15.79:6789/0 conn(0x7f39c40a9a60 legacy=0x7f39c40a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39caa07700 1 -- 172.21.15.44:0/2906531349 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c40a95c0 con 0x7f39c4067380 2020-11-20T12:03:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39ca206700 1 --2- 172.21.15.44:0/2906531349 >> v2:172.21.15.79:3300/0 conn(0x7f39c40a4cf0 0x7f39c413a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39caa07700 1 --2- 172.21.15.44:0/2906531349 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c4067380 0x7f39c4067830 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f39bc009c90 tx=0x7f39bc0091d0).ready entity=mon.2 client_cookie=bf786f5779ccc762 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39c9204700 1 -- 172.21.15.44:0/2906531349 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00ba70 con 0x7f39c4067380 2020-11-20T12:03:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39c9204700 1 -- 172.21.15.44:0/2906531349 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc00bbd0 con 0x7f39c4067380 2020-11-20T12:03:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.499+0000 7f39c9204700 1 -- 172.21.15.44:0/2906531349 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc011400 con 0x7f39c4067380 2020-11-20T12:03:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 -- 172.21.15.44:0/2906531349 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c4067380 msgr2=0x7f39c4067830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 --2- 172.21.15.44:0/2906531349 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c4067380 0x7f39c4067830 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f39bc009c90 tx=0x7f39bc0091d0).stop 2020-11-20T12:03:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 -- 172.21.15.44:0/2906531349 shutdown_connections 2020-11-20T12:03:12.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 --2- 172.21.15.44:0/2906531349 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c4067380 0x7f39c4067830 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 --2- 172.21.15.44:0/2906531349 >> v2:172.21.15.79:3300/0 conn(0x7f39c40a4cf0 0x7f39c413a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 -- 172.21.15.44:0/2906531349 >> 172.21.15.44:0/2906531349 conn(0x7f39c4061d80 msgr2=0x7f39c4060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:12.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 -- 172.21.15.44:0/2906531349 shutdown_connections 2020-11-20T12:03:12.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.500+0000 7f39cb208700 1 -- 172.21.15.44:0/2906531349 wait complete. 2020-11-20T12:03:12.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 Processor -- start 2020-11-20T12:03:12.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 -- start start 2020-11-20T12:03:12.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c4067380 0x7f39c412cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c40a4cf0 0x7f39c4133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c40a9a60 0x7f39c412ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40ad570 con 0x7f39c4067380 2020-11-20T12:03:12.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c4130cb0 con 0x7f39c40a4cf0 2020-11-20T12:03:12.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.501+0000 7f39cb208700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c4130f50 con 0x7f39c40a9a60 2020-11-20T12:03:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c4067380 0x7f39c412cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c4067380 0x7f39c412cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36492/0 (socket says 172.21.15.44:36492) 2020-11-20T12:03:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 -- 172.21.15.44:0/2355853425 learned_addr learned my addr 172.21.15.44:0/2355853425 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39c9a05700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c40a4cf0 0x7f39c4133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39caa07700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c40a9a60 0x7f39c412ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 -- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c40a9a60 msgr2=0x7f39c412ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c40a9a60 0x7f39c412ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 -- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c40a4cf0 msgr2=0x7f39c4133a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c40a4cf0 0x7f39c4133a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 -- 172.21.15.44:0/2355853425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4134ac0 con 0x7f39c4067380 2020-11-20T12:03:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39ca206700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c4067380 0x7f39c412cf80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f39c000ef20 tx=0x7f39c0006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39b27fc700 1 -- 172.21.15.44:0/2355853425 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000cbc0 con 0x7f39c4067380 2020-11-20T12:03:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39b27fc700 1 -- 172.21.15.44:0/2355853425 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c000cd20 con 0x7f39c4067380 2020-11-20T12:03:12.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39b27fc700 1 -- 172.21.15.44:0/2355853425 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0012600 con 0x7f39c4067380 2020-11-20T12:03:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c41308c0 con 0x7f39c4067380 2020-11-20T12:03:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.502+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c40ad9c0 con 0x7f39c4067380 2020-11-20T12:03:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.503+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c4131110 con 0x7f39c4067380 2020-11-20T12:03:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.504+0000 7f39b27fc700 1 -- 172.21.15.44:0/2355853425 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f39c0022070 con 0x7f39c4067380 2020-11-20T12:03:12.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.504+0000 7f39b27fc700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f399c056d50 0x7f399c059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.504+0000 7f39b27fc700 1 -- 172.21.15.44:0/2355853425 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f39c0070690 con 0x7f39c4067380 2020-11-20T12:03:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.504+0000 7f39c9a05700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f399c056d50 0x7f399c059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.504+0000 7f39c9a05700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f399c056d50 0x7f399c059210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f39c412dcc0 tx=0x7f39b4007b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.508+0000 7f39b27fc700 1 -- 172.21.15.44:0/2355853425 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f39c0045980 con 0x7f39c4067380 2020-11-20T12:03:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.681+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f39c4133f20 con 0x7f39c4067380 2020-11-20T12:03:12.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.681+0000 7f39b27fc700 1 -- 172.21.15.44:0/2355853425 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f39c0048fa0 con 0x7f39c4067380 2020-11-20T12:03:12.685 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2020-11-20T12:03:12.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f399c056d50 msgr2=0x7f399c059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f399c056d50 0x7f399c059210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f39c412dcc0 tx=0x7f39b4007b80).stop 2020-11-20T12:03:12.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c4067380 msgr2=0x7f39c412cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c4067380 0x7f39c412cf80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f39c000ef20 tx=0x7f39c0006eb0).stop 2020-11-20T12:03:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 shutdown_connections 2020-11-20T12:03:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f399c056d50 0x7f399c059210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c40a9a60 0x7f39c412ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c4067380 0x7f39c412cf80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 --2- 172.21.15.44:0/2355853425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c40a4cf0 0x7f39c4133a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 >> 172.21.15.44:0/2355853425 conn(0x7f39c4061d80 msgr2=0x7f39c413a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:12.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 shutdown_connections 2020-11-20T12:03:12.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.683+0000 7f39cb208700 1 -- 172.21.15.44:0/2355853425 wait complete. 2020-11-20T12:03:12.698 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:03:12.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.824+0000 7f44d23c7700 1 Processor -- start 2020-11-20T12:03:12.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.824+0000 7f44d23c7700 1 -- start start 2020-11-20T12:03:12.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44d23c7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f44cc0a3ad0 0x7f44cc1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44d23c7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc142d40 con 0x7f44cc13c580 2020-11-20T12:03:12.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44d23c7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc0a57f0 con 0x7f44cc1428c0 2020-11-20T12:03:12.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44d23c7700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44cc0a5a90 con 0x7f44cc0a3ad0 2020-11-20T12:03:12.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44cbfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f44cc0a3ad0 0x7f44cc1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44cbfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f44cc0a3ad0 0x7f44cc1423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36500/0 (socket says 172.21.15.44:36500) 2020-11-20T12:03:12.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44cbfff700 1 -- 172.21.15.44:0/4142245185 learned_addr learned my addr 172.21.15.44:0/4142245185 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:12.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44d0964700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f44cc13c580 0x7f44cc149770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40252/0 (socket says 172.21.15.44:40252) 2020-11-20T12:03:12.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44cbfff700 1 -- 172.21.15.44:0/4142245185 >> v1:172.21.15.79:6789/0 conn(0x7f44cc1428c0 legacy=0x7f44cc13be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:12.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44cbfff700 1 -- 172.21.15.44:0/4142245185 >> v1:172.21.15.44:6789/0 conn(0x7f44cc13c580 legacy=0x7f44cc149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44cbfff700 1 -- 172.21.15.44:0/4142245185 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0a5d30 con 0x7f44cc0a3ad0 2020-11-20T12:03:12.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.825+0000 7f44cbfff700 1 --2- 172.21.15.44:0/4142245185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc0a3ad0 0x7f44cc1423d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f44bc006fd0 tx=0x7f44bc006eb0).ready entity=mon.0 client_cookie=3a3c22729b038d47 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44caffd700 1 -- 172.21.15.44:0/4142245185 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0097d0 con 0x7f44cc0a3ad0 2020-11-20T12:03:12.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44caffd700 1 -- 172.21.15.44:0/4142245185 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc009930 con 0x7f44cc0a3ad0 2020-11-20T12:03:12.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44caffd700 1 -- 172.21.15.44:0/4142245185 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc009cb0 con 0x7f44cc0a3ad0 2020-11-20T12:03:12.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44d23c7700 1 -- 172.21.15.44:0/4142245185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc0a3ad0 msgr2=0x7f44cc1423d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44d23c7700 1 --2- 172.21.15.44:0/4142245185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc0a3ad0 0x7f44cc1423d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f44bc006fd0 tx=0x7f44bc006eb0).stop 2020-11-20T12:03:12.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44d23c7700 1 -- 172.21.15.44:0/4142245185 shutdown_connections 2020-11-20T12:03:12.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44d23c7700 1 --2- 172.21.15.44:0/4142245185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc0a3ad0 0x7f44cc1423d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44d23c7700 1 -- 172.21.15.44:0/4142245185 >> 172.21.15.44:0/4142245185 conn(0x7f44cc0624b0 msgr2=0x7f44cc0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:12.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44d23c7700 1 -- 172.21.15.44:0/4142245185 shutdown_connections 2020-11-20T12:03:12.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.826+0000 7f44d23c7700 1 -- 172.21.15.44:0/4142245185 wait complete. 2020-11-20T12:03:12.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.827+0000 7f44d23c7700 1 Processor -- start 2020-11-20T12:03:12.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.827+0000 7f44d23c7700 1 -- start start 2020-11-20T12:03:12.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d23c7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc0a3ad0 0x7f44cc0aac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d23c7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc13c580 0x7f44cc151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d23c7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44cc1428c0 0x7f44cc0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d23c7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0acb50 con 0x7f44cc13c580 2020-11-20T12:03:12.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d23c7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0ab2b0 con 0x7f44cc0a3ad0 2020-11-20T12:03:12.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d23c7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0ab550 con 0x7f44cc1428c0 2020-11-20T12:03:12.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc13c580 0x7f44cc151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc13c580 0x7f44cc151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36506/0 (socket says 172.21.15.44:36506) 2020-11-20T12:03:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 -- 172.21.15.44:0/716213202 learned_addr learned my addr 172.21.15.44:0/716213202 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d0964700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44cc1428c0 0x7f44cc0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cbfff700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc0a3ad0 0x7f44cc0aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 -- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44cc1428c0 msgr2=0x7f44cc0abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44cc1428c0 0x7f44cc0abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 -- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc0a3ad0 msgr2=0x7f44cc0aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc0a3ad0 0x7f44cc0aac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 -- 172.21.15.44:0/716213202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0ab7f0 con 0x7f44cc13c580 2020-11-20T12:03:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44d0964700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44cc1428c0 0x7f44cc0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:12.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44cb7fe700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc13c580 0x7f44cc151b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f44c00096b0 tx=0x7f44c0006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.828+0000 7f44abfff700 1 -- 172.21.15.44:0/716213202 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c000d560 con 0x7f44cc13c580 2020-11-20T12:03:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.829+0000 7f44abfff700 1 -- 172.21.15.44:0/716213202 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c0000e90 con 0x7f44cc13c580 2020-11-20T12:03:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.829+0000 7f44abfff700 1 -- 172.21.15.44:0/716213202 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c000d910 con 0x7f44cc13c580 2020-11-20T12:03:12.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.829+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc0ac180 con 0x7f44cc13c580 2020-11-20T12:03:12.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.829+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc152640 con 0x7f44cc13c580 2020-11-20T12:03:12.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.829+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc0ab980 con 0x7f44cc13c580 2020-11-20T12:03:12.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.830+0000 7f44abfff700 1 -- 172.21.15.44:0/716213202 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f44c000f030 con 0x7f44cc13c580 2020-11-20T12:03:12.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.830+0000 7f44abfff700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f44a4056d20 0x7f44a40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:12.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.830+0000 7f44abfff700 1 -- 172.21.15.44:0/716213202 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f44c006fd60 con 0x7f44cc13c580 2020-11-20T12:03:12.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.830+0000 7f44cbfff700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f44a4056d20 0x7f44a40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:12.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.831+0000 7f44cbfff700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f44a4056d20 0x7f44a40591e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f44bc006c40 tx=0x7f44bc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:12.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:12.834+0000 7f44abfff700 1 -- 172.21.15.44:0/716213202 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f44c0045050 con 0x7f44cc13c580 2020-11-20T12:03:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.011+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44cc0ac310 con 0x7f44cc13c580 2020-11-20T12:03:13.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.011+0000 7f44abfff700 1 -- 172.21.15.44:0/716213202 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f44c0048670 con 0x7f44cc13c580 2020-11-20T12:03:13.015 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f44a4056d20 msgr2=0x7f44a40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f44a4056d20 0x7f44a40591e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f44bc006c40 tx=0x7f44bc015000).stop 2020-11-20T12:03:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc13c580 msgr2=0x7f44cc151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc13c580 0x7f44cc151b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f44c00096b0 tx=0x7f44c0006d50).stop 2020-11-20T12:03:13.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 shutdown_connections 2020-11-20T12:03:13.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f44a4056d20 0x7f44a40591e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44cc1428c0 0x7f44cc0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44cc0a3ad0 0x7f44cc0aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 --2- 172.21.15.44:0/716213202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44cc13c580 0x7f44cc151b30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 >> 172.21.15.44:0/716213202 conn(0x7f44cc0624b0 msgr2=0x7f44cc143490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 shutdown_connections 2020-11-20T12:03:13.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.013+0000 7f44d23c7700 1 -- 172.21.15.44:0/716213202 wait complete. 2020-11-20T12:03:13.028 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:03:13.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.153+0000 7fd445cbd700 1 Processor -- start 2020-11-20T12:03:13.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.153+0000 7fd445cbd700 1 -- start start 2020-11-20T12:03:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd445cbd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd4400a8ea0 0x7fd4401430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd445cbd700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4400a7200 con 0x7fd4400ad930 2020-11-20T12:03:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd445cbd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd440135240 con 0x7fd4400ad550 2020-11-20T12:03:13.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd445cbd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4400a6cc0 con 0x7fd4400a8ea0 2020-11-20T12:03:13.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43effd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd4400ad930 0x7fd4400a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40262/0 (socket says 172.21.15.44:40262) 2020-11-20T12:03:13.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43effd700 1 -- 172.21.15.44:0/2858715692 learned_addr learned my addr 172.21.15.44:0/2858715692 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:13.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43ffff700 1 --2- 172.21.15.44:0/2858715692 >> v2:172.21.15.79:3300/0 conn(0x7fd4400a8ea0 0x7fd4401430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205000330 0 0) 0x7fd4400a7200 con 0x7fd4400ad930 2020-11-20T12:03:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd42c003240 con 0x7fd4400ad930 2020-11-20T12:03:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 670190575 0 0) 0x7fd42c003240 con 0x7fd4400ad930 2020-11-20T12:03:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 >> v2:172.21.15.79:3300/0 conn(0x7fd4400a8ea0 msgr2=0x7fd4401430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 --2- 172.21.15.44:0/2858715692 >> v2:172.21.15.79:3300/0 conn(0x7fd4400a8ea0 0x7fd4401430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 >> v1:172.21.15.79:6789/0 conn(0x7fd4400ad550 legacy=0x7fd4400a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400a6f60 con 0x7fd4400ad930 2020-11-20T12:03:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fd4340014e0 con 0x7fd4400ad930 2020-11-20T12:03:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 >> v1:172.21.15.44:6789/0 conn(0x7fd4400ad930 legacy=0x7fd4400a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c003730 0x7fd42c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42c0044e0 0x7fd42c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c004e50 0x7fd42c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400a6cc0 con 0x7fd42c0044e0 2020-11-20T12:03:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c004260 con 0x7fd42c003730 2020-11-20T12:03:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c00bc20 con 0x7fd42c004e50 2020-11-20T12:03:13.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.154+0000 7fd43effd700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42c0044e0 0x7fd42c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43f7fe700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c003730 0x7fd42c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43ffff700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c004e50 0x7fd42c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43effd700 1 -- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c004e50 msgr2=0x7fd42c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:13.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43effd700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c004e50 0x7fd42c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43effd700 1 -- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c003730 msgr2=0x7fd42c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43effd700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd42c003730 0x7fd42c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43effd700 1 -- 172.21.15.44:0/2858715692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c003240 con 0x7fd42c0044e0 2020-11-20T12:03:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43ffff700 1 -- 172.21.15.44:0/2858715692 reap_dead start 2020-11-20T12:03:13.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43effd700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42c0044e0 0x7fd42c004960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd43400c410 tx=0x7fd4400a5d30).ready entity=mon.0 client_cookie=555781297264f0a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4340070d0 con 0x7fd42c0044e0 2020-11-20T12:03:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434007230 con 0x7fd42c0044e0 2020-11-20T12:03:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd43e7fc700 1 -- 172.21.15.44:0/2858715692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4340055f0 con 0x7fd42c0044e0 2020-11-20T12:03:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd445cbd700 1 -- 172.21.15.44:0/2858715692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42c0044e0 msgr2=0x7fd42c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd445cbd700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42c0044e0 0x7fd42c004960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd43400c410 tx=0x7fd4400a5d30).stop 2020-11-20T12:03:13.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd445cbd700 1 -- 172.21.15.44:0/2858715692 shutdown_connections 2020-11-20T12:03:13.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd445cbd700 1 --2- 172.21.15.44:0/2858715692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd42c0044e0 0x7fd42c004960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.155+0000 7fd445cbd700 1 -- 172.21.15.44:0/2858715692 >> 172.21.15.44:0/2858715692 conn(0x7fd4400624b0 msgr2=0x7fd4400ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:13.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.156+0000 7fd445cbd700 1 -- 172.21.15.44:0/2858715692 shutdown_connections 2020-11-20T12:03:13.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.156+0000 7fd445cbd700 1 -- 172.21.15.44:0/2858715692 wait complete. 2020-11-20T12:03:13.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 Processor -- start 2020-11-20T12:03:13.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 -- start start 2020-11-20T12:03:13.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c0044e0 0x7fd4400603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd440066790 0x7fd4400a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4400a2420 0x7fd44013a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400a28a0 con 0x7fd4400a2420 2020-11-20T12:03:13.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400a32f0 con 0x7fd440066790 2020-11-20T12:03:13.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd445cbd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400a3590 con 0x7fd42c0044e0 2020-11-20T12:03:13.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd43ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4400a2420 0x7fd44013a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd43ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4400a2420 0x7fd44013a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36528/0 (socket says 172.21.15.44:36528) 2020-11-20T12:03:13.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd43ffff700 1 -- 172.21.15.44:0/394229770 learned_addr learned my addr 172.21.15.44:0/394229770 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:13.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd43effd700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd440066790 0x7fd4400a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.157+0000 7fd43f7fe700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c0044e0 0x7fd4400603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd43ffff700 1 -- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c0044e0 msgr2=0x7fd4400603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd43ffff700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c0044e0 0x7fd4400603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd43ffff700 1 -- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd440066790 msgr2=0x7fd4400a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd43ffff700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd440066790 0x7fd4400a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd43ffff700 1 -- 172.21.15.44:0/394229770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400a2f70 con 0x7fd4400a2420 2020-11-20T12:03:13.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd43ffff700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4400a2420 0x7fd44013a800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fd4380090e0 tx=0x7fd4380091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd41f7fe700 1 -- 172.21.15.44:0/394229770 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43800faa0 con 0x7fd4400a2420 2020-11-20T12:03:13.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd41f7fe700 1 -- 172.21.15.44:0/394229770 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43800fc00 con 0x7fd4400a2420 2020-11-20T12:03:13.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd41f7fe700 1 -- 172.21.15.44:0/394229770 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438014440 con 0x7fd4400a2420 2020-11-20T12:03:13.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd440060c40 con 0x7fd4400a2420 2020-11-20T12:03:13.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.158+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44013aeb0 con 0x7fd4400a2420 2020-11-20T12:03:13.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.159+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4400a3160 con 0x7fd4400a2420 2020-11-20T12:03:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.159+0000 7fd41f7fe700 1 -- 172.21.15.44:0/394229770 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd438014830 con 0x7fd4400a2420 2020-11-20T12:03:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.159+0000 7fd41f7fe700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd418056d20 0x7fd4180591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.159+0000 7fd41f7fe700 1 -- 172.21.15.44:0/394229770 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd438070bc0 con 0x7fd4400a2420 2020-11-20T12:03:13.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.159+0000 7fd43f7fe700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd418056d20 0x7fd4180591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.160+0000 7fd43f7fe700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd418056d20 0x7fd4180591e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd430007fd0 tx=0x7fd430007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.163+0000 7fd41f7fe700 1 -- 172.21.15.44:0/394229770 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd438045eb0 con 0x7fd4400a2420 2020-11-20T12:03:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.338+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4400615e0 con 0x7fd4400a2420 2020-11-20T12:03:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.338+0000 7fd41f7fe700 1 -- 172.21.15.44:0/394229770 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd4380494d0 con 0x7fd4400a2420 2020-11-20T12:03:13.343 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd418056d20 msgr2=0x7fd4180591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd418056d20 0x7fd4180591e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd430007fd0 tx=0x7fd430007b90).stop 2020-11-20T12:03:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4400a2420 msgr2=0x7fd44013a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4400a2420 0x7fd44013a800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fd4380090e0 tx=0x7fd4380091d0).stop 2020-11-20T12:03:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 shutdown_connections 2020-11-20T12:03:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd418056d20 0x7fd4180591e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4400a2420 0x7fd44013a800 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd42c0044e0 0x7fd4400603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 --2- 172.21.15.44:0/394229770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd440066790 0x7fd4400a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 >> 172.21.15.44:0/394229770 conn(0x7fd4400624b0 msgr2=0x7fd440136fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:13.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 shutdown_connections 2020-11-20T12:03:13.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.340+0000 7fd445cbd700 1 -- 172.21.15.44:0/394229770 wait complete. 2020-11-20T12:03:13.355 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:03:13.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.480+0000 7f2034054700 1 Processor -- start 2020-11-20T12:03:13.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.480+0000 7f2034054700 1 -- start start 2020-11-20T12:03:13.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2034054700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 0x7f202c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2034054700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c0a7200 con 0x7f202c0a8ea0 2020-11-20T12:03:13.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2034054700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c135240 con 0x7f202c0ad550 2020-11-20T12:03:13.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2034054700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0a6cc0 con 0x7f202c0ad930 2020-11-20T12:03:13.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f20325f1700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f202c0a8ea0 0x7f202c1420b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40284/0 (socket says 172.21.15.44:40284) 2020-11-20T12:03:13.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f20325f1700 1 -- 172.21.15.44:0/1933726631 learned_addr learned my addr 172.21.15.44:0/1933726631 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:13.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581375227 0 0) 0x7f202c0a7200 con 0x7f202c0a8ea0 2020-11-20T12:03:13.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f20315ef700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 0x7f202c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2014003240 con 0x7f202c0a8ea0 2020-11-20T12:03:13.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1621597039 0 0) 0x7f2014003240 con 0x7f202c0a8ea0 2020-11-20T12:03:13.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 msgr2=0x7f202c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:13.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 0x7f202c0a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 >> v1:172.21.15.79:6789/0 conn(0x7f202c0ad550 legacy=0x7f202c0a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0a6f60 con 0x7f202c0a8ea0 2020-11-20T12:03:13.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f2024001860 con 0x7f202c0a8ea0 2020-11-20T12:03:13.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.481+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 >> v1:172.21.15.44:6789/0 conn(0x7f202c0a8ea0 legacy=0x7f202c1420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 0x7f2014003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2014003c50 0x7f20140044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140049d0 0x7f2014004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0a6cc0 con 0x7f20140049d0 2020-11-20T12:03:13.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014004170 con 0x7f2014003c50 2020-11-20T12:03:13.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2014004330 con 0x7f202c0ad930 2020-11-20T12:03:13.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20325f1700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140049d0 0x7f2014004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2031df0700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 0x7f2014003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20315ef700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2014003c50 0x7f20140044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20325f1700 1 -- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 msgr2=0x7f2014003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:13.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20325f1700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0ad930 0x7f2014003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20325f1700 1 -- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2014003c50 msgr2=0x7f20140044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:13.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20325f1700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2014003c50 0x7f20140044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20325f1700 1 -- 172.21.15.44:0/1933726631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014003240 con 0x7f20140049d0 2020-11-20T12:03:13.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f20325f1700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140049d0 0x7f2014004e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f202400c600 tx=0x7f202c0a70f0).ready entity=mon.0 client_cookie=8f25485174cc846 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024005b80 con 0x7f20140049d0 2020-11-20T12:03:13.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2024005ce0 con 0x7f20140049d0 2020-11-20T12:03:13.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.482+0000 7f2030dee700 1 -- 172.21.15.44:0/1933726631 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024007320 con 0x7f20140049d0 2020-11-20T12:03:13.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.483+0000 7f2034054700 1 -- 172.21.15.44:0/1933726631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140049d0 msgr2=0x7f2014004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.483+0000 7f2034054700 1 --2- 172.21.15.44:0/1933726631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20140049d0 0x7f2014004e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f202400c600 tx=0x7f202c0a70f0).stop 2020-11-20T12:03:13.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.483+0000 7f20325f1700 1 -- 172.21.15.44:0/1933726631 reap_dead start 2020-11-20T12:03:13.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.483+0000 7f2034054700 1 -- 172.21.15.44:0/1933726631 shutdown_connections 2020-11-20T12:03:13.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.483+0000 7f2034054700 1 -- 172.21.15.44:0/1933726631 >> 172.21.15.44:0/1933726631 conn(0x7f202c0624b0 msgr2=0x7f202c0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:13.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.483+0000 7f2034054700 1 -- 172.21.15.44:0/1933726631 shutdown_connections 2020-11-20T12:03:13.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.483+0000 7f2034054700 1 -- 172.21.15.44:0/1933726631 wait complete. 2020-11-20T12:03:13.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 Processor -- start 2020-11-20T12:03:13.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 -- start start 2020-11-20T12:03:13.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0678b0 0x7f202c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f202c139b70 0x7f202c139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c13a4e0 0x7f202c133230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c09d7b0 con 0x7f202c13a4e0 2020-11-20T12:03:13.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c13ce80 con 0x7f202c139b70 2020-11-20T12:03:13.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2034054700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c13d120 con 0x7f202c0678b0 2020-11-20T12:03:13.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f20325f1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c13a4e0 0x7f202c133230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f20325f1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c13a4e0 0x7f202c133230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36548/0 (socket says 172.21.15.44:36548) 2020-11-20T12:03:13.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f2031df0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0678b0 0x7f202c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f20325f1700 1 -- 172.21.15.44:0/3837671740 learned_addr learned my addr 172.21.15.44:0/3837671740 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:13.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.484+0000 7f20315ef700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f202c139b70 0x7f202c139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f20325f1700 1 -- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0678b0 msgr2=0x7f202c148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f20325f1700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0678b0 0x7f202c148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f20325f1700 1 -- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f202c139b70 msgr2=0x7f202c139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f20325f1700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f202c139b70 0x7f202c139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f20325f1700 1 -- 172.21.15.44:0/3837671740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c067c90 con 0x7f202c13a4e0 2020-11-20T12:03:13.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f20325f1700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c13a4e0 0x7f202c133230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2024000f40 tx=0x7f2024015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f2019ffb700 1 -- 172.21.15.44:0/3837671740 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024005b80 con 0x7f202c13a4e0 2020-11-20T12:03:13.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f2019ffb700 1 -- 172.21.15.44:0/3837671740 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20240072e0 con 0x7f202c13a4e0 2020-11-20T12:03:13.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c13a920 con 0x7f202c13a4e0 2020-11-20T12:03:13.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f2019ffb700 1 -- 172.21.15.44:0/3837671740 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024001860 con 0x7f202c13a4e0 2020-11-20T12:03:13.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.485+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c13d5c0 con 0x7f202c13a4e0 2020-11-20T12:03:13.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.486+0000 7f2019ffb700 1 -- 172.21.15.44:0/3837671740 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2024010070 con 0x7f202c13a4e0 2020-11-20T12:03:13.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.486+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c0615e0 con 0x7f202c13a4e0 2020-11-20T12:03:13.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.487+0000 7f2019ffb700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2004056d20 0x7f20040591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.487+0000 7f2019ffb700 1 -- 172.21.15.44:0/3837671740 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2024071250 con 0x7f202c13a4e0 2020-11-20T12:03:13.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.487+0000 7f2031df0700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2004056d20 0x7f20040591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.487+0000 7f2031df0700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2004056d20 0x7f20040591e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f201c007fd0 tx=0x7f201c007b70).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.490+0000 7f2019ffb700 1 -- 172.21.15.44:0/3837671740 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2024046540 con 0x7f202c13a4e0 2020-11-20T12:03:13.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.663+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f202c0446d0 con 0x7f202c13a4e0 2020-11-20T12:03:13.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.664+0000 7f2019ffb700 1 -- 172.21.15.44:0/3837671740 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2024049b60 con 0x7f202c13a4e0 2020-11-20T12:03:13.671 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:13.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2004056d20 msgr2=0x7f20040591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2004056d20 0x7f20040591e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f201c007fd0 tx=0x7f201c007b70).stop 2020-11-20T12:03:13.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c13a4e0 msgr2=0x7f202c133230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c13a4e0 0x7f202c133230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2024000f40 tx=0x7f2024015000).stop 2020-11-20T12:03:13.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 shutdown_connections 2020-11-20T12:03:13.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2004056d20 0x7f20040591e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f202c13a4e0 0x7f202c133230 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f202c0678b0 0x7f202c148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 --2- 172.21.15.44:0/3837671740 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f202c139b70 0x7f202c139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.665+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 >> 172.21.15.44:0/3837671740 conn(0x7f202c0624b0 msgr2=0x7f202c0ac6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:13.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.666+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 shutdown_connections 2020-11-20T12:03:13.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.666+0000 7f2034054700 1 -- 172.21.15.44:0/3837671740 wait complete. 2020-11-20T12:03:13.681 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:03:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.810+0000 7f20f3e53700 1 Processor -- start 2020-11-20T12:03:13.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.810+0000 7f20f3e53700 1 -- start start 2020-11-20T12:03:13.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f3e53700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f20ec13a990 0x7f20ec13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f3e53700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f3e53700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec0a6260 con 0x7f20ec060330 2020-11-20T12:03:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f3e53700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec0a52e0 con 0x7f20ec13a990 2020-11-20T12:03:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f3e53700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0a6490 con 0x7f20ec13f800 2020-11-20T12:03:13.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f23f0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f13ee700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f20ec13a990 0x7f20ec13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f13ee700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f20ec13a990 0x7f20ec13ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:56834/0 (socket says 172.21.15.44:56834) 2020-11-20T12:03:13.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.811+0000 7f20f13ee700 1 -- 172.21.15.44:0/2038840889 learned_addr learned my addr 172.21.15.44:0/2038840889 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:13.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f13ee700 1 -- 172.21.15.44:0/2038840889 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 msgr2=0x7f20ec141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:13.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f13ee700 1 --2- 172.21.15.44:0/2038840889 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f13ee700 1 -- 172.21.15.44:0/2038840889 >> v1:172.21.15.79:6789/0 conn(0x7f20ec060330 legacy=0x7f20ec0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f0bed700 1 -- 172.21.15.44:0/2038840889 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564692653 0 0) 0x7f20ec0a6260 con 0x7f20ec060330 2020-11-20T12:03:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f13ee700 1 -- 172.21.15.44:0/2038840889 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0a6730 con 0x7f20ec13a990 2020-11-20T12:03:13.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f13ee700 1 --2- 172.21.15.44:0/2038840889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec13a990 0x7f20ec13ae20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f20e800ba20 tx=0x7f20e8006eb0).ready entity=mon.1 client_cookie=fdbcbe606dfe3fcd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f0bed700 1 -- 172.21.15.44:0/2038840889 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80097d0 con 0x7f20ec13a990 2020-11-20T12:03:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f0bed700 1 -- 172.21.15.44:0/2038840889 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8009930 con 0x7f20ec13a990 2020-11-20T12:03:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f0bed700 1 -- 172.21.15.44:0/2038840889 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8009cb0 con 0x7f20ec13a990 2020-11-20T12:03:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f3e53700 1 -- 172.21.15.44:0/2038840889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec13a990 msgr2=0x7f20ec13ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.812+0000 7f20f3e53700 1 --2- 172.21.15.44:0/2038840889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec13a990 0x7f20ec13ae20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f20e800ba20 tx=0x7f20e8006eb0).stop 2020-11-20T12:03:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.813+0000 7f20f3e53700 1 -- 172.21.15.44:0/2038840889 shutdown_connections 2020-11-20T12:03:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.813+0000 7f20f3e53700 1 --2- 172.21.15.44:0/2038840889 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.813+0000 7f20f3e53700 1 --2- 172.21.15.44:0/2038840889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec13a990 0x7f20ec13ae20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.813+0000 7f20f3e53700 1 -- 172.21.15.44:0/2038840889 >> 172.21.15.44:0/2038840889 conn(0x7f20ec062690 msgr2=0x7f20ec0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:13.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.813+0000 7f20f3e53700 1 -- 172.21.15.44:0/2038840889 shutdown_connections 2020-11-20T12:03:13.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.813+0000 7f20f3e53700 1 -- 172.21.15.44:0/2038840889 wait complete. 2020-11-20T12:03:13.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 Processor -- start 2020-11-20T12:03:13.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 -- start start 2020-11-20T12:03:13.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec060330 0x7f20ec0aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20ec13a990 0x7f20ec135b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1364b0 con 0x7f20ec13a990 2020-11-20T12:03:13.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec136750 con 0x7f20ec060330 2020-11-20T12:03:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f3e53700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec136a70 con 0x7f20ec13f800 2020-11-20T12:03:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f13ee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20ec13a990 0x7f20ec135b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f13ee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20ec13a990 0x7f20ec135b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36558/0 (socket says 172.21.15.44:36558) 2020-11-20T12:03:13.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f13ee700 1 -- 172.21.15.44:0/555722529 learned_addr learned my addr 172.21.15.44:0/555722529 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f23f0700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.814+0000 7f20f1bef700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec060330 0x7f20ec0aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f13ee700 1 -- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 msgr2=0x7f20ec136f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f13ee700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec136f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f13ee700 1 -- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec060330 msgr2=0x7f20ec0aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f13ee700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec060330 0x7f20ec0aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f13ee700 1 -- 172.21.15.44:0/555722529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec135130 con 0x7f20ec13a990 2020-11-20T12:03:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f13ee700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20ec13a990 0x7f20ec135b30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f20e800bee0 tx=0x7f20e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20d9ffb700 1 -- 172.21.15.44:0/555722529 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011d50 con 0x7f20ec13a990 2020-11-20T12:03:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20d9ffb700 1 -- 172.21.15.44:0/555722529 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e801e460 con 0x7f20ec13a990 2020-11-20T12:03:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec136020 con 0x7f20ec13a990 2020-11-20T12:03:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20d9ffb700 1 -- 172.21.15.44:0/555722529 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e801e800 con 0x7f20ec13a990 2020-11-20T12:03:13.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.815+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec137640 con 0x7f20ec13a990 2020-11-20T12:03:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.816+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec1362f0 con 0x7f20ec13a990 2020-11-20T12:03:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.816+0000 7f20d9ffb700 1 -- 172.21.15.44:0/555722529 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f20e801a040 con 0x7f20ec13a990 2020-11-20T12:03:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.817+0000 7f20d9ffb700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20c4056d20 0x7f20c40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.817+0000 7f20d9ffb700 1 -- 172.21.15.44:0/555722529 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f20e80714c0 con 0x7f20ec13a990 2020-11-20T12:03:13.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.817+0000 7f20f1bef700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20c4056d20 0x7f20c40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.817+0000 7f20f1bef700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20c4056d20 0x7f20c40591e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f20dc007fd0 tx=0x7f20dc007a00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:13.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:13.820+0000 7f20d9ffb700 1 -- 172.21.15.44:0/555722529 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f20e80467b0 con 0x7f20ec13a990 2020-11-20T12:03:14.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.052+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20ec136c30 con 0x7f20ec13a990 2020-11-20T12:03:14.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.052+0000 7f20d9ffb700 1 -- 172.21.15.44:0/555722529 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2131 (secure 0 0 0) 0x7f20e8023020 con 0x7f20ec13a990 2020-11-20T12:03:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:14.058 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/4254760701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":4254760701},{"type":"v1","addr":"172.21.15.79:6835","nonce":4254760701}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:03:00.178663+0000","modified":"2020-11-20T12:03:03.184476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/2557888073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2557888073},{"type":"v1","addr":"172.21.15.44:6833","nonce":2557888073}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:03:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20c4056d20 msgr2=0x7f20c40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20c4056d20 0x7f20c40591e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f20dc007fd0 tx=0x7f20dc007a00).stop 2020-11-20T12:03:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20ec13a990 msgr2=0x7f20ec135b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:14.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20ec13a990 0x7f20ec135b30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f20e800bee0 tx=0x7f20e8015000).stop 2020-11-20T12:03:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 shutdown_connections 2020-11-20T12:03:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20c4056d20 0x7f20c40591e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20ec13f800 0x7f20ec136f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20ec060330 0x7f20ec0aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:14.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 --2- 172.21.15.44:0/555722529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20ec13a990 0x7f20ec135b30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 >> 172.21.15.44:0/555722529 conn(0x7f20ec062690 msgr2=0x7f20ec134410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 shutdown_connections 2020-11-20T12:03:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:14.054+0000 7f20f3e53700 1 -- 172.21.15.44:0/555722529 wait complete. 2020-11-20T12:03:14.061 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2020-11-20T12:03:14.271 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:03:00.178663+0000', 'modified': '2020-11-20T12:03:03.184476+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6833/2557888073', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 2557888073}, {'type': 'v1', 'addr': '172.21.15.44:6833', 'nonce': 2557888073}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:03:14.271 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:03:14.272 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:03:14.424 INFO:teuthology.orchestra.run.smithi044.stdout:{"cluster_fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","whoami":0,"id":4284,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.232080205999999,"mdsmap_epoch":7,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":20.688021558999999} 2020-11-20T12:03:14.424 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", "fs_name": "cephfs", "id": 4284, "mdsmap_epoch": 7, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 14.232080206, "state": "up:active", "uptime": 20.688021559, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:03:14.425 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-11-20T12:03:14.451 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-11-20T12:03:14.453 INFO:tasks.ceph_fuse:config is None 2020-11-20T12:03:14.453 INFO:tasks.ceph_fuse:new config is {'client.0': None} 2020-11-20T12:03:14.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:14.455 INFO:teuthology.orchestra.run.smithi044:> ip netns list 2020-11-20T12:03:14.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:14.477 INFO:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2020-11-20T12:03:14.552 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2020-11-20T12:03:14.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:03:14.554 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-11-20T12:03:14.554 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:03:14.555 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2020-11-20T12:03:14.555 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:03:14.555 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:03:14.556 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:03:14.556 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-20T12:03:14.556 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:03:14.556 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:03:14.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:14.557 INFO:teuthology.orchestra.run.smithi044:> ip addr 2020-11-20T12:03:14.614 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:03:14.615 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:03:14.616 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:03:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:03:14.617 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2020-11-20T12:03:14.618 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:03:14.618 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-11-20T12:03:14.619 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2020-11-20T12:03:14.620 INFO:teuthology.orchestra.run.smithi044.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:03:14.620 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:03:14.621 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2020-11-20T12:03:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42806sec preferred_lft 42806sec 2020-11-20T12:03:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link 2020-11-20T12:03:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:03:14.623 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-11-20T12:03:14.623 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2020-11-20T12:03:14.624 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2020-11-20T12:03:14.624 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:03:14.624 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-11-20T12:03:14.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:14.625 INFO:teuthology.orchestra.run.smithi044:> (cd / && exec sudo bash -c ' 2020-11-20T12:03:14.626 INFO:teuthology.orchestra.run.smithi044:> set -e 2020-11-20T12:03:14.626 INFO:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2020-11-20T12:03:14.626 INFO:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2020-11-20T12:03:14.627 INFO:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2020-11-20T12:03:14.627 INFO:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2020-11-20T12:03:14.627 INFO:teuthology.orchestra.run.smithi044:> ') 2020-11-20T12:03:14.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:14.811 INFO:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:03:14.851 INFO:teuthology.orchestra.run.smithi044.stdout:1 2020-11-20T12:03:14.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:14.852 INFO:teuthology.orchestra.run.smithi044:> route 2020-11-20T12:03:14.927 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2020-11-20T12:03:14.927 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:03:14.928 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 103 0 0 ens1f0 2020-11-20T12:03:14.928 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 103 0 0 ens1f0 2020-11-20T12:03:14.928 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:03:14.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:14.930 INFO:teuthology.orchestra.run.smithi044:> (cd / && exec sudo bash -c ' 2020-11-20T12:03:14.930 INFO:teuthology.orchestra.run.smithi044:> set -e 2020-11-20T12:03:14.931 INFO:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2020-11-20T12:03:14.931 INFO:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2020-11-20T12:03:14.932 INFO:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2020-11-20T12:03:14.932 INFO:teuthology.orchestra.run.smithi044:> ') 2020-11-20T12:03:15.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:15.187 INFO:teuthology.orchestra.run.smithi044:> ip netns list 2020-11-20T12:03:15.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:15.209 INFO:teuthology.orchestra.run.smithi044:> ip netns list-id 2020-11-20T12:03:15.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:15.272 INFO:teuthology.orchestra.run.smithi044:> (cd / && exec sudo bash -c ' 2020-11-20T12:03:15.272 INFO:teuthology.orchestra.run.smithi044:> set -e 2020-11-20T12:03:15.272 INFO:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2020-11-20T12:03:15.273 INFO:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2020-11-20T12:03:15.273 INFO:teuthology.orchestra.run.smithi044:> ') 2020-11-20T12:03:15.386 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-11-20T12:03:15.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:15.387 INFO:teuthology.orchestra.run.smithi044:> (cd / && exec sudo bash -c ' 2020-11-20T12:03:15.387 INFO:teuthology.orchestra.run.smithi044:> set -e 2020-11-20T12:03:15.388 INFO:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2020-11-20T12:03:15.388 INFO:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2020-11-20T12:03:15.388 INFO:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2020-11-20T12:03:15.389 INFO:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2020-11-20T12:03:15.389 INFO:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2020-11-20T12:03:15.389 INFO:teuthology.orchestra.run.smithi044:> ') 2020-11-20T12:03:15.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:03:15.621 INFO:teuthology.orchestra.run.smithi044:> (cd / && exec sudo bash -c ' 2020-11-20T12:03:15.621 INFO:teuthology.orchestra.run.smithi044:> set -e 2020-11-20T12:03:15.622 INFO:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2020-11-20T12:03:15.622 INFO:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2020-11-20T12:03:15.622 INFO:teuthology.orchestra.run.smithi044:> ') 2020-11-20T12:03:15.734 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.859+0000 7f61f5cce700 1 Processor -- start 2020-11-20T12:03:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.859+0000 7f61f5cce700 1 -- start start 2020-11-20T12:03:15.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.860+0000 7f61f5cce700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f61f00aa1e0 0x7f61f0065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:15.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.860+0000 7f61f5cce700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f61f013c880 0x7f61f0066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:15.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.860+0000 7f61f5cce700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f00a8bf0 con 0x7f61f00a7810 2020-11-20T12:03:15.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.860+0000 7f61f5cce700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f0066520 con 0x7f61f013c880 2020-11-20T12:03:15.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.861+0000 7f61f5cce700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f00a86b0 con 0x7f61f00aa1e0 2020-11-20T12:03:15.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.861+0000 7f61effff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f61f013c880 0x7f61f0066030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:15.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.861+0000 7f61eeffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f61f00a7810 0x7f61f013c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:43188/0 (socket says 172.21.15.44:43188) 2020-11-20T12:03:15.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.861+0000 7f61eeffd700 1 -- 172.21.15.44:0/3425688586 learned_addr learned my addr 172.21.15.44:0/3425688586 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:15.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.861+0000 7f61ef7fe700 1 --2- 172.21.15.44:0/3425688586 >> v2:172.21.15.79:3300/0 conn(0x7f61f00aa1e0 0x7f61f0065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:15.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.862+0000 7f61effff700 1 -- 172.21.15.44:0/3425688586 >> v2:172.21.15.79:3300/0 conn(0x7f61f00aa1e0 msgr2=0x7f61f0065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:15.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.862+0000 7f61effff700 1 --2- 172.21.15.44:0/3425688586 >> v2:172.21.15.79:3300/0 conn(0x7f61f00aa1e0 0x7f61f0065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:15.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.862+0000 7f61effff700 1 -- 172.21.15.44:0/3425688586 >> v1:172.21.15.79:6789/0 conn(0x7f61f00a7810 legacy=0x7f61f013c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:15.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.862+0000 7f61effff700 1 -- 172.21.15.44:0/3425688586 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00a8950 con 0x7f61f013c880 2020-11-20T12:03:15.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.862+0000 7f61effff700 1 --2- 172.21.15.44:0/3425688586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 0x7f61f0066030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f61f00adec0 tx=0x7f61e80091d0).ready entity=mon.0 client_cookie=128aee1173fc1a3b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:15.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.862+0000 7f61ee7fc700 1 -- 172.21.15.44:0/3425688586 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800ba90 con 0x7f61f013c880 2020-11-20T12:03:15.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.863+0000 7f61ee7fc700 1 -- 172.21.15.44:0/3425688586 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e800bbf0 con 0x7f61f013c880 2020-11-20T12:03:15.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.863+0000 7f61ee7fc700 1 -- 172.21.15.44:0/3425688586 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8011400 con 0x7f61f013c880 2020-11-20T12:03:15.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 -- 172.21.15.44:0/3425688586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 msgr2=0x7f61f0066030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:15.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 --2- 172.21.15.44:0/3425688586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 0x7f61f0066030 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f61f00adec0 tx=0x7f61e80091d0).stop 2020-11-20T12:03:15.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 -- 172.21.15.44:0/3425688586 shutdown_connections 2020-11-20T12:03:15.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 --2- 172.21.15.44:0/3425688586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 0x7f61f0066030 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:15.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 --2- 172.21.15.44:0/3425688586 >> v2:172.21.15.79:3300/0 conn(0x7f61f00aa1e0 0x7f61f0065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:15.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 -- 172.21.15.44:0/3425688586 >> 172.21.15.44:0/3425688586 conn(0x7f61f0062690 msgr2=0x7f61f00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:15.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 -- 172.21.15.44:0/3425688586 shutdown_connections 2020-11-20T12:03:15.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.864+0000 7f61f5cce700 1 -- 172.21.15.44:0/3425688586 wait complete. 2020-11-20T12:03:15.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.867+0000 7f61f5cce700 1 Processor -- start 2020-11-20T12:03:15.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.867+0000 7f61f5cce700 1 -- start start 2020-11-20T12:03:15.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.868+0000 7f61f5cce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f00a7810 0x7f61f00a3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:15.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.868+0000 7f61f5cce700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f61f00aa1e0 0x7f61f0147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:15.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.868+0000 7f61f5cce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 0x7f61f0115910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:15.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.868+0000 7f61f5cce700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0139b50 con 0x7f61f013c880 2020-11-20T12:03:15.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.868+0000 7f61f5cce700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f014b620 con 0x7f61f00a7810 2020-11-20T12:03:15.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.868+0000 7f61f5cce700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f014b8c0 con 0x7f61f00aa1e0 2020-11-20T12:03:15.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.869+0000 7f61eeffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f61f00aa1e0 0x7f61f0147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:15.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.869+0000 7f61eeffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f61f00aa1e0 0x7f61f0147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48010/0 (socket says 172.21.15.44:48010) 2020-11-20T12:03:15.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.869+0000 7f61eeffd700 1 -- 172.21.15.44:0/1448891415 learned_addr learned my addr 172.21.15.44:0/1448891415 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:15.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.869+0000 7f61effff700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 0x7f61f0115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:15.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.871+0000 7f61eeffd700 1 -- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f00a7810 msgr2=0x7f61f00a3220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2020-11-20T12:03:15.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.871+0000 7f61eeffd700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f00a7810 0x7f61f00a3220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:15.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.871+0000 7f61eeffd700 1 -- 172.21.15.44:0/1448891415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 msgr2=0x7f61f0115910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:15.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.871+0000 7f61eeffd700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 0x7f61f0115910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:15.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.871+0000 7f61eeffd700 1 -- 172.21.15.44:0/1448891415 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f014bb60 con 0x7f61f00aa1e0 2020-11-20T12:03:15.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.871+0000 7f61eeffd700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f61f00aa1e0 0x7f61f0147d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f61e400c760 tx=0x7f61e4007d20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:15.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.872+0000 7f61d77fe700 1 -- 172.21.15.44:0/1448891415 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4007320 con 0x7f61f00aa1e0 2020-11-20T12:03:15.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.872+0000 7f61d77fe700 1 -- 172.21.15.44:0/1448891415 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e4007480 con 0x7f61f00aa1e0 2020-11-20T12:03:15.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.872+0000 7f61d77fe700 1 -- 172.21.15.44:0/1448891415 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40057a0 con 0x7f61f00aa1e0 2020-11-20T12:03:15.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.872+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f00a3770 con 0x7f61f00aa1e0 2020-11-20T12:03:15.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.872+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f014b3d0 con 0x7f61f00aa1e0 2020-11-20T12:03:15.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.874+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f00a8220 con 0x7f61f00aa1e0 2020-11-20T12:03:15.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.874+0000 7f61d77fe700 1 -- 172.21.15.44:0/1448891415 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f61e4010030 con 0x7f61f00aa1e0 2020-11-20T12:03:15.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.874+0000 7f61d77fe700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61cc056d70 0x7f61cc059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:15.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.874+0000 7f61d77fe700 1 -- 172.21.15.44:0/1448891415 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f61e406fdb0 con 0x7f61f00aa1e0 2020-11-20T12:03:15.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.875+0000 7f61ef7fe700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61cc056d70 0x7f61cc059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:15.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.875+0000 7f61ef7fe700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61cc056d70 0x7f61cc059230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f61e0006fd0 tx=0x7f61e0006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:15.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:15.879+0000 7f61d77fe700 1 -- 172.21.15.44:0/1448891415 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f61e4045120 con 0x7f61f00aa1e0 2020-11-20T12:03:16.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.054+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61f0061650 con 0x7f61f00aa1e0 2020-11-20T12:03:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.056+0000 7f61d77fe700 1 -- 172.21.15.44:0/1448891415 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7f61e4048740 con 0x7f61f00aa1e0 2020-11-20T12:03:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:16.060 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:16.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61cc056d70 msgr2=0x7f61cc059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61cc056d70 0x7f61cc059230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f61e0006fd0 tx=0x7f61e0006df0).stop 2020-11-20T12:03:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f61f00aa1e0 msgr2=0x7f61f0147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f61f00aa1e0 0x7f61f0147d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f61e400c760 tx=0x7f61e4007d20).stop 2020-11-20T12:03:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 shutdown_connections 2020-11-20T12:03:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61cc056d70 0x7f61cc059230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61f013c880 0x7f61f0115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61f00a7810 0x7f61f00a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 --2- 172.21.15.44:0/1448891415 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f61f00aa1e0 0x7f61f0147d40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 >> 172.21.15.44:0/1448891415 conn(0x7f61f0062690 msgr2=0x7f61f0066da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 shutdown_connections 2020-11-20T12:03:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.057+0000 7f61f5cce700 1 -- 172.21.15.44:0/1448891415 wait complete. 2020-11-20T12:03:16.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:02:44.611733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:02:52.195125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:02:55.303534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:02:57.791405+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:03:16.074 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:03:16.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.198+0000 7f0d04913700 1 Processor -- start 2020-11-20T12:03:16.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.198+0000 7f0d04913700 1 -- start start 2020-11-20T12:03:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0d04913700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0d000ad550 0x7f0d000a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0d04913700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0d000ad930 0x7f0d000a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0d04913700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000a7200 con 0x7f0d000a8ea0 2020-11-20T12:03:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0d04913700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d00135240 con 0x7f0d000ad550 2020-11-20T12:03:16.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0d04913700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d000a6cc0 con 0x7f0d000ad930 2020-11-20T12:03:16.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0cfe59c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0d000ad550 0x7f0d000a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0cfed9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0d000a8ea0 0x7f0d0013ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40330/0 (socket says 172.21.15.44:40330) 2020-11-20T12:03:16.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0cfed9d700 1 -- 172.21.15.44:0/2460478670 learned_addr learned my addr 172.21.15.44:0/2460478670 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.199+0000 7f0cfdd9b700 1 --2- 172.21.15.44:0/2460478670 >> v2:172.21.15.79:3300/0 conn(0x7f0d000ad930 0x7f0d000a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfe59c700 1 -- 172.21.15.44:0/2460478670 >> v2:172.21.15.79:3300/0 conn(0x7f0d000ad930 msgr2=0x7f0d000a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfe59c700 1 --2- 172.21.15.44:0/2460478670 >> v2:172.21.15.79:3300/0 conn(0x7f0d000ad930 0x7f0d000a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfe59c700 1 -- 172.21.15.44:0/2460478670 >> v1:172.21.15.44:6789/0 conn(0x7f0d000a8ea0 legacy=0x7f0d0013ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfe59c700 1 -- 172.21.15.44:0/2460478670 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000a6f60 con 0x7f0d000ad550 2020-11-20T12:03:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfd59a700 1 -- 172.21.15.44:0/2460478670 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905110417 0 0) 0x7f0d000a7200 con 0x7f0d000a8ea0 2020-11-20T12:03:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfe59c700 1 --2- 172.21.15.44:0/2460478670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000ad550 0x7f0d000a5070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f0cf8009c90 tx=0x7f0cf80091d0).ready entity=mon.0 client_cookie=309b11d0e6f1ce92 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfd59a700 1 -- 172.21.15.44:0/2460478670 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800bb80 con 0x7f0d000ad550 2020-11-20T12:03:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfd59a700 1 -- 172.21.15.44:0/2460478670 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf800bce0 con 0x7f0d000ad550 2020-11-20T12:03:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0cfd59a700 1 -- 172.21.15.44:0/2460478670 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8011610 con 0x7f0d000ad550 2020-11-20T12:03:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 -- 172.21.15.44:0/2460478670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000ad550 msgr2=0x7f0d000a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 --2- 172.21.15.44:0/2460478670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000ad550 0x7f0d000a5070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f0cf8009c90 tx=0x7f0cf80091d0).stop 2020-11-20T12:03:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 -- 172.21.15.44:0/2460478670 shutdown_connections 2020-11-20T12:03:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 --2- 172.21.15.44:0/2460478670 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000ad550 0x7f0d000a5070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 --2- 172.21.15.44:0/2460478670 >> v2:172.21.15.79:3300/0 conn(0x7f0d000ad930 0x7f0d000a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 -- 172.21.15.44:0/2460478670 >> 172.21.15.44:0/2460478670 conn(0x7f0d000624b0 msgr2=0x7f0d000ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 -- 172.21.15.44:0/2460478670 shutdown_connections 2020-11-20T12:03:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.200+0000 7f0d04913700 1 -- 172.21.15.44:0/2460478670 wait complete. 2020-11-20T12:03:16.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.201+0000 7f0d04913700 1 Processor -- start 2020-11-20T12:03:16.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.201+0000 7f0d04913700 1 -- start start 2020-11-20T12:03:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0d04913700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000a8ea0 0x7f0d0013c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0d04913700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0d000ad550 0x7f0d00136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0d04913700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d000ad930 0x7f0d0014a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0d04913700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0013c930 con 0x7f0d000a8ea0 2020-11-20T12:03:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0d04913700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0013cbd0 con 0x7f0d000ad930 2020-11-20T12:03:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0d04913700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0014a0e0 con 0x7f0d000ad550 2020-11-20T12:03:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000a8ea0 0x7f0d0013c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000a8ea0 0x7f0d0013c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36584/0 (socket says 172.21.15.44:36584) 2020-11-20T12:03:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 -- 172.21.15.44:0/1954466917 learned_addr learned my addr 172.21.15.44:0/1954466917 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfdd9b700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0d000ad550 0x7f0d00136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfed9d700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d000ad930 0x7f0d0014a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 -- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0d000ad550 msgr2=0x7f0d00136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0d000ad550 0x7f0d00136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 -- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d000ad930 msgr2=0x7f0d0014a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d000ad930 0x7f0d0014a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 -- 172.21.15.44:0/1954466917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00067a60 con 0x7f0d000a8ea0 2020-11-20T12:03:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfdd9b700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0d000ad550 0x7f0d00136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfed9d700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d000ad930 0x7f0d0014a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.202+0000 7f0cfe59c700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000a8ea0 0x7f0d0013c290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f0cf8009e70 tx=0x7f0cf8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.203+0000 7f0cf67fc700 1 -- 172.21.15.44:0/1954466917 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf801c6c0 con 0x7f0d000a8ea0 2020-11-20T12:03:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.203+0000 7f0cf67fc700 1 -- 172.21.15.44:0/1954466917 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf801c820 con 0x7f0d000a8ea0 2020-11-20T12:03:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.203+0000 7f0cf67fc700 1 -- 172.21.15.44:0/1954466917 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf801cc20 con 0x7f0d000a8ea0 2020-11-20T12:03:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.203+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d00067cb0 con 0x7f0d000a8ea0 2020-11-20T12:03:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.203+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0014acb0 con 0x7f0d000a8ea0 2020-11-20T12:03:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.204+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0013c780 con 0x7f0d000a8ea0 2020-11-20T12:03:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.204+0000 7f0cf67fc700 1 -- 172.21.15.44:0/1954466917 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0cf8024030 con 0x7f0d000a8ea0 2020-11-20T12:03:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.204+0000 7f0cf67fc700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0cdc056d50 0x7f0cdc059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.204+0000 7f0cf67fc700 1 -- 172.21.15.44:0/1954466917 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0cf8071b60 con 0x7f0d000a8ea0 2020-11-20T12:03:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.204+0000 7f0cfdd9b700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0cdc056d50 0x7f0cdc059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.205+0000 7f0cfdd9b700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0cdc056d50 0x7f0cdc059210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0d000aa770 tx=0x7f0ce8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.208+0000 7f0cf67fc700 1 -- 172.21.15.44:0/1954466917 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0cf8046f00 con 0x7f0d000a8ea0 2020-11-20T12:03:16.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.381+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0d000abb20 con 0x7f0d000a8ea0 2020-11-20T12:03:16.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.381+0000 7f0cf67fc700 1 -- 172.21.15.44:0/1954466917 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0cf8022070 con 0x7f0d000a8ea0 2020-11-20T12:03:16.386 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2020-11-20T12:03:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0cdc056d50 msgr2=0x7f0cdc059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0cdc056d50 0x7f0cdc059210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0d000aa770 tx=0x7f0ce8008040).stop 2020-11-20T12:03:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000a8ea0 msgr2=0x7f0d0013c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000a8ea0 0x7f0d0013c290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f0cf8009e70 tx=0x7f0cf8016000).stop 2020-11-20T12:03:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 shutdown_connections 2020-11-20T12:03:16.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0cdc056d50 0x7f0cdc059210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d000ad930 0x7f0d0014a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d000a8ea0 0x7f0d0013c290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 --2- 172.21.15.44:0/1954466917 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0d000ad550 0x7f0d00136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 >> 172.21.15.44:0/1954466917 conn(0x7f0d000624b0 msgr2=0x7f0d000a7710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 shutdown_connections 2020-11-20T12:03:16.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.383+0000 7f0d04913700 1 -- 172.21.15.44:0/1954466917 wait complete. 2020-11-20T12:03:16.398 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:03:16.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.523+0000 7fb62c6da700 1 Processor -- start 2020-11-20T12:03:16.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.523+0000 7fb62c6da700 1 -- start start 2020-11-20T12:03:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62c6da700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb624060550 0x7fb6241375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62c6da700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624137aa0 0x7fb624141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62c6da700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6240a4280 con 0x7fb624060170 2020-11-20T12:03:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62c6da700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6240a4dc0 con 0x7fb624060550 2020-11-20T12:03:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62c6da700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240a5b20 con 0x7fb624137aa0 2020-11-20T12:03:16.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb624060550 0x7fb6241375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62ac77700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624137aa0 0x7fb624141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb624060550 0x7fb6241375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:56876/0 (socket says 172.21.15.44:56876) 2020-11-20T12:03:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62a476700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb624060170 0x7fb6240a46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:43216/0 (socket says 172.21.15.44:43216) 2020-11-20T12:03:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb62ac77700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624137aa0 0x7fb624141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48034/0 (socket says 172.21.15.44:48034) 2020-11-20T12:03:16.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 -- 172.21.15.44:0/1114157424 learned_addr learned my addr 172.21.15.44:0/1114157424 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 -- 172.21.15.44:0/1114157424 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624137aa0 msgr2=0x7fb624141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 --2- 172.21.15.44:0/1114157424 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624137aa0 0x7fb624141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 -- 172.21.15.44:0/1114157424 >> v1:172.21.15.79:6789/0 conn(0x7fb624060170 legacy=0x7fb6240a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629474700 1 -- 172.21.15.44:0/1114157424 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326170618 0 0) 0x7fb6240a4280 con 0x7fb624060170 2020-11-20T12:03:16.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 -- 172.21.15.44:0/1114157424 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624137f10 con 0x7fb624060550 2020-11-20T12:03:16.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.524+0000 7fb629c75700 1 --2- 172.21.15.44:0/1114157424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624060550 0x7fb6241375b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb614006fd0 tx=0x7fb614006eb0).ready entity=mon.1 client_cookie=37c4177276af1b19 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb629474700 1 -- 172.21.15.44:0/1114157424 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009810 con 0x7fb624060550 2020-11-20T12:03:16.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb629474700 1 -- 172.21.15.44:0/1114157424 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614009970 con 0x7fb624060550 2020-11-20T12:03:16.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb629474700 1 -- 172.21.15.44:0/1114157424 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009cf0 con 0x7fb624060550 2020-11-20T12:03:16.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 -- 172.21.15.44:0/1114157424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624060550 msgr2=0x7fb6241375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 --2- 172.21.15.44:0/1114157424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624060550 0x7fb6241375b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb614006fd0 tx=0x7fb614006eb0).stop 2020-11-20T12:03:16.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 -- 172.21.15.44:0/1114157424 shutdown_connections 2020-11-20T12:03:16.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 --2- 172.21.15.44:0/1114157424 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624137aa0 0x7fb624141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 --2- 172.21.15.44:0/1114157424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624060550 0x7fb6241375b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 -- 172.21.15.44:0/1114157424 >> 172.21.15.44:0/1114157424 conn(0x7fb6240624b0 msgr2=0x7fb6240ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:16.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 -- 172.21.15.44:0/1114157424 shutdown_connections 2020-11-20T12:03:16.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.525+0000 7fb62c6da700 1 -- 172.21.15.44:0/1114157424 wait complete. 2020-11-20T12:03:16.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.526+0000 7fb62c6da700 1 Processor -- start 2020-11-20T12:03:16.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.526+0000 7fb62c6da700 1 -- start start 2020-11-20T12:03:16.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62c6da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb624060170 0x7fb6240ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62c6da700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624060550 0x7fb624149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62c6da700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624137aa0 0x7fb624149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62c6da700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624149e00 con 0x7fb624060170 2020-11-20T12:03:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62c6da700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624148fc0 con 0x7fb624137aa0 2020-11-20T12:03:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62c6da700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240acfd0 con 0x7fb624060550 2020-11-20T12:03:16.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62a476700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb624060170 0x7fb6240ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62a476700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb624060170 0x7fb6240ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36598/0 (socket says 172.21.15.44:36598) 2020-11-20T12:03:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62a476700 1 -- 172.21.15.44:0/103661177 learned_addr learned my addr 172.21.15.44:0/103661177 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62ac77700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624137aa0 0x7fb624149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb629c75700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624060550 0x7fb624149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62ac77700 1 -- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624060550 msgr2=0x7fb624149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62ac77700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624060550 0x7fb624149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62ac77700 1 -- 172.21.15.44:0/103661177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb624060170 msgr2=0x7fb6240ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62ac77700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb624060170 0x7fb6240ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62ac77700 1 -- 172.21.15.44:0/103661177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624066e90 con 0x7fb624137aa0 2020-11-20T12:03:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62a476700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb624060170 0x7fb6240ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.527+0000 7fb62ac77700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624137aa0 0x7fb624149910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb620010e10 tx=0x7fb620006e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.528+0000 7fb6127fc700 1 -- 172.21.15.44:0/103661177 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62000dda0 con 0x7fb624137aa0 2020-11-20T12:03:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.528+0000 7fb6127fc700 1 -- 172.21.15.44:0/103661177 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620011450 con 0x7fb624137aa0 2020-11-20T12:03:16.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.528+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb624067130 con 0x7fb624137aa0 2020-11-20T12:03:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.528+0000 7fb6127fc700 1 -- 172.21.15.44:0/103661177 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200115e0 con 0x7fb624137aa0 2020-11-20T12:03:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.528+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62409e0c0 con 0x7fb624137aa0 2020-11-20T12:03:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.528+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6240615e0 con 0x7fb624137aa0 2020-11-20T12:03:16.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.529+0000 7fb6127fc700 1 -- 172.21.15.44:0/103661177 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb620021070 con 0x7fb624137aa0 2020-11-20T12:03:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.529+0000 7fb6127fc700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5fc056d20 0x7fb5fc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.529+0000 7fb6127fc700 1 -- 172.21.15.44:0/103661177 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb620072aa0 con 0x7fb624137aa0 2020-11-20T12:03:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.529+0000 7fb62a476700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5fc056d20 0x7fb5fc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.530+0000 7fb62a476700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5fc056d20 0x7fb5fc0591e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb61c001520 tx=0x7fb61c007e60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.534+0000 7fb6127fc700 1 -- 172.21.15.44:0/103661177 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb620047d90 con 0x7fb624137aa0 2020-11-20T12:03:16.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.707+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb624149150 con 0x7fb624137aa0 2020-11-20T12:03:16.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.707+0000 7fb6127fc700 1 -- 172.21.15.44:0/103661177 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb62004b3b0 con 0x7fb624137aa0 2020-11-20T12:03:16.711 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:16.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5fc056d20 msgr2=0x7fb5fc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5fc056d20 0x7fb5fc0591e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb61c001520 tx=0x7fb61c007e60).stop 2020-11-20T12:03:16.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624137aa0 msgr2=0x7fb624149910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624137aa0 0x7fb624149910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb620010e10 tx=0x7fb620006e80).stop 2020-11-20T12:03:16.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 shutdown_connections 2020-11-20T12:03:16.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb5fc056d20 0x7fb5fc0591e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb624137aa0 0x7fb624149910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb624060170 0x7fb6240ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 --2- 172.21.15.44:0/103661177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb624060550 0x7fb624149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 >> 172.21.15.44:0/103661177 conn(0x7fb6240624b0 msgr2=0x7fb624135790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:16.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 shutdown_connections 2020-11-20T12:03:16.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.709+0000 7fb62c6da700 1 -- 172.21.15.44:0/103661177 wait complete. 2020-11-20T12:03:16.724 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:03:16.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.850+0000 7f4f92586700 1 Processor -- start 2020-11-20T12:03:16.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f92586700 1 -- start start 2020-11-20T12:03:16.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f92586700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4f8c064e60 0x7f4f8c0ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f92586700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4f8c065240 0x7f4f8c0ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f92586700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c13ad60 0x7f4f8c137e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f92586700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8c0ac9a0 con 0x7f4f8c064e60 2020-11-20T12:03:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f92586700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8c0aa900 con 0x7f4f8c065240 2020-11-20T12:03:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f92586700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c0ab6d0 con 0x7f4f8c13ad60 2020-11-20T12:03:16.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f8bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4f8c064e60 0x7f4f8c0ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f8bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4f8c064e60 0x7f4f8c0ab1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36606/0 (socket says 172.21.15.44:36606) 2020-11-20T12:03:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f8bfff700 1 -- 172.21.15.44:0/2235327451 learned_addr learned my addr 172.21.15.44:0/2235327451 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f8b7fe700 1 --2- 172.21.15.44:0/2235327451 >> v2:172.21.15.79:3300/0 conn(0x7f4f8c065240 0x7f4f8c0ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.851+0000 7f4f90b23700 1 --2- 172.21.15.44:0/2235327451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c13ad60 0x7f4f8c137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8bfff700 1 -- 172.21.15.44:0/2235327451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c13ad60 msgr2=0x7f4f8c137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8bfff700 1 --2- 172.21.15.44:0/2235327451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c13ad60 0x7f4f8c137e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8bfff700 1 -- 172.21.15.44:0/2235327451 >> v2:172.21.15.79:3300/0 conn(0x7f4f8c065240 msgr2=0x7f4f8c0ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8bfff700 1 --2- 172.21.15.44:0/2235327451 >> v2:172.21.15.79:3300/0 conn(0x7f4f8c065240 0x7f4f8c0ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8bfff700 1 -- 172.21.15.44:0/2235327451 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c13b200 con 0x7f4f8c064e60 2020-11-20T12:03:16.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8b7fe700 1 --2- 172.21.15.44:0/2235327451 >> v2:172.21.15.79:3300/0 conn(0x7f4f8c065240 0x7f4f8c0ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:16.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f90b23700 1 --2- 172.21.15.44:0/2235327451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c13ad60 0x7f4f8c137e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:03:16.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8bfff700 1 --2- 172.21.15.44:0/2235327451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c064e60 0x7f4f8c0ab1e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4f8c0a9600 tx=0x7f4f7c006eb0).ready entity=mon.0 client_cookie=d169d91db2b64252 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8affd700 1 -- 172.21.15.44:0/2235327451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c009960 con 0x7f4f8c064e60 2020-11-20T12:03:16.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8affd700 1 -- 172.21.15.44:0/2235327451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c009ac0 con 0x7f4f8c064e60 2020-11-20T12:03:16.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f8affd700 1 -- 172.21.15.44:0/2235327451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c009e40 con 0x7f4f8c064e60 2020-11-20T12:03:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 -- 172.21.15.44:0/2235327451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c064e60 msgr2=0x7f4f8c0ab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 --2- 172.21.15.44:0/2235327451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c064e60 0x7f4f8c0ab1e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4f8c0a9600 tx=0x7f4f7c006eb0).stop 2020-11-20T12:03:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 -- 172.21.15.44:0/2235327451 shutdown_connections 2020-11-20T12:03:16.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 --2- 172.21.15.44:0/2235327451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c064e60 0x7f4f8c0ab1e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 --2- 172.21.15.44:0/2235327451 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c13ad60 0x7f4f8c137e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 --2- 172.21.15.44:0/2235327451 >> v2:172.21.15.79:3300/0 conn(0x7f4f8c065240 0x7f4f8c0ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 -- 172.21.15.44:0/2235327451 >> 172.21.15.44:0/2235327451 conn(0x7f4f8c060c40 msgr2=0x7f4f8c064750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:16.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 -- 172.21.15.44:0/2235327451 shutdown_connections 2020-11-20T12:03:16.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.852+0000 7f4f92586700 1 -- 172.21.15.44:0/2235327451 wait complete. 2020-11-20T12:03:16.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.853+0000 7f4f92586700 1 Processor -- start 2020-11-20T12:03:16.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.853+0000 7f4f92586700 1 -- start start 2020-11-20T12:03:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f92586700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f8c064e60 0x7f4f8c140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f92586700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c065240 0x7f4f8c0a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f92586700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c13ad60 0x7f4f8c14a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f92586700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c141080 con 0x7f4f8c13ad60 2020-11-20T12:03:16.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f92586700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c13f730 con 0x7f4f8c064e60 2020-11-20T12:03:16.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f92586700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c13f9d0 con 0x7f4f8c065240 2020-11-20T12:03:16.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c13ad60 0x7f4f8c14a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c13ad60 0x7f4f8c14a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36616/0 (socket says 172.21.15.44:36616) 2020-11-20T12:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 -- 172.21.15.44:0/2771596981 learned_addr learned my addr 172.21.15.44:0/2771596981 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f8b7fe700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c065240 0x7f4f8c0a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f8bfff700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f8c064e60 0x7f4f8c140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 -- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c065240 msgr2=0x7f4f8c0a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c065240 0x7f4f8c0a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 -- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f8c064e60 msgr2=0x7f4f8c140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f8c064e60 0x7f4f8c140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.854+0000 7f4f90b23700 1 -- 172.21.15.44:0/2771596981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c13fc70 con 0x7f4f8c13ad60 2020-11-20T12:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.855+0000 7f4f90b23700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c13ad60 0x7f4f8c14a0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f4f8400cb50 tx=0x7f4f84008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.855+0000 7f4f73fff700 1 -- 172.21.15.44:0/2771596981 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f840077b0 con 0x7f4f8c13ad60 2020-11-20T12:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.855+0000 7f4f73fff700 1 -- 172.21.15.44:0/2771596981 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84007910 con 0x7f4f8c13ad60 2020-11-20T12:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.855+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c140110 con 0x7f4f8c13ad60 2020-11-20T12:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.855+0000 7f4f73fff700 1 -- 172.21.15.44:0/2771596981 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84007c90 con 0x7f4f8c13ad60 2020-11-20T12:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.855+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c0a9130 con 0x7f4f8c13ad60 2020-11-20T12:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.856+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c13fe00 con 0x7f4f8c13ad60 2020-11-20T12:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.856+0000 7f4f73fff700 1 -- 172.21.15.44:0/2771596981 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4f84011620 con 0x7f4f8c13ad60 2020-11-20T12:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.856+0000 7f4f73fff700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4f68056d20 0x7f4f680591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.857+0000 7f4f73fff700 1 -- 172.21.15.44:0/2771596981 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4f840702e0 con 0x7f4f8c13ad60 2020-11-20T12:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.857+0000 7f4f8bfff700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4f68056d20 0x7f4f680591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.857+0000 7f4f8bfff700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4f68056d20 0x7f4f680591e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f4f7c006ef0 tx=0x7f4f7c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:16.860+0000 7f4f73fff700 1 -- 172.21.15.44:0/2771596981 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4f840455d0 con 0x7f4f8c13ad60 2020-11-20T12:03:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.035+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f8c05cef0 con 0x7f4f8c13ad60 2020-11-20T12:03:17.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.035+0000 7f4f73fff700 1 -- 172.21.15.44:0/2771596981 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4f84048bf0 con 0x7f4f8c13ad60 2020-11-20T12:03:17.039 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4f68056d20 msgr2=0x7f4f680591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4f68056d20 0x7f4f680591e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f4f7c006ef0 tx=0x7f4f7c015000).stop 2020-11-20T12:03:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c13ad60 msgr2=0x7f4f8c14a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c13ad60 0x7f4f8c14a0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f4f8400cb50 tx=0x7f4f84008040).stop 2020-11-20T12:03:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 shutdown_connections 2020-11-20T12:03:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4f68056d20 0x7f4f680591e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f8c13ad60 0x7f4f8c14a0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f8c064e60 0x7f4f8c140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 --2- 172.21.15.44:0/2771596981 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f8c065240 0x7f4f8c0a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 >> 172.21.15.44:0/2771596981 conn(0x7f4f8c060c40 msgr2=0x7f4f8c0aacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 shutdown_connections 2020-11-20T12:03:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.037+0000 7f4f92586700 1 -- 172.21.15.44:0/2771596981 wait complete. 2020-11-20T12:03:17.052 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:03:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.177+0000 7f0376692700 1 Processor -- start 2020-11-20T12:03:17.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.177+0000 7f0376692700 1 -- start start 2020-11-20T12:03:17.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f0376692700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03700ad550 0x7f03700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f0376692700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700a8ea0 0x7f03701420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f0376692700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03700a7200 con 0x7f03700ad930 2020-11-20T12:03:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f0376692700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0370135240 con 0x7f03700ad550 2020-11-20T12:03:17.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f0376692700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03700a6cc0 con 0x7f03700a8ea0 2020-11-20T12:03:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03700ad550 0x7f03700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03700ad550 0x7f03700a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36620/0 (socket says 172.21.15.44:36620) 2020-11-20T12:03:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 -- 172.21.15.44:0/346608493 learned_addr learned my addr 172.21.15.44:0/346608493 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f0374c2f700 1 --2- 172.21.15.44:0/346608493 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700a8ea0 0x7f03701420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 -- 172.21.15.44:0/346608493 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700a8ea0 msgr2=0x7f03701420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 --2- 172.21.15.44:0/346608493 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700a8ea0 0x7f03701420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 -- 172.21.15.44:0/346608493 >> v1:172.21.15.79:6789/0 conn(0x7f03700ad930 legacy=0x7f03700a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 -- 172.21.15.44:0/346608493 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03700a6f60 con 0x7f03700ad550 2020-11-20T12:03:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036ffff700 1 --2- 172.21.15.44:0/346608493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad550 0x7f03700a5070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f0360006fd0 tx=0x7f0360006eb0).ready entity=mon.0 client_cookie=a09b41ad5f7f34f0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:17.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036effd700 1 -- 172.21.15.44:0/346608493 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600098d0 con 0x7f03700ad550 2020-11-20T12:03:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036effd700 1 -- 172.21.15.44:0/346608493 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360009a30 con 0x7f03700ad550 2020-11-20T12:03:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.178+0000 7f036effd700 1 -- 172.21.15.44:0/346608493 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360009db0 con 0x7f03700ad550 2020-11-20T12:03:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 -- 172.21.15.44:0/346608493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad550 msgr2=0x7f03700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 --2- 172.21.15.44:0/346608493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad550 0x7f03700a5070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f0360006fd0 tx=0x7f0360006eb0).stop 2020-11-20T12:03:17.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 -- 172.21.15.44:0/346608493 shutdown_connections 2020-11-20T12:03:17.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 --2- 172.21.15.44:0/346608493 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700a8ea0 0x7f03701420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 --2- 172.21.15.44:0/346608493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad550 0x7f03700a5070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 -- 172.21.15.44:0/346608493 >> 172.21.15.44:0/346608493 conn(0x7f03700624b0 msgr2=0x7f03700ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:17.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 -- 172.21.15.44:0/346608493 shutdown_connections 2020-11-20T12:03:17.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.179+0000 7f0376692700 1 -- 172.21.15.44:0/346608493 wait complete. 2020-11-20T12:03:17.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.180+0000 7f0376692700 1 Processor -- start 2020-11-20T12:03:17.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.180+0000 7f0376692700 1 -- start start 2020-11-20T12:03:17.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03700a8ea0 0x7f0370145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700ad550 0x7f03700672a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad930 0x7f0370138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0370067790 con 0x7f03700ad930 2020-11-20T12:03:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0370067cc0 con 0x7f03700a8ea0 2020-11-20T12:03:17.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03701460d0 con 0x7f03700ad550 2020-11-20T12:03:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad930 0x7f0370138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad930 0x7f0370138ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36630/0 (socket says 172.21.15.44:36630) 2020-11-20T12:03:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 -- 172.21.15.44:0/1063365163 learned_addr learned my addr 172.21.15.44:0/1063365163 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f036ffff700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03700a8ea0 0x7f0370145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f036f7fe700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700ad550 0x7f03700672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 -- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700ad550 msgr2=0x7f03700672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700ad550 0x7f03700672a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 -- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03700a8ea0 msgr2=0x7f0370145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03700a8ea0 0x7f0370145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 -- 172.21.15.44:0/1063365163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0370146370 con 0x7f03700ad930 2020-11-20T12:03:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0374c2f700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad930 0x7f0370138ba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f036800eb40 tx=0x7f0368008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f034ffff700 1 -- 172.21.15.44:0/1063365163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036800c670 con 0x7f03700ad930 2020-11-20T12:03:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f034ffff700 1 -- 172.21.15.44:0/1063365163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036800c7d0 con 0x7f03700ad930 2020-11-20T12:03:17.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0370146610 con 0x7f03700ad930 2020-11-20T12:03:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f034ffff700 1 -- 172.21.15.44:0/1063365163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036800cb70 con 0x7f03700ad930 2020-11-20T12:03:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.181+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037009e2c0 con 0x7f03700ad930 2020-11-20T12:03:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.182+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03701467a0 con 0x7f03700ad930 2020-11-20T12:03:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.183+0000 7f034ffff700 1 -- 172.21.15.44:0/1063365163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f036801d460 con 0x7f03700ad930 2020-11-20T12:03:17.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.183+0000 7f034ffff700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0348056d20 0x7f03480591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.183+0000 7f034ffff700 1 -- 172.21.15.44:0/1063365163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0368070020 con 0x7f03700ad930 2020-11-20T12:03:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.183+0000 7f036ffff700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0348056d20 0x7f03480591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.184+0000 7f036ffff700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0348056d20 0x7f03480591e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f036000b1b0 tx=0x7f0360015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.187+0000 7f034ffff700 1 -- 172.21.15.44:0/1063365163 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f03680452b0 con 0x7f03700ad930 2020-11-20T12:03:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.361+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03700615e0 con 0x7f03700ad930 2020-11-20T12:03:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.361+0000 7f034ffff700 1 -- 172.21.15.44:0/1063365163 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f03680488d0 con 0x7f03700ad930 2020-11-20T12:03:17.365 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0348056d20 msgr2=0x7f03480591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0348056d20 0x7f03480591e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f036000b1b0 tx=0x7f0360015000).stop 2020-11-20T12:03:17.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad930 msgr2=0x7f0370138ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad930 0x7f0370138ba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f036800eb40 tx=0x7f0368008040).stop 2020-11-20T12:03:17.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 shutdown_connections 2020-11-20T12:03:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0348056d20 0x7f03480591e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03700ad930 0x7f0370138ba0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03700a8ea0 0x7f0370145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 --2- 172.21.15.44:0/1063365163 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03700ad550 0x7f03700672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 >> 172.21.15.44:0/1063365163 conn(0x7f03700624b0 msgr2=0x7f0370133ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 shutdown_connections 2020-11-20T12:03:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.363+0000 7f0376692700 1 -- 172.21.15.44:0/1063365163 wait complete. 2020-11-20T12:03:17.377 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:03:17.378 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:03:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.500+0000 7f2c62ef6700 1 Processor -- start 2020-11-20T12:03:17.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.500+0000 7f2c62ef6700 1 -- start start 2020-11-20T12:03:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c62ef6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c0aa020 0x7f2c5c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c62ef6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2c5c13ac60 0x7f2c5c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c62ef6700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c0a9770 con 0x7f2c5c0aa400 2020-11-20T12:03:17.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c62ef6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0a56c0 con 0x7f2c5c13ac60 2020-11-20T12:03:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c62ef6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0a6da0 con 0x7f2c5c0aa020 2020-11-20T12:03:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c5bfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2c5c0aa400 0x7f2c5c0a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40382/0 (socket says 172.21.15.44:40382) 2020-11-20T12:03:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c5bfff700 1 -- 172.21.15.44:0/3833801181 learned_addr learned my addr 172.21.15.44:0/3833801181 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c61493700 1 --2- 172.21.15.44:0/3833801181 >> v2:172.21.15.79:3300/0 conn(0x7f2c5c13ac60 0x7f2c5c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c60c92700 1 --2- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c0aa020 0x7f2c5c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c5b7fe700 1 -- 172.21.15.44:0/3833801181 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142599131 0 0) 0x7f2c500010d0 con 0x7f2c5c0aa400 2020-11-20T12:03:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c5b7fe700 1 -- 172.21.15.44:0/3833801181 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c48003240 con 0x7f2c5c0aa400 2020-11-20T12:03:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c61493700 1 -- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c0aa020 msgr2=0x7f2c5c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:17.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.501+0000 7f2c61493700 1 --2- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c0aa020 0x7f2c5c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c61493700 1 -- 172.21.15.44:0/3833801181 >> v1:172.21.15.44:6789/0 conn(0x7f2c5c0aa400 legacy=0x7f2c5c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c61493700 1 -- 172.21.15.44:0/3833801181 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c0a7040 con 0x7f2c5c13ac60 2020-11-20T12:03:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c61493700 1 --2- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c13ac60 0x7f2c5c0a3700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f2c54009c90 tx=0x7f2c540091d0).ready entity=mon.1 client_cookie=3b8b50210046ccea server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c5b7fe700 1 -- 172.21.15.44:0/3833801181 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400b9c0 con 0x7f2c5c13ac60 2020-11-20T12:03:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c5b7fe700 1 -- 172.21.15.44:0/3833801181 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5400bb20 con 0x7f2c5c13ac60 2020-11-20T12:03:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c5b7fe700 1 -- 172.21.15.44:0/3833801181 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400bea0 con 0x7f2c5c13ac60 2020-11-20T12:03:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 -- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c13ac60 msgr2=0x7f2c5c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c13ac60 0x7f2c5c0a3700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f2c54009c90 tx=0x7f2c540091d0).stop 2020-11-20T12:03:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 -- 172.21.15.44:0/3833801181 shutdown_connections 2020-11-20T12:03:17.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c13ac60 0x7f2c5c0a3700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/3833801181 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c0aa020 0x7f2c5c0a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 -- 172.21.15.44:0/3833801181 >> 172.21.15.44:0/3833801181 conn(0x7f2c5c062690 msgr2=0x7f2c5c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:17.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 -- 172.21.15.44:0/3833801181 shutdown_connections 2020-11-20T12:03:17.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.502+0000 7f2c62ef6700 1 -- 172.21.15.44:0/3833801181 wait complete. 2020-11-20T12:03:17.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.503+0000 7f2c62ef6700 1 Processor -- start 2020-11-20T12:03:17.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.503+0000 7f2c62ef6700 1 -- start start 2020-11-20T12:03:17.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c62ef6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c0aa020 0x7f2c5c13a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c62ef6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c0aa400 0x7f2c5c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c62ef6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c13ac60 0x7f2c5c12c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c62ef6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c135390 con 0x7f2c5c0aa400 2020-11-20T12:03:17.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c62ef6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c134a50 con 0x7f2c5c0aa020 2020-11-20T12:03:17.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c62ef6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c134cf0 con 0x7f2c5c13ac60 2020-11-20T12:03:17.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c5bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c0aa400 0x7f2c5c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c5bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c0aa400 0x7f2c5c0acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36640/0 (socket says 172.21.15.44:36640) 2020-11-20T12:03:17.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c5bfff700 1 -- 172.21.15.44:0/2865576266 learned_addr learned my addr 172.21.15.44:0/2865576266 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:17.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c60c92700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c0aa020 0x7f2c5c13a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c61493700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c13ac60 0x7f2c5c12c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c5bfff700 1 -- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c13ac60 msgr2=0x7f2c5c12c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c5bfff700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c13ac60 0x7f2c5c12c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c5bfff700 1 -- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c0aa020 msgr2=0x7f2c5c13a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.504+0000 7f2c5bfff700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c0aa020 0x7f2c5c13a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.505+0000 7f2c5bfff700 1 -- 172.21.15.44:0/2865576266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c134f90 con 0x7f2c5c0aa400 2020-11-20T12:03:17.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.505+0000 7f2c5bfff700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c0aa400 0x7f2c5c0acdd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f2c5c0a4990 tx=0x7f2c50007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.505+0000 7f2c58ff9700 1 -- 172.21.15.44:0/2865576266 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500072e0 con 0x7f2c5c0aa400 2020-11-20T12:03:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.505+0000 7f2c58ff9700 1 -- 172.21.15.44:0/2865576266 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50007440 con 0x7f2c5c0aa400 2020-11-20T12:03:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.505+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c0ad5c0 con 0x7f2c5c0aa400 2020-11-20T12:03:17.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.505+0000 7f2c58ff9700 1 -- 172.21.15.44:0/2865576266 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50005840 con 0x7f2c5c0aa400 2020-11-20T12:03:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.505+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c067240 con 0x7f2c5c0aa400 2020-11-20T12:03:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.506+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c0a9530 con 0x7f2c5c0aa400 2020-11-20T12:03:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.506+0000 7f2c58ff9700 1 -- 172.21.15.44:0/2865576266 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2c50010070 con 0x7f2c5c0aa400 2020-11-20T12:03:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.506+0000 7f2c58ff9700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2c34056d20 0x7f2c340591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.507+0000 7f2c58ff9700 1 -- 172.21.15.44:0/2865576266 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2c5006f4b0 con 0x7f2c5c0aa400 2020-11-20T12:03:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.507+0000 7f2c60c92700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2c34056d20 0x7f2c340591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.507+0000 7f2c60c92700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2c34056d20 0x7f2c340591e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2c4c006fd0 tx=0x7f2c4c006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.510+0000 7f2c58ff9700 1 -- 172.21.15.44:0/2865576266 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2c500447a0 con 0x7f2c5c0aa400 2020-11-20T12:03:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.743+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c5c0a9770 con 0x7f2c5c0aa400 2020-11-20T12:03:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.743+0000 7f2c58ff9700 1 -- 172.21.15.44:0/2865576266 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2131 (secure 0 0 0) 0x7f2c5c0a9770 con 0x7f2c5c0aa400 2020-11-20T12:03:17.747 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:17.748 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/4254760701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":4254760701},{"type":"v1","addr":"172.21.15.79:6835","nonce":4254760701}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:03:00.178663+0000","modified":"2020-11-20T12:03:03.184476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/2557888073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2557888073},{"type":"v1","addr":"172.21.15.44:6833","nonce":2557888073}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:03:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2c34056d20 msgr2=0x7f2c340591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2c34056d20 0x7f2c340591e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f2c4c006fd0 tx=0x7f2c4c006df0).stop 2020-11-20T12:03:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c0aa400 msgr2=0x7f2c5c0acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c0aa400 0x7f2c5c0acdd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f2c5c0a4990 tx=0x7f2c50007860).stop 2020-11-20T12:03:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 shutdown_connections 2020-11-20T12:03:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2c34056d20 0x7f2c340591e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2c5c13ac60 0x7f2c5c12c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c5c0aa020 0x7f2c5c13a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 --2- 172.21.15.44:0/2865576266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c5c0aa400 0x7f2c5c0acdd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 >> 172.21.15.44:0/2865576266 conn(0x7f2c5c062690 msgr2=0x7f2c5c060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 shutdown_connections 2020-11-20T12:03:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:17.745+0000 7f2c62ef6700 1 -- 172.21.15.44:0/2865576266 wait complete. 2020-11-20T12:03:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2020-11-20T12:03:17.963 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:03:00.178663+0000', 'modified': '2020-11-20T12:03:03.184476+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6833/2557888073', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6832', 'nonce': 2557888073}, {'type': 'v1', 'addr': '172.21.15.44:6833', 'nonce': 2557888073}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:03:17.964 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:03:17.964 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:03:18.114 INFO:teuthology.orchestra.run.smithi044.stdout:{"cluster_fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","whoami":0,"id":4284,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.922307388,"mdsmap_epoch":7,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":24.378248932000002} 2020-11-20T12:03:18.115 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", "fs_name": "cephfs", "id": 4284, "mdsmap_epoch": 7, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 17.922307388, "state": "up:active", "uptime": 24.378248932, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:03:18.115 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T12:03:18.116 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-20T12:03:18.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:03:18.116 INFO:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p -v /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:03:18.131 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:03:18.132 INFO:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2020-11-20T12:03:18.263 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T12:03:18.264 INFO:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T12:03:18.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:03:18.297 INFO:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2020-11-20T12:03:18.351 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-20T12:03:18.351 INFO:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --id 0 /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:03:18.394 INFO:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2020-11-20T12:03:18.425 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T12:03:18.425 INFO:teuthology.orchestra.run.smithi044:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T12:03:18.496 INFO:teuthology.orchestra.run.smithi044.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T12:03:18.498 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T12:03:18.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:03:18.498 INFO:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2020-11-20T12:03:18.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:2020-11-20T12:03:18.506+0000 7feba9cba200 -1 init, newargv = 0x561a73310060 newargc=15 2020-11-20T12:03:18.511 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[29987]: starting ceph client 2020-11-20T12:03:18.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[29987]: starting fuse 2020-11-20T12:03:18.529 INFO:teuthology.orchestra.run.smithi044.stdout:47 2020-11-20T12:03:18.530 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-20T12:03:18.530 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:03:18.530 INFO:teuthology.orchestra.run.smithi044:> import glob 2020-11-20T12:03:18.531 INFO:teuthology.orchestra.run.smithi044:> import re 2020-11-20T12:03:18.531 INFO:teuthology.orchestra.run.smithi044:> import os 2020-11-20T12:03:18.531 INFO:teuthology.orchestra.run.smithi044:> import subprocess 2020-11-20T12:03:18.532 INFO:teuthology.orchestra.run.smithi044:> 2020-11-20T12:03:18.532 INFO:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2020-11-20T12:03:18.532 INFO:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-20T12:03:18.532 INFO:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2020-11-20T12:03:18.533 INFO:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2020-11-20T12:03:18.533 INFO:teuthology.orchestra.run.smithi044:> 2020-11-20T12:03:18.533 INFO:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2020-11-20T12:03:18.534 INFO:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2020-11-20T12:03:18.534 INFO:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2020-11-20T12:03:18.534 INFO:teuthology.orchestra.run.smithi044:> return files[0] 2020-11-20T12:03:18.535 INFO:teuthology.orchestra.run.smithi044:> 2020-11-20T12:03:18.535 INFO:teuthology.orchestra.run.smithi044:> for f in files: 2020-11-20T12:03:18.535 INFO:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T12:03:18.536 INFO:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T12:03:18.536 INFO:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2020-11-20T12:03:18.536 INFO:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2020-11-20T12:03:18.537 INFO:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2020-11-20T12:03:18.537 INFO:teuthology.orchestra.run.smithi044:> return f 2020-11-20T12:03:18.537 INFO:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T12:03:18.538 INFO:teuthology.orchestra.run.smithi044:> 2020-11-20T12:03:18.538 INFO:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.0")) 2020-11-20T12:03:18.538 INFO:teuthology.orchestra.run.smithi044:> ' 2020-11-20T12:03:18.661 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.0.29987.asok 2020-11-20T12:03:18.842 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.29987.asok 2020-11-20T12:03:18.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:03:18.843 INFO:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.29987.asok status 2020-11-20T12:03:19.006 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:03:19.007 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2020-11-20T12:03:19.007 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "b86e691bf191566922a8f08e2a5abf64fde8cfcb", 2020-11-20T12:03:19.007 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 16.0.0-7397-gb86e691b (b86e691bf191566922a8f08e2a5abf64fde8cfcb) pacific (dev)", 2020-11-20T12:03:19.007 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "0", 2020-11-20T12:03:19.008 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2020-11-20T12:03:19.008 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-20T12:03:19.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "29987", 2020-11-20T12:03:19.008 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2020-11-20T12:03:19.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:19.009 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2020-11-20T12:03:19.009 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2020-11-20T12:03:19.009 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4638, 2020-11-20T12:03:19.009 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2020-11-20T12:03:19.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2020-11-20T12:03:19.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2020-11-20T12:03:19.010 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4638 2020-11-20T12:03:19.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:19.010 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2020-11-20T12:03:19.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:19.011 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2020-11-20T12:03:19.011 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 323175068 2020-11-20T12:03:19.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:19.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:19.012 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2020-11-20T12:03:19.012 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:19.012 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2020-11-20T12:03:19.012 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 323175068 2020-11-20T12:03:19.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:19.013 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.4638 192.168.0.1:0/323175068", 2020-11-20T12:03:19.013 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/323175068", 2020-11-20T12:03:19.013 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2020-11-20T12:03:19.013 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 7, 2020-11-20T12:03:19.014 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 25, 2020-11-20T12:03:19.014 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2020-11-20T12:03:19.014 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2020-11-20T12:03:19.014 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2020-11-20T12:03:19.014 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:03:19.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:03:19.016 INFO:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T12:03:19.016 INFO:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:03:19.032 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2020-11-20T12:03:19.032 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T12:03:19.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:03:19.033 INFO:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:03:19.104 INFO:teuthology.run_tasks:Running task mon_thrash... 2020-11-20T12:03:19.120 INFO:tasks.mon_thrash:Beginning mon_thrash... 2020-11-20T12:03:19.121 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:19.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f304e821700 1 Processor -- start 2020-11-20T12:03:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f304e821700 1 -- start start 2020-11-20T12:03:19.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f304e821700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f30480a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f304e821700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f304813eae0 0x7f304813a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f304e821700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304813aea0 con 0x7f30480a3450 2020-11-20T12:03:19.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f304e821700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30480a3af0 con 0x7f304813eae0 2020-11-20T12:03:19.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f304e821700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304813e100 con 0x7f304813b4a0 2020-11-20T12:03:19.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.267+0000 7f3047fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f30480a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f304cdbe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f304813eae0 0x7f304813a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f3047fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f30480a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48096/0 (socket says 172.21.15.44:48096) 2020-11-20T12:03:19.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f3047fff700 1 -- 172.21.15.44:0/2389322992 learned_addr learned my addr 172.21.15.44:0/2389322992 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:19.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f304cdbe700 1 -- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 msgr2=0x7f30480a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:19.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f304cdbe700 1 --2- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f30480a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f304cdbe700 1 -- 172.21.15.44:0/2389322992 >> v1:172.21.15.79:6789/0 conn(0x7f30480a3450 legacy=0x7f30480aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f30477fe700 1 -- 172.21.15.44:0/2389322992 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961820796 0 0) 0x7f30300010d0 con 0x7f30480a3450 2020-11-20T12:03:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f304cdbe700 1 -- 172.21.15.44:0/2389322992 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304813e3a0 con 0x7f304813eae0 2020-11-20T12:03:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f3047fff700 1 --2- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f30480a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f304cdbe700 1 --2- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304813a9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3040009c90 tx=0x7f30400091d0).ready entity=mon.1 client_cookie=da62ff60494fa3e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f30477fe700 1 -- 172.21.15.44:0/2389322992 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304000ba70 con 0x7f304813eae0 2020-11-20T12:03:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f30477fe700 1 -- 172.21.15.44:0/2389322992 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304000bbd0 con 0x7f304813eae0 2020-11-20T12:03:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.268+0000 7f30477fe700 1 -- 172.21.15.44:0/2389322992 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040011400 con 0x7f304813eae0 2020-11-20T12:03:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 -- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 msgr2=0x7f304813a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 --2- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304813a9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3040009c90 tx=0x7f30400091d0).stop 2020-11-20T12:03:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 -- 172.21.15.44:0/2389322992 shutdown_connections 2020-11-20T12:03:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 --2- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304813a9b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 --2- 172.21.15.44:0/2389322992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f30480a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 -- 172.21.15.44:0/2389322992 >> 172.21.15.44:0/2389322992 conn(0x7f3048062690 msgr2=0x7f30480a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 -- 172.21.15.44:0/2389322992 shutdown_connections 2020-11-20T12:03:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.269+0000 7f304e821700 1 -- 172.21.15.44:0/2389322992 wait complete. 2020-11-20T12:03:19.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 Processor -- start 2020-11-20T12:03:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 -- start start 2020-11-20T12:03:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30480a3450 0x7f30480a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f304814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304814a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048067bc0 con 0x7f30480a3450 2020-11-20T12:03:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048066070 con 0x7f304813eae0 2020-11-20T12:03:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f304e821700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30480662a0 con 0x7f304813b4a0 2020-11-20T12:03:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f3047fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30480a3450 0x7f30480a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f3047fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30480a3450 0x7f30480a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36664/0 (socket says 172.21.15.44:36664) 2020-11-20T12:03:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.270+0000 7f3047fff700 1 -- 172.21.15.44:0/1903749104 learned_addr learned my addr 172.21.15.44:0/1903749104 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304cdbe700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304814a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f303ffff700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f304814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304cdbe700 1 -- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 msgr2=0x7f304814a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304cdbe700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f304814a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304cdbe700 1 -- 172.21.15.44:0/1903749104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30480a3450 msgr2=0x7f30480a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304cdbe700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30480a3450 0x7f30480a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304cdbe700 1 -- 172.21.15.44:0/1903749104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048066540 con 0x7f304813eae0 2020-11-20T12:03:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f3047fff700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30480a3450 0x7f30480a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:19.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304cdbe700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304814a950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f3040013000 tx=0x7f3040016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f3044ff9700 1 -- 172.21.15.44:0/1903749104 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304001c5f0 con 0x7f304813eae0 2020-11-20T12:03:19.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f3044ff9700 1 -- 172.21.15.44:0/1903749104 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304001c750 con 0x7f304813eae0 2020-11-20T12:03:19.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30480669e0 con 0x7f304813eae0 2020-11-20T12:03:19.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f3044ff9700 1 -- 172.21.15.44:0/1903749104 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304001caf0 con 0x7f304813eae0 2020-11-20T12:03:19.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.271+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3048066ea0 con 0x7f304813eae0 2020-11-20T12:03:19.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.272+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048061650 con 0x7f304813eae0 2020-11-20T12:03:19.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.272+0000 7f3044ff9700 1 -- 172.21.15.44:0/1903749104 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3040024030 con 0x7f304813eae0 2020-11-20T12:03:19.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.273+0000 7f3044ff9700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3024056d20 0x7f30240591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.273+0000 7f3044ff9700 1 -- 172.21.15.44:0/1903749104 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f30400717f0 con 0x7f304813eae0 2020-11-20T12:03:19.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.273+0000 7f3047fff700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3024056d20 0x7f30240591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.273+0000 7f3047fff700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3024056d20 0x7f30240591e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3038006b50 tx=0x7f3038006eb0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.277+0000 7f3044ff9700 1 -- 172.21.15.44:0/1903749104 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3040046b00 con 0x7f304813eae0 2020-11-20T12:03:19.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.449+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30480666d0 con 0x7f304813eae0 2020-11-20T12:03:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.451+0000 7f3044ff9700 1 -- 172.21.15.44:0/1903749104 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7f30400302a0 con 0x7f304813eae0 2020-11-20T12:03:19.454 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:19.454 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3024056d20 msgr2=0x7f30240591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3024056d20 0x7f30240591e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3038006b50 tx=0x7f3038006eb0).stop 2020-11-20T12:03:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 msgr2=0x7f304814a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304814a950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f3040013000 tx=0x7f3040016000).stop 2020-11-20T12:03:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 shutdown_connections 2020-11-20T12:03:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3024056d20 0x7f30240591e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f304813eae0 0x7f304814a950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30480a3450 0x7f30480a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 --2- 172.21.15.44:0/1903749104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f304813b4a0 0x7f304814a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 >> 172.21.15.44:0/1903749104 conn(0x7f3048062690 msgr2=0x7f304813be80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 shutdown_connections 2020-11-20T12:03:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.452+0000 7f304e821700 1 -- 172.21.15.44:0/1903749104 wait complete. 2020-11-20T12:03:19.467 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:02:44.611733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:02:52.195125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:02:55.303534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:02:57.791405+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:03:19.468 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:03:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.593+0000 7f12000c3700 1 Processor -- start 2020-11-20T12:03:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.593+0000 7f12000c3700 1 -- start start 2020-11-20T12:03:19.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f12000c3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11f80ad930 0x7f11f80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f12000c3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80a8ea0 0x7f11f80a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f12000c3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80a7200 con 0x7f11f80ad550 2020-11-20T12:03:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f12000c3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f8135240 con 0x7f11f80ad930 2020-11-20T12:03:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f12000c3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80a6cc0 con 0x7f11f80a8ea0 2020-11-20T12:03:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f11fe660700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80a8ea0 0x7f11f80a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f11fd65e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11f80ad930 0x7f11f80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f11fe660700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80a8ea0 0x7f11f80a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48110/0 (socket says 172.21.15.44:48110) 2020-11-20T12:03:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.594+0000 7f11fe660700 1 -- 172.21.15.44:0/738807815 learned_addr learned my addr 172.21.15.44:0/738807815 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fd65e700 1 -- 172.21.15.44:0/738807815 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80a8ea0 msgr2=0x7f11f80a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fd65e700 1 --2- 172.21.15.44:0/738807815 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80a8ea0 0x7f11f80a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fd65e700 1 -- 172.21.15.44:0/738807815 >> v1:172.21.15.79:6789/0 conn(0x7f11f80ad550 legacy=0x7f11f80a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fd65e700 1 -- 172.21.15.44:0/738807815 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80a6f60 con 0x7f11f80ad930 2020-11-20T12:03:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fce5d700 1 -- 172.21.15.44:0/738807815 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094373320 0 0) 0x7f11f80a7200 con 0x7f11f80ad550 2020-11-20T12:03:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fd65e700 1 --2- 172.21.15.44:0/738807815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad930 0x7f11f80a3700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f11f400dc30 tx=0x7f11f4006e80).ready entity=mon.1 client_cookie=10d30b275bb6c2d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fce5d700 1 -- 172.21.15.44:0/738807815 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400bbb0 con 0x7f11f80ad930 2020-11-20T12:03:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fce5d700 1 -- 172.21.15.44:0/738807815 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f400bd10 con 0x7f11f80ad930 2020-11-20T12:03:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f11fce5d700 1 -- 172.21.15.44:0/738807815 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4010680 con 0x7f11f80ad930 2020-11-20T12:03:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f12000c3700 1 -- 172.21.15.44:0/738807815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad930 msgr2=0x7f11f80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f12000c3700 1 --2- 172.21.15.44:0/738807815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad930 0x7f11f80a3700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f11f400dc30 tx=0x7f11f4006e80).stop 2020-11-20T12:03:19.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f12000c3700 1 -- 172.21.15.44:0/738807815 shutdown_connections 2020-11-20T12:03:19.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f12000c3700 1 --2- 172.21.15.44:0/738807815 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80a8ea0 0x7f11f80a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f12000c3700 1 --2- 172.21.15.44:0/738807815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad930 0x7f11f80a3700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.595+0000 7f12000c3700 1 -- 172.21.15.44:0/738807815 >> 172.21.15.44:0/738807815 conn(0x7f11f80624b0 msgr2=0x7f11f80ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:19.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.596+0000 7f12000c3700 1 -- 172.21.15.44:0/738807815 shutdown_connections 2020-11-20T12:03:19.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.596+0000 7f12000c3700 1 -- 172.21.15.44:0/738807815 wait complete. 2020-11-20T12:03:19.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.596+0000 7f12000c3700 1 Processor -- start 2020-11-20T12:03:19.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f12000c3700 1 -- start start 2020-11-20T12:03:19.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f12000c3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f80a8ea0 0x7f11f813bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f12000c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad550 0x7f11f8145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f12000c3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80ad930 0x7f11f81460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f12000c3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81465a0 con 0x7f11f80a8ea0 2020-11-20T12:03:19.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f12000c3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8067ba0 con 0x7f11f80ad550 2020-11-20T12:03:19.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f12000c3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8067660 con 0x7f11f80ad930 2020-11-20T12:03:19.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f11fde5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f80a8ea0 0x7f11f813bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f11fde5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f80a8ea0 0x7f11f813bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36676/0 (socket says 172.21.15.44:36676) 2020-11-20T12:03:19.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f11fde5f700 1 -- 172.21.15.44:0/1105543902 learned_addr learned my addr 172.21.15.44:0/1105543902 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:19.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f11fd65e700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad550 0x7f11f8145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.597+0000 7f11fe660700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80ad930 0x7f11f81460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11fde5f700 1 -- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80ad930 msgr2=0x7f11f81460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11fde5f700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80ad930 0x7f11f81460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11fde5f700 1 -- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad550 msgr2=0x7f11f8145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11fde5f700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad550 0x7f11f8145bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11fde5f700 1 -- 172.21.15.44:0/1105543902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f8067900 con 0x7f11f80a8ea0 2020-11-20T12:03:19.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11fde5f700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f80a8ea0 0x7f11f813bef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f11e800c6e0 tx=0x7f11e8007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11e5ffb700 1 -- 172.21.15.44:0/1105543902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800e490 con 0x7f11f80a8ea0 2020-11-20T12:03:19.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11e5ffb700 1 -- 172.21.15.44:0/1105543902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800e5f0 con 0x7f11f80a8ea0 2020-11-20T12:03:19.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f8137bb0 con 0x7f11f80a8ea0 2020-11-20T12:03:19.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f11e5ffb700 1 -- 172.21.15.44:0/1105543902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800e9b0 con 0x7f11f80a8ea0 2020-11-20T12:03:19.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.598+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f809ea30 con 0x7f11f80a8ea0 2020-11-20T12:03:19.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.599+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f80615e0 con 0x7f11f80a8ea0 2020-11-20T12:03:19.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.599+0000 7f11e5ffb700 1 -- 172.21.15.44:0/1105543902 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f11e8019030 con 0x7f11f80a8ea0 2020-11-20T12:03:19.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.599+0000 7f11e5ffb700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f11d0056d20 0x7f11d00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.599+0000 7f11e5ffb700 1 -- 172.21.15.44:0/1105543902 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f11e806f640 con 0x7f11f80a8ea0 2020-11-20T12:03:19.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.600+0000 7f11fd65e700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f11d0056d20 0x7f11d00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.600+0000 7f11fd65e700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f11d0056d20 0x7f11d00591e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f11f4006ec0 tx=0x7f11f80a70f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.603+0000 7f11e5ffb700 1 -- 172.21.15.44:0/1105543902 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f11e8044930 con 0x7f11f80a8ea0 2020-11-20T12:03:19.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.779+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f11f80446d0 con 0x7f11f80a8ea0 2020-11-20T12:03:19.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.780+0000 7f11e5ffb700 1 -- 172.21.15.44:0/1105543902 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f11e8047f50 con 0x7f11f80a8ea0 2020-11-20T12:03:19.784 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2020-11-20T12:03:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f11d0056d20 msgr2=0x7f11d00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f11d0056d20 0x7f11d00591e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f11f4006ec0 tx=0x7f11f80a70f0).stop 2020-11-20T12:03:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f80a8ea0 msgr2=0x7f11f813bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f80a8ea0 0x7f11f813bef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f11e800c6e0 tx=0x7f11e8007ac0).stop 2020-11-20T12:03:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 shutdown_connections 2020-11-20T12:03:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f11d0056d20 0x7f11d00591e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11f80ad930 0x7f11f81460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11f80a8ea0 0x7f11f813bef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 --2- 172.21.15.44:0/1105543902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11f80ad550 0x7f11f8145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 >> 172.21.15.44:0/1105543902 conn(0x7f11f80624b0 msgr2=0x7f11f805ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 shutdown_connections 2020-11-20T12:03:19.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.781+0000 7f12000c3700 1 -- 172.21.15.44:0/1105543902 wait complete. 2020-11-20T12:03:19.796 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:03:19.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.922+0000 7f2b258b5700 1 Processor -- start 2020-11-20T12:03:19.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.922+0000 7f2b258b5700 1 -- start start 2020-11-20T12:03:19.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b258b5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 0x7f2b201375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b258b5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b20137aa0 0x7f2b20141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b258b5700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b200a4280 con 0x7f2b20060170 2020-11-20T12:03:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b258b5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b200a4dc0 con 0x7f2b20137aa0 2020-11-20T12:03:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b258b5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200a5b20 con 0x7f2b20060550 2020-11-20T12:03:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b1effd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2b20060170 0x7f2b200a46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40432/0 (socket says 172.21.15.44:40432) 2020-11-20T12:03:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b1effd700 1 -- 172.21.15.44:0/184767762 learned_addr learned my addr 172.21.15.44:0/184767762 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b1f7fe700 1 --2- 172.21.15.44:0/184767762 >> v2:172.21.15.79:3300/0 conn(0x7f2b20137aa0 0x7f2b20141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469479639 0 0) 0x7f2b200a4280 con 0x7f2b20060170 2020-11-20T12:03:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b1e7fc700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 0x7f2b201375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.923+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b0c003240 con 0x7f2b20060170 2020-11-20T12:03:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2863064790 0 0) 0x7f2b0c003240 con 0x7f2b20060170 2020-11-20T12:03:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 msgr2=0x7f2b201375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 0x7f2b201375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 >> v2:172.21.15.79:3300/0 conn(0x7f2b20137aa0 msgr2=0x7f2b20141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 --2- 172.21.15.44:0/184767762 >> v2:172.21.15.79:3300/0 conn(0x7f2b20137aa0 0x7f2b20141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b20137f30 con 0x7f2b20060170 2020-11-20T12:03:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f2b140014e0 con 0x7f2b20060170 2020-11-20T12:03:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 >> v1:172.21.15.44:6789/0 conn(0x7f2b20060170 legacy=0x7f2b200a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b0c003730 0x7f2b0c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0044e0 0x7f2b0c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 0x7f2b0c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200a4dc0 con 0x7f2b0c003730 2020-11-20T12:03:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c005390 con 0x7f2b0c0044e0 2020-11-20T12:03:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c004310 con 0x7f2b20060550 2020-11-20T12:03:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1effd700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b0c003730 0x7f2b0c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1e7fc700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0044e0 0x7f2b0c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1f7fe700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 0x7f2b0c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1effd700 1 -- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 msgr2=0x7f2b0c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1effd700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 0x7f2b0c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1effd700 1 -- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0044e0 msgr2=0x7f2b0c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1effd700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b0c0044e0 0x7f2b0c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1effd700 1 -- 172.21.15.44:0/184767762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c003240 con 0x7f2b0c003730 2020-11-20T12:03:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1f7fe700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20060550 0x7f2b0c004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1effd700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b0c003730 0x7f2b0c003b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2b1400c470 tx=0x7f2b200a6140).ready entity=mon.0 client_cookie=e484be377bb91f15 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b140072e0 con 0x7f2b0c003730 2020-11-20T12:03:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14007440 con 0x7f2b0c003730 2020-11-20T12:03:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.924+0000 7f2b1dffb700 1 -- 172.21.15.44:0/184767762 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14005800 con 0x7f2b0c003730 2020-11-20T12:03:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.925+0000 7f2b258b5700 1 -- 172.21.15.44:0/184767762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b0c003730 msgr2=0x7f2b0c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.925+0000 7f2b258b5700 1 --2- 172.21.15.44:0/184767762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b0c003730 0x7f2b0c003b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f2b1400c470 tx=0x7f2b200a6140).stop 2020-11-20T12:03:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.925+0000 7f2b1f7fe700 1 -- 172.21.15.44:0/184767762 reap_dead start 2020-11-20T12:03:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.925+0000 7f2b258b5700 1 -- 172.21.15.44:0/184767762 shutdown_connections 2020-11-20T12:03:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.925+0000 7f2b258b5700 1 -- 172.21.15.44:0/184767762 >> 172.21.15.44:0/184767762 conn(0x7f2b200624b0 msgr2=0x7f2b200ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.925+0000 7f2b258b5700 1 -- 172.21.15.44:0/184767762 shutdown_connections 2020-11-20T12:03:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.925+0000 7f2b258b5700 1 -- 172.21.15.44:0/184767762 wait complete. 2020-11-20T12:03:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.926+0000 7f2b258b5700 1 Processor -- start 2020-11-20T12:03:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.926+0000 7f2b258b5700 1 -- start start 2020-11-20T12:03:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.926+0000 7f2b258b5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20133980 0x7f2b2009d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.926+0000 7f2b258b5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b20066970 0x7f2b20066df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.926+0000 7f2b258b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b200a2a20 0x7f2b200a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.926+0000 7f2b258b5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b20149f00 con 0x7f2b200a2a20 2020-11-20T12:03:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.926+0000 7f2b258b5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2009ed60 con 0x7f2b20066970 2020-11-20T12:03:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b258b5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2009f000 con 0x7f2b20133980 2020-11-20T12:03:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b200a2a20 0x7f2b200a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b200a2a20 0x7f2b200a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36698/0 (socket says 172.21.15.44:36698) 2020-11-20T12:03:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1effd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20133980 0x7f2b2009d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 -- 172.21.15.44:0/3776974287 learned_addr learned my addr 172.21.15.44:0/3776974287 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1e7fc700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b20066970 0x7f2b20066df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 -- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20133980 msgr2=0x7f2b2009d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20133980 0x7f2b2009d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 -- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b20066970 msgr2=0x7f2b20066df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b20066970 0x7f2b20066df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 -- 172.21.15.44:0/3776974287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2009f2a0 con 0x7f2b200a2a20 2020-11-20T12:03:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b1f7fe700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b200a2a20 0x7f2b200a2e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2b20137e80 tx=0x7f2b18008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b06ffd700 1 -- 172.21.15.44:0/3776974287 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800e960 con 0x7f2b200a2a20 2020-11-20T12:03:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b06ffd700 1 -- 172.21.15.44:0/3776974287 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800eae0 con 0x7f2b200a2a20 2020-11-20T12:03:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b20133d60 con 0x7f2b200a2a20 2020-11-20T12:03:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b06ffd700 1 -- 172.21.15.44:0/3776974287 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800eea0 con 0x7f2b200a2a20 2020-11-20T12:03:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.927+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b20067770 con 0x7f2b200a2a20 2020-11-20T12:03:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.928+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b200615e0 con 0x7f2b200a2a20 2020-11-20T12:03:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.928+0000 7f2b06ffd700 1 -- 172.21.15.44:0/3776974287 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2b1801e030 con 0x7f2b200a2a20 2020-11-20T12:03:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.929+0000 7f2b06ffd700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2afc056d50 0x7f2afc059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.929+0000 7f2b06ffd700 1 -- 172.21.15.44:0/3776974287 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2b1806fd20 con 0x7f2b200a2a20 2020-11-20T12:03:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.929+0000 7f2b1effd700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2afc056d50 0x7f2afc059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.929+0000 7f2b1effd700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2afc056d50 0x7f2afc059210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f2b2005d1e0 tx=0x7f2b14013000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:19.932+0000 7f2b06ffd700 1 -- 172.21.15.44:0/3776974287 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2b18045010 con 0x7f2b200a2a20 2020-11-20T12:03:20.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.108+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2b200446d0 con 0x7f2b200a2a20 2020-11-20T12:03:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.108+0000 7f2b06ffd700 1 -- 172.21.15.44:0/3776974287 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2b18048630 con 0x7f2b200a2a20 2020-11-20T12:03:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.109+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2afc056d50 msgr2=0x7f2afc059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.109+0000 7f2b258b5700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2afc056d50 0x7f2afc059210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f2b2005d1e0 tx=0x7f2b14013000).stop 2020-11-20T12:03:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b200a2a20 msgr2=0x7f2b200a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b200a2a20 0x7f2b200a2e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2b20137e80 tx=0x7f2b18008040).stop 2020-11-20T12:03:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 shutdown_connections 2020-11-20T12:03:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2afc056d50 0x7f2afc059210 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b200a2a20 0x7f2b200a2e80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2b20133980 0x7f2b2009d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 --2- 172.21.15.44:0/3776974287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b20066970 0x7f2b20066df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 >> 172.21.15.44:0/3776974287 conn(0x7f2b200624b0 msgr2=0x7f2b200aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 shutdown_connections 2020-11-20T12:03:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.110+0000 7f2b258b5700 1 -- 172.21.15.44:0/3776974287 wait complete. 2020-11-20T12:03:20.179 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:20.179 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:03:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.316+0000 7f92c2762700 1 Processor -- start 2020-11-20T12:03:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.316+0000 7f92c2762700 1 -- start start 2020-11-20T12:03:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.316+0000 7f92c2762700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 0x7f92bc137e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.316+0000 7f92c2762700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0a9710 con 0x7f92bc143350 2020-11-20T12:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.316+0000 7f92c2762700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0a2fe0 con 0x7f92bc0ad710 2020-11-20T12:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.316+0000 7f92c2762700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0a2df0 con 0x7f92bc137a10 2020-11-20T12:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92c0cff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f92bc143350 0x7f92bc145740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40454/0 (socket says 172.21.15.44:40454) 2020-11-20T12:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92c0cff700 1 -- 172.21.15.44:0/2217357389 learned_addr learned my addr 172.21.15.44:0/2217357389 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 0x7f92bc137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823449780 0 0) 0x7f92bc0a9710 con 0x7f92bc143350 2020-11-20T12:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f929c003240 con 0x7f92bc143350 2020-11-20T12:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184210270 0 0) 0x7f92bc0a2fe0 con 0x7f92bc0ad710 2020-11-20T12:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92bc0a9710 con 0x7f92bc0ad710 2020-11-20T12:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 542945266 0 0) 0x7f929c003240 con 0x7f92bc143350 2020-11-20T12:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 msgr2=0x7f92bc137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 0x7f92bc137e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 >> v1:172.21.15.79:6789/0 conn(0x7f92bc0ad710 legacy=0x7f92bc0a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0a3280 con 0x7f92bc143350 2020-11-20T12:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f92b4001860 con 0x7f92bc143350 2020-11-20T12:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 >> v1:172.21.15.44:6789/0 conn(0x7f92bc143350 legacy=0x7f92bc145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f929c0038f0 0x7f929c003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c0044e0 0x7f929c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 0x7f929c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c005340 con 0x7f929c0044e0 2020-11-20T12:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c00b730 con 0x7f929c0038f0 2020-11-20T12:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c00b8c0 con 0x7f92bc137a10 2020-11-20T12:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c0044e0 0x7f929c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92bbfff700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f929c0038f0 0x7f929c003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.317+0000 7f92c0cff700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 0x7f929c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92b3fff700 1 -- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 msgr2=0x7f929c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc137a10 0x7f929c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92b3fff700 1 -- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f929c0038f0 msgr2=0x7f929c003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f929c0038f0 0x7f929c003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92b3fff700 1 -- 172.21.15.44:0/2217357389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c003240 con 0x7f929c0044e0 2020-11-20T12:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c0044e0 0x7f929c004960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f92a400bc30 tx=0x7f92a4006df0).ready entity=mon.0 client_cookie=58b7ee6fc432b1d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400b310 con 0x7f929c0044e0 2020-11-20T12:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a400ea30 con 0x7f929c0044e0 2020-11-20T12:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92bb7fe700 1 -- 172.21.15.44:0/2217357389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400edb0 con 0x7f929c0044e0 2020-11-20T12:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92c2762700 1 -- 172.21.15.44:0/2217357389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c0044e0 msgr2=0x7f929c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92c2762700 1 --2- 172.21.15.44:0/2217357389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f929c0044e0 0x7f929c004960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f92a400bc30 tx=0x7f92a4006df0).stop 2020-11-20T12:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92c0cff700 1 -- 172.21.15.44:0/2217357389 reap_dead start 2020-11-20T12:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92c2762700 1 -- 172.21.15.44:0/2217357389 shutdown_connections 2020-11-20T12:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.318+0000 7f92c2762700 1 -- 172.21.15.44:0/2217357389 >> 172.21.15.44:0/2217357389 conn(0x7f92bc0624b0 msgr2=0x7f92bc0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.319+0000 7f92c2762700 1 -- 172.21.15.44:0/2217357389 shutdown_connections 2020-11-20T12:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.319+0000 7f92c2762700 1 -- 172.21.15.44:0/2217357389 wait complete. 2020-11-20T12:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.319+0000 7f92c2762700 1 Processor -- start 2020-11-20T12:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.319+0000 7f92c2762700 1 -- start start 2020-11-20T12:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92c2762700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc0ac200 0x7f92bc14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92c2762700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc0ac5e0 0x7f92bc12e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92c2762700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92bc12a080 0x7f92bc12a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92c2762700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc14a0a0 con 0x7f92bc0ac200 2020-11-20T12:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92c2762700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc14a340 con 0x7f92bc12a080 2020-11-20T12:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92c2762700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc14a5e0 con 0x7f92bc0ac5e0 2020-11-20T12:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc0ac200 0x7f92bc14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc0ac200 0x7f92bc14aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36714/0 (socket says 172.21.15.44:36714) 2020-11-20T12:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 -- 172.21.15.44:0/2249801303 learned_addr learned my addr 172.21.15.44:0/2249801303 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92c0cff700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92bc12a080 0x7f92bc12a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc0ac5e0 0x7f92bc12e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 -- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc0ac5e0 msgr2=0x7f92bc12e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc0ac5e0 0x7f92bc12e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 -- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92bc12a080 msgr2=0x7f92bc12a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92bc12a080 0x7f92bc12a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.320+0000 7f92bbfff700 1 -- 172.21.15.44:0/2249801303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc12a9f0 con 0x7f92bc0ac200 2020-11-20T12:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.321+0000 7f92bbfff700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc0ac200 0x7f92bc14aa80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f92ac002c70 tx=0x7f92ac007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.321+0000 7f92b8ff9700 1 -- 172.21.15.44:0/2249801303 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00fa10 con 0x7f92bc0ac200 2020-11-20T12:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.321+0000 7f92b8ff9700 1 -- 172.21.15.44:0/2249801303 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac00fb70 con 0x7f92bc0ac200 2020-11-20T12:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.321+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc0aca10 con 0x7f92bc0ac200 2020-11-20T12:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.321+0000 7f92b8ff9700 1 -- 172.21.15.44:0/2249801303 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac014450 con 0x7f92bc0ac200 2020-11-20T12:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.321+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc13a1e0 con 0x7f92bc0ac200 2020-11-20T12:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.322+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc0615e0 con 0x7f92bc0ac200 2020-11-20T12:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.322+0000 7f92b8ff9700 1 -- 172.21.15.44:0/2249801303 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f92ac01b030 con 0x7f92bc0ac200 2020-11-20T12:03:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.322+0000 7f92b8ff9700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9298056d50 0x7f9298059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.322+0000 7f92b8ff9700 1 -- 172.21.15.44:0/2249801303 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f92ac06ff20 con 0x7f92bc0ac200 2020-11-20T12:03:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.322+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9298056d50 0x7f9298059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.323+0000 7f92b3fff700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9298056d50 0x7f9298059210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f92bc05e620 tx=0x7f92a4019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.326+0000 7f92b8ff9700 1 -- 172.21.15.44:0/2249801303 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f92ac0452c0 con 0x7f92bc0ac200 2020-11-20T12:03:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.502+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92bc0446d0 con 0x7f92bc0ac200 2020-11-20T12:03:20.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.502+0000 7f92b8ff9700 1 -- 172.21.15.44:0/2249801303 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f92ac0488e0 con 0x7f92bc0ac200 2020-11-20T12:03:20.507 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:20.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9298056d50 msgr2=0x7f9298059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9298056d50 0x7f9298059210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f92bc05e620 tx=0x7f92a4019000).stop 2020-11-20T12:03:20.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc0ac200 msgr2=0x7f92bc14aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc0ac200 0x7f92bc14aa80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f92ac002c70 tx=0x7f92ac007b90).stop 2020-11-20T12:03:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 shutdown_connections 2020-11-20T12:03:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9298056d50 0x7f9298059210 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92bc12a080 0x7f92bc12a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92bc0ac200 0x7f92bc14aa80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 --2- 172.21.15.44:0/2249801303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92bc0ac5e0 0x7f92bc12e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 >> 172.21.15.44:0/2249801303 conn(0x7f92bc0624b0 msgr2=0x7f92bc12ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 shutdown_connections 2020-11-20T12:03:20.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.504+0000 7f92c2762700 1 -- 172.21.15.44:0/2249801303 wait complete. 2020-11-20T12:03:20.519 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:03:20.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.644+0000 7f233eda1700 1 Processor -- start 2020-11-20T12:03:20.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.645+0000 7f233eda1700 1 -- start start 2020-11-20T12:03:20.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.645+0000 7f233eda1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2338063a80 0x7f2338146600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.645+0000 7f233eda1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23380ab450 con 0x7f23380b1b80 2020-11-20T12:03:20.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.645+0000 7f233eda1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23380a9390 con 0x7f23380b17a0 2020-11-20T12:03:20.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.645+0000 7f233eda1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380a8ec0 con 0x7f2338063a80 2020-11-20T12:03:20.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.645+0000 7f233d59e700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f23380b1b80 0x7f2338137ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40470/0 (socket says 172.21.15.44:40470) 2020-11-20T12:03:20.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.645+0000 7f233d59e700 1 -- 172.21.15.44:0/3890974965 learned_addr learned my addr 172.21.15.44:0/3890974965 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:20.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233e5a0700 1 --2- 172.21.15.44:0/3890974965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2338063a80 0x7f2338146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233cd9d700 1 -- 172.21.15.44:0/3890974965 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432886148 0 0) 0x7f23380ab450 con 0x7f23380b1b80 2020-11-20T12:03:20.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233cd9d700 1 -- 172.21.15.44:0/3890974965 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2320003240 con 0x7f23380b1b80 2020-11-20T12:03:20.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233cd9d700 1 -- 172.21.15.44:0/3890974965 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720631607 0 0) 0x7f23380a9390 con 0x7f23380b17a0 2020-11-20T12:03:20.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233cd9d700 1 -- 172.21.15.44:0/3890974965 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23380ab450 con 0x7f23380b17a0 2020-11-20T12:03:20.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233e5a0700 1 -- 172.21.15.44:0/3890974965 >> v1:172.21.15.79:6789/0 conn(0x7f23380b17a0 legacy=0x7f233813e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233e5a0700 1 -- 172.21.15.44:0/3890974965 >> v1:172.21.15.44:6789/0 conn(0x7f23380b1b80 legacy=0x7f2338137ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233e5a0700 1 -- 172.21.15.44:0/3890974965 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380a9630 con 0x7f2338063a80 2020-11-20T12:03:20.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233e5a0700 1 --2- 172.21.15.44:0/3890974965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2338063a80 0x7f2338146600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f233000b9d0 tx=0x7f2330006df0).ready entity=mon.2 client_cookie=2ed63d98ab738aec server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233cd9d700 1 -- 172.21.15.44:0/3890974965 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2330009680 con 0x7f2338063a80 2020-11-20T12:03:20.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233cd9d700 1 -- 172.21.15.44:0/3890974965 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23300097e0 con 0x7f2338063a80 2020-11-20T12:03:20.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.646+0000 7f233cd9d700 1 -- 172.21.15.44:0/3890974965 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2330009b60 con 0x7f2338063a80 2020-11-20T12:03:20.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.647+0000 7f233eda1700 1 -- 172.21.15.44:0/3890974965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2338063a80 msgr2=0x7f2338146600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.647+0000 7f233eda1700 1 --2- 172.21.15.44:0/3890974965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2338063a80 0x7f2338146600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f233000b9d0 tx=0x7f2330006df0).stop 2020-11-20T12:03:20.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.647+0000 7f233eda1700 1 -- 172.21.15.44:0/3890974965 shutdown_connections 2020-11-20T12:03:20.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.647+0000 7f233eda1700 1 --2- 172.21.15.44:0/3890974965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2338063a80 0x7f2338146600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.647+0000 7f233eda1700 1 -- 172.21.15.44:0/3890974965 >> 172.21.15.44:0/3890974965 conn(0x7f2338060c40 msgr2=0x7f23380b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:20.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.647+0000 7f233eda1700 1 -- 172.21.15.44:0/3890974965 shutdown_connections 2020-11-20T12:03:20.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.647+0000 7f233eda1700 1 -- 172.21.15.44:0/3890974965 wait complete. 2020-11-20T12:03:20.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 Processor -- start 2020-11-20T12:03:20.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 -- start start 2020-11-20T12:03:20.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2338063a80 0x7f23380b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23380b17a0 0x7f233813acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23380b1b80 0x7f2338141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23381388b0 con 0x7f2338063a80 2020-11-20T12:03:20.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338138b50 con 0x7f23380b1b80 2020-11-20T12:03:20.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233eda1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2338140890 con 0x7f23380b17a0 2020-11-20T12:03:20.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233dd9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2338063a80 0x7f23380b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233dd9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2338063a80 0x7f23380b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36726/0 (socket says 172.21.15.44:36726) 2020-11-20T12:03:20.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.648+0000 7f233dd9f700 1 -- 172.21.15.44:0/2130192074 learned_addr learned my addr 172.21.15.44:0/2130192074 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:20.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233e5a0700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23380b1b80 0x7f2338141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233d59e700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23380b17a0 0x7f233813acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233dd9f700 1 -- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23380b17a0 msgr2=0x7f233813acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233dd9f700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23380b17a0 0x7f233813acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233dd9f700 1 -- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23380b1b80 msgr2=0x7f2338141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233dd9f700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23380b1b80 0x7f2338141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233dd9f700 1 -- 172.21.15.44:0/2130192074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233813b1a0 con 0x7f2338063a80 2020-11-20T12:03:20.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233e5a0700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23380b1b80 0x7f2338141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:20.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233dd9f700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2338063a80 0x7f23380b0f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f232800c710 tx=0x7f2328007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f2325ffb700 1 -- 172.21.15.44:0/2130192074 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800e490 con 0x7f2338063a80 2020-11-20T12:03:20.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f2325ffb700 1 -- 172.21.15.44:0/2130192074 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232800e5f0 con 0x7f2338063a80 2020-11-20T12:03:20.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233813b450 con 0x7f2338063a80 2020-11-20T12:03:20.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f2325ffb700 1 -- 172.21.15.44:0/2130192074 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800e990 con 0x7f2338063a80 2020-11-20T12:03:20.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.649+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2338141a20 con 0x7f2338063a80 2020-11-20T12:03:20.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.650+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338057d80 con 0x7f2338063a80 2020-11-20T12:03:20.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.650+0000 7f2325ffb700 1 -- 172.21.15.44:0/2130192074 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2328019030 con 0x7f2338063a80 2020-11-20T12:03:20.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.651+0000 7f2325ffb700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2310056d20 0x7f23100591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.651+0000 7f2325ffb700 1 -- 172.21.15.44:0/2130192074 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f232806f550 con 0x7f2338063a80 2020-11-20T12:03:20.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.651+0000 7f233d59e700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2310056d20 0x7f23100591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.651+0000 7f233d59e700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2310056d20 0x7f23100591e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f23380ad1a0 tx=0x7f2334009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.654+0000 7f2325ffb700 1 -- 172.21.15.44:0/2130192074 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f23280447e0 con 0x7f2338063a80 2020-11-20T12:03:20.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.831+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23380446d0 con 0x7f2338063a80 2020-11-20T12:03:20.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.831+0000 7f2325ffb700 1 -- 172.21.15.44:0/2130192074 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2328047e00 con 0x7f2338063a80 2020-11-20T12:03:20.836 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:20.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2310056d20 msgr2=0x7f23100591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2310056d20 0x7f23100591e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f23380ad1a0 tx=0x7f2334009040).stop 2020-11-20T12:03:20.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2338063a80 msgr2=0x7f23380b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2338063a80 0x7f23380b0f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f232800c710 tx=0x7f2328007a90).stop 2020-11-20T12:03:20.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 shutdown_connections 2020-11-20T12:03:20.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2310056d20 0x7f23100591e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23380b1b80 0x7f2338141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2338063a80 0x7f23380b0f70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 --2- 172.21.15.44:0/2130192074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23380b17a0 0x7f233813acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 >> 172.21.15.44:0/2130192074 conn(0x7f2338060c40 msgr2=0x7f23381377a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:20.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 shutdown_connections 2020-11-20T12:03:20.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.833+0000 7f233eda1700 1 -- 172.21.15.44:0/2130192074 wait complete. 2020-11-20T12:03:20.849 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2020-11-20T12:03:20.849 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-20T12:03:20.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.972+0000 7fcdf7305700 1 Processor -- start 2020-11-20T12:03:20.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.972+0000 7fcdf7305700 1 -- start start 2020-11-20T12:03:20.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf7305700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcdf00ad790 0x7fcdf00a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf7305700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00a41e0 con 0x7fcdf00a6df0 2020-11-20T12:03:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf7305700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00a4480 con 0x7fcdf00ad3b0 2020-11-20T12:03:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf7305700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00a4720 con 0x7fcdf00ad790 2020-11-20T12:03:20.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf6b04700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcdf00a6df0 0x7fcdf00674c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40486/0 (socket says 172.21.15.44:40486) 2020-11-20T12:03:20.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf6b04700 1 -- 172.21.15.44:0/1565636654 learned_addr learned my addr 172.21.15.44:0/1565636654 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:20.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5b02700 1 --2- 172.21.15.44:0/1565636654 >> v2:172.21.15.79:3300/0 conn(0x7fcdf00ad790 0x7fcdf00a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891538510 0 0) 0x7fcdf00a41e0 con 0x7fcdf00a6df0 2020-11-20T12:03:20.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcddc003240 con 0x7fcdf00a6df0 2020-11-20T12:03:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038371704 0 0) 0x7fcdf00a4480 con 0x7fcdf00ad3b0 2020-11-20T12:03:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf00a41e0 con 0x7fcdf00ad3b0 2020-11-20T12:03:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3494067329 0 0) 0x7fcddc003240 con 0x7fcdf00a6df0 2020-11-20T12:03:20.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 >> v2:172.21.15.79:3300/0 conn(0x7fcdf00ad790 msgr2=0x7fcdf00a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 --2- 172.21.15.44:0/1565636654 >> v2:172.21.15.79:3300/0 conn(0x7fcdf00ad790 0x7fcdf00a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 >> v1:172.21.15.79:6789/0 conn(0x7fcdf00ad3b0 legacy=0x7fcdf013ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf013b270 con 0x7fcdf00a6df0 2020-11-20T12:03:20.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fcde00014e0 con 0x7fcdf00a6df0 2020-11-20T12:03:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 >> v1:172.21.15.44:6789/0 conn(0x7fcdf00a6df0 legacy=0x7fcdf00674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcddc003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc0044e0 0x7fcddc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcddc004e50 0x7fcddc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00a4720 con 0x7fcddc003910 2020-11-20T12:03:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00bc20 con 0x7fcddc0044e0 2020-11-20T12:03:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.973+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00bd60 con 0x7fcddc004e50 2020-11-20T12:03:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6303700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcddc003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf5b02700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc0044e0 0x7fcddc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6b04700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcddc004e50 0x7fcddc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6303700 1 -- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcddc004e50 msgr2=0x7fcddc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:20.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6303700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcddc004e50 0x7fcddc00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6303700 1 -- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc0044e0 msgr2=0x7fcddc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:20.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6303700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcddc0044e0 0x7fcddc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6303700 1 -- 172.21.15.44:0/1565636654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc003240 con 0x7fcddc003910 2020-11-20T12:03:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6b04700 1 -- 172.21.15.44:0/1565636654 reap_dead start 2020-11-20T12:03:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf6303700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcddc003d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fcde8001fd0 tx=0x7fcde8007f10).ready entity=mon.0 client_cookie=7f95ad33f8e77825 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8015070 con 0x7fcddc003910 2020-11-20T12:03:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80016a0 con 0x7fcddc003910 2020-11-20T12:03:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.974+0000 7fcdf5301700 1 -- 172.21.15.44:0/1565636654 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde800cd00 con 0x7fcddc003910 2020-11-20T12:03:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.975+0000 7fcdf7305700 1 -- 172.21.15.44:0/1565636654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 msgr2=0x7fcddc003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.975+0000 7fcdf7305700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcddc003d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fcde8001fd0 tx=0x7fcde8007f10).stop 2020-11-20T12:03:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.975+0000 7fcdf7305700 1 -- 172.21.15.44:0/1565636654 shutdown_connections 2020-11-20T12:03:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.975+0000 7fcdf7305700 1 --2- 172.21.15.44:0/1565636654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcddc003d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.975+0000 7fcdf7305700 1 -- 172.21.15.44:0/1565636654 >> 172.21.15.44:0/1565636654 conn(0x7fcdf0061b60 msgr2=0x7fcdf0061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.975+0000 7fcdf7305700 1 -- 172.21.15.44:0/1565636654 shutdown_connections 2020-11-20T12:03:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.975+0000 7fcdf7305700 1 -- 172.21.15.44:0/1565636654 wait complete. 2020-11-20T12:03:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 Processor -- start 2020-11-20T12:03:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 -- start start 2020-11-20T12:03:20.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcdf01369b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdf01490e0 0x7fcdf0138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcdf00651d0 0x7fcdf0138e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00ac500 con 0x7fcddc003910 2020-11-20T12:03:20.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0149fd0 con 0x7fcdf01490e0 2020-11-20T12:03:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf7305700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0149670 con 0x7fcdf00651d0 2020-11-20T12:03:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf6303700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcdf01369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf6303700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcdf01369b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36746/0 (socket says 172.21.15.44:36746) 2020-11-20T12:03:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf6303700 1 -- 172.21.15.44:0/2465111730 learned_addr learned my addr 172.21.15.44:0/2465111730 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:20.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf5b02700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdf01490e0 0x7fcdf0138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.976+0000 7fcdf6b04700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcdf00651d0 0x7fcdf0138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf6303700 1 -- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcdf00651d0 msgr2=0x7fcdf0138e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf6303700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcdf00651d0 0x7fcdf0138e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf6303700 1 -- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdf01490e0 msgr2=0x7fcdf0138990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf6303700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdf01490e0 0x7fcdf0138990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf6303700 1 -- 172.21.15.44:0/2465111730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0149b30 con 0x7fcddc003910 2020-11-20T12:03:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf6303700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcdf01369b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fcddc0053d0 tx=0x7fcde8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdee7fc700 1 -- 172.21.15.44:0/2465111730 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde801e450 con 0x7fcddc003910 2020-11-20T12:03:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdee7fc700 1 -- 172.21.15.44:0/2465111730 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde801e5b0 con 0x7fcddc003910 2020-11-20T12:03:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf0139c50 con 0x7fcddc003910 2020-11-20T12:03:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdee7fc700 1 -- 172.21.15.44:0/2465111730 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde801e970 con 0x7fcddc003910 2020-11-20T12:03:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.977+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf00666c0 con 0x7fcddc003910 2020-11-20T12:03:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.978+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf0149cc0 con 0x7fcddc003910 2020-11-20T12:03:20.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.978+0000 7fcdee7fc700 1 -- 172.21.15.44:0/2465111730 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcde801ece0 con 0x7fcddc003910 2020-11-20T12:03:20.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.979+0000 7fcdee7fc700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdc8056d20 0x7fcdc80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:20.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.979+0000 7fcdee7fc700 1 -- 172.21.15.44:0/2465111730 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcde80746e0 con 0x7fcddc003910 2020-11-20T12:03:20.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.979+0000 7fcdf5b02700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdc8056d20 0x7fcdc80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:20.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.979+0000 7fcdf5b02700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdc8056d20 0x7fcdc80591e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fcdf0149520 tx=0x7fcde4006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:20.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:20.982+0000 7fcdee7fc700 1 -- 172.21.15.44:0/2465111730 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcde80499d0 con 0x7fcddc003910 2020-11-20T12:03:21.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.157+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 --> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdf0057d80 con 0x7fcdc8056d20 2020-11-20T12:03:21.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.161+0000 7fcdee7fc700 1 -- 172.21.15.44:0/2465111730 <== mgr.4105 v2:172.21.15.79:6832/24383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82195 (secure 0 0 0) 0x7fcdf0057d80 con 0x7fcdc8056d20 2020-11-20T12:03:21.166 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:21.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdc8056d20 msgr2=0x7fcdc80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdc8056d20 0x7fcdc80591e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fcdf0149520 tx=0x7fcde4006df0).stop 2020-11-20T12:03:21.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 msgr2=0x7fcdf01369b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcdf01369b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fcddc0053d0 tx=0x7fcde8017000).stop 2020-11-20T12:03:21.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 shutdown_connections 2020-11-20T12:03:21.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdc8056d20 0x7fcdc80591e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcdf00651d0 0x7fcdf0138e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcddc003910 0x7fcdf01369b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 --2- 172.21.15.44:0/2465111730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcdf01490e0 0x7fcdf0138990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 >> 172.21.15.44:0/2465111730 conn(0x7fcdf0061b60 msgr2=0x7fcdf0137600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:21.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 shutdown_connections 2020-11-20T12:03:21.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.162+0000 7fcdf7305700 1 -- 172.21.15.44:0/2465111730 wait complete. 2020-11-20T12:03:21.170 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2020-11-20T12:03:21.177 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2020-11-20T12:03:20.578628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2747,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41216,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":39040,"kb_avail":754933504,"statfs":{"total":773094113280,"available":773051908096,"internally_reserved":0,"allocated":2162688,"data_stored":837891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39976960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":443,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002981"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873331+0000","last_change":"2020-11-20T12:02:59.871769+0000","last_active":"2020-11-20T12:02:59.873331+0000","last_peered":"2020-11-20T12:02:59.873331+0000","last_clean":"2020-11-20T12:02:59.873331+0000","last_became_active":"2020-11-20T12:02:59.871151+0000","last_became_peered":"2020-11-20T12:02:59.871151+0000","last_unstale":"2020-11-20T12:02:59.873331+0000","last_undegraded":"2020-11-20T12:02:59.873331+0000","last_fullsized":"2020-11-20T12:02:59.873331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.774547+0000","last_change":"2020-11-20T12:02:53.744074+0000","last_active":"2020-11-20T12:02:59.774547+0000","last_peered":"2020-11-20T12:02:59.774547+0000","last_clean":"2020-11-20T12:02:59.774547+0000","last_became_active":"2020-11-20T12:02:53.743596+0000","last_became_peered":"2020-11-20T12:02:53.743596+0000","last_unstale":"2020-11-20T12:02:59.774547+0000","last_undegraded":"2020-11-20T12:02:59.774547+0000","last_fullsized":"2020-11-20T12:02:59.774547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.195451+0000","last_change":"2020-11-20T12:02:56.751652+0000","last_active":"2020-11-20T12:03:00.195451+0000","last_peered":"2020-11-20T12:03:00.195451+0000","last_clean":"2020-11-20T12:03:00.195451+0000","last_became_active":"2020-11-20T12:02:56.751344+0000","last_became_peered":"2020-11-20T12:02:56.751344+0000","last_unstale":"2020-11-20T12:03:00.195451+0000","last_undegraded":"2020-11-20T12:03:00.195451+0000","last_fullsized":"2020-11-20T12:03:00.195451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.780084+0000","last_change":"2020-11-20T12:02:59.779112+0000","last_active":"2020-11-20T12:02:59.780084+0000","last_peered":"2020-11-20T12:02:59.780084+0000","last_clean":"2020-11-20T12:02:59.780084+0000","last_became_active":"2020-11-20T12:02:59.778434+0000","last_became_peered":"2020-11-20T12:02:59.778434+0000","last_unstale":"2020-11-20T12:02:59.780084+0000","last_undegraded":"2020-11-20T12:02:59.780084+0000","last_fullsized":"2020-11-20T12:02:59.780084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.867987+0000","last_change":"2020-11-20T12:02:53.623419+0000","last_active":"2020-11-20T12:02:59.867987+0000","last_peered":"2020-11-20T12:02:59.867987+0000","last_clean":"2020-11-20T12:02:59.867987+0000","last_became_active":"2020-11-20T12:02:53.623135+0000","last_became_peered":"2020-11-20T12:02:53.623135+0000","last_unstale":"2020-11-20T12:02:59.867987+0000","last_undegraded":"2020-11-20T12:02:59.867987+0000","last_fullsized":"2020-11-20T12:02:59.867987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.191541+0000","last_change":"2020-11-20T12:02:56.750366+0000","last_active":"2020-11-20T12:03:00.191541+0000","last_peered":"2020-11-20T12:03:00.191541+0000","last_clean":"2020-11-20T12:03:00.191541+0000","last_became_active":"2020-11-20T12:02:56.749896+0000","last_became_peered":"2020-11-20T12:02:56.749896+0000","last_unstale":"2020-11-20T12:03:00.191541+0000","last_undegraded":"2020-11-20T12:03:00.191541+0000","last_fullsized":"2020-11-20T12:03:00.191541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873473+0000","last_change":"2020-11-20T12:02:59.871656+0000","last_active":"2020-11-20T12:02:59.873473+0000","last_peered":"2020-11-20T12:02:59.873473+0000","last_clean":"2020-11-20T12:02:59.873473+0000","last_became_active":"2020-11-20T12:02:59.871033+0000","last_became_peered":"2020-11-20T12:02:59.871033+0000","last_unstale":"2020-11-20T12:02:59.873473+0000","last_undegraded":"2020-11-20T12:02:59.873473+0000","last_fullsized":"2020-11-20T12:02:59.873473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769916+0000","last_change":"2020-11-20T12:02:53.620206+0000","last_active":"2020-11-20T12:02:59.769916+0000","last_peered":"2020-11-20T12:02:59.769916+0000","last_clean":"2020-11-20T12:02:59.769916+0000","last_became_active":"2020-11-20T12:02:53.619848+0000","last_became_peered":"2020-11-20T12:02:53.619848+0000","last_unstale":"2020-11-20T12:02:59.769916+0000","last_undegraded":"2020-11-20T12:02:59.769916+0000","last_fullsized":"2020-11-20T12:02:59.769916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.196841+0000","last_change":"2020-11-20T12:02:56.751843+0000","last_active":"2020-11-20T12:03:00.196841+0000","last_peered":"2020-11-20T12:03:00.196841+0000","last_clean":"2020-11-20T12:03:00.196841+0000","last_became_active":"2020-11-20T12:02:56.751444+0000","last_became_peered":"2020-11-20T12:02:56.751444+0000","last_unstale":"2020-11-20T12:03:00.196841+0000","last_undegraded":"2020-11-20T12:03:00.196841+0000","last_fullsized":"2020-11-20T12:03:00.196841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871493+0000","last_change":"2020-11-20T12:02:59.870900+0000","last_active":"2020-11-20T12:02:59.871493+0000","last_peered":"2020-11-20T12:02:59.871493+0000","last_clean":"2020-11-20T12:02:59.871493+0000","last_became_active":"2020-11-20T12:02:59.870194+0000","last_became_peered":"2020-11-20T12:02:59.870194+0000","last_unstale":"2020-11-20T12:02:59.871493+0000","last_undegraded":"2020-11-20T12:02:59.871493+0000","last_fullsized":"2020-11-20T12:02:59.871493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868625+0000","last_change":"2020-11-20T12:02:53.621673+0000","last_active":"2020-11-20T12:02:59.868625+0000","last_peered":"2020-11-20T12:02:59.868625+0000","last_clean":"2020-11-20T12:02:59.868625+0000","last_became_active":"2020-11-20T12:02:53.621317+0000","last_became_peered":"2020-11-20T12:02:53.621317+0000","last_unstale":"2020-11-20T12:02:59.868625+0000","last_undegraded":"2020-11-20T12:02:59.868625+0000","last_fullsized":"2020-11-20T12:02:59.868625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868438+0000","last_change":"2020-11-20T12:02:49.777943+0000","last_active":"2020-11-20T12:02:59.868438+0000","last_peered":"2020-11-20T12:02:59.868438+0000","last_clean":"2020-11-20T12:02:59.868438+0000","last_became_active":"2020-11-20T12:02:49.777287+0000","last_became_peered":"2020-11-20T12:02:49.777287+0000","last_unstale":"2020-11-20T12:02:59.868438+0000","last_undegraded":"2020-11-20T12:02:59.868438+0000","last_fullsized":"2020-11-20T12:02:59.868438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_clean_scrub_stamp":"2020-11-20T12:02:45.568237+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":"13","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.182489+0000","last_change":"2020-11-20T12:02:56.751911+0000","last_active":"2020-11-20T12:03:00.182489+0000","last_peered":"2020-11-20T12:03:00.182489+0000","last_clean":"2020-11-20T12:03:00.182489+0000","last_became_active":"2020-11-20T12:02:56.751558+0000","last_became_peered":"2020-11-20T12:02:56.751558+0000","last_unstale":"2020-11-20T12:03:00.182489+0000","last_undegraded":"2020-11-20T12:03:00.182489+0000","last_fullsized":"2020-11-20T12:03:00.182489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.870381+0000","last_change":"2020-11-20T12:02:59.869783+0000","last_active":"2020-11-20T12:02:59.870381+0000","last_peered":"2020-11-20T12:02:59.870381+0000","last_clean":"2020-11-20T12:02:59.870381+0000","last_became_active":"2020-11-20T12:02:59.869158+0000","last_became_peered":"2020-11-20T12:02:59.869158+0000","last_unstale":"2020-11-20T12:02:59.870381+0000","last_undegraded":"2020-11-20T12:02:59.870381+0000","last_fullsized":"2020-11-20T12:02:59.870381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868367+0000","last_change":"2020-11-20T12:02:53.621849+0000","last_active":"2020-11-20T12:02:59.868367+0000","last_peered":"2020-11-20T12:02:59.868367+0000","last_clean":"2020-11-20T12:02:59.868367+0000","last_became_active":"2020-11-20T12:02:53.621479+0000","last_became_peered":"2020-11-20T12:02:53.621479+0000","last_unstale":"2020-11-20T12:02:59.868367+0000","last_undegraded":"2020-11-20T12:02:59.868367+0000","last_fullsized":"2020-11-20T12:02:59.868367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.770789+0000","last_change":"2020-11-20T12:02:56.750191+0000","last_active":"2020-11-20T12:02:59.770789+0000","last_peered":"2020-11-20T12:02:59.770789+0000","last_clean":"2020-11-20T12:02:59.770789+0000","last_became_active":"2020-11-20T12:02:56.749787+0000","last_became_peered":"2020-11-20T12:02:56.749787+0000","last_unstale":"2020-11-20T12:02:59.770789+0000","last_undegraded":"2020-11-20T12:02:59.770789+0000","last_fullsized":"2020-11-20T12:02:59.770789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873151+0000","last_change":"2020-11-20T12:02:59.871503+0000","last_active":"2020-11-20T12:02:59.873151+0000","last_peered":"2020-11-20T12:02:59.873151+0000","last_clean":"2020-11-20T12:02:59.873151+0000","last_became_active":"2020-11-20T12:02:59.870839+0000","last_became_peered":"2020-11-20T12:02:59.870839+0000","last_unstale":"2020-11-20T12:02:59.873151+0000","last_undegraded":"2020-11-20T12:02:59.873151+0000","last_fullsized":"2020-11-20T12:02:59.873151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868440+0000","last_change":"2020-11-20T12:02:53.746269+0000","last_active":"2020-11-20T12:02:59.868440+0000","last_peered":"2020-11-20T12:02:59.868440+0000","last_clean":"2020-11-20T12:02:59.868440+0000","last_became_active":"2020-11-20T12:02:53.745919+0000","last_became_peered":"2020-11-20T12:02:53.745919+0000","last_unstale":"2020-11-20T12:02:59.868440+0000","last_undegraded":"2020-11-20T12:02:59.868440+0000","last_fullsized":"2020-11-20T12:02:59.868440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'10","reported_seq":"22","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:18.511508+0000","last_change":"2020-11-20T12:02:56.752631+0000","last_active":"2020-11-20T12:03:18.511508+0000","last_peered":"2020-11-20T12:03:18.511508+0000","last_clean":"2020-11-20T12:03:18.511508+0000","last_became_active":"2020-11-20T12:02:56.752287+0000","last_became_peered":"2020-11-20T12:02:56.752287+0000","last_unstale":"2020-11-20T12:03:18.511508+0000","last_undegraded":"2020-11-20T12:03:18.511508+0000","last_fullsized":"2020-11-20T12:03:18.511508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1978,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871787+0000","last_change":"2020-11-20T12:02:59.869976+0000","last_active":"2020-11-20T12:02:59.871787+0000","last_peered":"2020-11-20T12:02:59.871787+0000","last_clean":"2020-11-20T12:02:59.871787+0000","last_became_active":"2020-11-20T12:02:59.869214+0000","last_became_peered":"2020-11-20T12:02:59.869214+0000","last_unstale":"2020-11-20T12:02:59.871787+0000","last_undegraded":"2020-11-20T12:02:59.871787+0000","last_fullsized":"2020-11-20T12:02:59.871787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192866+0000","last_change":"2020-11-20T12:02:57.108914+0000","last_active":"2020-11-20T12:03:00.192866+0000","last_peered":"2020-11-20T12:03:00.192866+0000","last_clean":"2020-11-20T12:03:00.192866+0000","last_became_active":"2020-11-20T12:02:57.108538+0000","last_became_peered":"2020-11-20T12:02:57.108538+0000","last_unstale":"2020-11-20T12:03:00.192866+0000","last_undegraded":"2020-11-20T12:03:00.192866+0000","last_fullsized":"2020-11-20T12:03:00.192866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.771858+0000","last_change":"2020-11-20T12:02:53.618862+0000","last_active":"2020-11-20T12:02:59.771858+0000","last_peered":"2020-11-20T12:02:59.771858+0000","last_clean":"2020-11-20T12:02:59.771858+0000","last_became_active":"2020-11-20T12:02:53.618438+0000","last_became_peered":"2020-11-20T12:02:53.618438+0000","last_unstale":"2020-11-20T12:02:59.771858+0000","last_undegraded":"2020-11-20T12:02:59.771858+0000","last_fullsized":"2020-11-20T12:02:59.771858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873226+0000","last_change":"2020-11-20T12:02:59.872290+0000","last_active":"2020-11-20T12:02:59.873226+0000","last_peered":"2020-11-20T12:02:59.873226+0000","last_clean":"2020-11-20T12:02:59.873226+0000","last_became_active":"2020-11-20T12:02:59.870620+0000","last_became_peered":"2020-11-20T12:02:59.870620+0000","last_unstale":"2020-11-20T12:02:59.873226+0000","last_undegraded":"2020-11-20T12:02:59.873226+0000","last_fullsized":"2020-11-20T12:02:59.873226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769189+0000","last_change":"2020-11-20T12:02:53.620514+0000","last_active":"2020-11-20T12:02:59.769189+0000","last_peered":"2020-11-20T12:02:59.769189+0000","last_clean":"2020-11-20T12:02:59.769189+0000","last_became_active":"2020-11-20T12:02:53.620221+0000","last_became_peered":"2020-11-20T12:02:53.620221+0000","last_unstale":"2020-11-20T12:02:59.769189+0000","last_undegraded":"2020-11-20T12:02:59.769189+0000","last_fullsized":"2020-11-20T12:02:59.769189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"16","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192738+0000","last_change":"2020-11-20T12:02:56.752071+0000","last_active":"2020-11-20T12:03:00.192738+0000","last_peered":"2020-11-20T12:03:00.192738+0000","last_clean":"2020-11-20T12:03:00.192738+0000","last_became_active":"2020-11-20T12:02:56.751653+0000","last_became_peered":"2020-11-20T12:02:56.751653+0000","last_unstale":"2020-11-20T12:03:00.192738+0000","last_undegraded":"2020-11-20T12:03:00.192738+0000","last_fullsized":"2020-11-20T12:03:00.192738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2747,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":104105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5196,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366644,"statfs":{"total":96636764160,"available":96631443456,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5140,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366700,"statfs":{"total":96636764160,"available":96631500800,"internally_reserved":0,"allocated":274432,"data_stored":105640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5204,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366636,"statfs":{"total":96636764160,"available":96631435264,"internally_reserved":0,"allocated":274432,"data_stored":106083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":278528,"data_stored":104840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5132,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366708,"statfs":{"total":96636764160,"available":96631508992,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]}]}} 2020-11-20T12:03:21.180 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-20T12:03:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.302+0000 7f1516baf700 1 Processor -- start 2020-11-20T12:03:21.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.302+0000 7f1516baf700 1 -- start start 2020-11-20T12:03:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.302+0000 7f1516baf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f151013a990 0x7f151013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.302+0000 7f1516baf700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f151013f800 0x7f1510141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.302+0000 7f1516baf700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100a6260 con 0x7f1510060330 2020-11-20T12:03:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.302+0000 7f1516baf700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15100a52e0 con 0x7f151013f800 2020-11-20T12:03:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.302+0000 7f1516baf700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15100a6490 con 0x7f151013a990 2020-11-20T12:03:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f151013f800 0x7f1510141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f151013f800 0x7f1510141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36758/0 (socket says 172.21.15.44:36758) 2020-11-20T12:03:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 -- 172.21.15.44:0/2671089333 learned_addr learned my addr 172.21.15.44:0/2671089333 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:21.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f150ffff700 1 --2- 172.21.15.44:0/2671089333 >> v2:172.21.15.79:3300/0 conn(0x7f151013a990 0x7f151013ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 -- 172.21.15.44:0/2671089333 >> v2:172.21.15.79:3300/0 conn(0x7f151013a990 msgr2=0x7f151013ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:21.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 --2- 172.21.15.44:0/2671089333 >> v2:172.21.15.79:3300/0 conn(0x7f151013a990 0x7f151013ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 -- 172.21.15.44:0/2671089333 >> v1:172.21.15.79:6789/0 conn(0x7f1510060330 legacy=0x7f15100a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 -- 172.21.15.44:0/2671089333 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100a6730 con 0x7f151013f800 2020-11-20T12:03:21.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f151514c700 1 --2- 172.21.15.44:0/2671089333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f1510141bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f15100a3a60 tx=0x7f1508009290).ready entity=mon.0 client_cookie=9c399e38a4a15345 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f150f7fe700 1 -- 172.21.15.44:0/2671089333 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800bbc0 con 0x7f151013f800 2020-11-20T12:03:21.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f150f7fe700 1 -- 172.21.15.44:0/2671089333 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150800bd20 con 0x7f151013f800 2020-11-20T12:03:21.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.303+0000 7f150f7fe700 1 -- 172.21.15.44:0/2671089333 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508011610 con 0x7f151013f800 2020-11-20T12:03:21.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 -- 172.21.15.44:0/2671089333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 msgr2=0x7f1510141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 --2- 172.21.15.44:0/2671089333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f1510141bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f15100a3a60 tx=0x7f1508009290).stop 2020-11-20T12:03:21.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 -- 172.21.15.44:0/2671089333 shutdown_connections 2020-11-20T12:03:21.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 --2- 172.21.15.44:0/2671089333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f1510141bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 --2- 172.21.15.44:0/2671089333 >> v2:172.21.15.79:3300/0 conn(0x7f151013a990 0x7f151013ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 -- 172.21.15.44:0/2671089333 >> 172.21.15.44:0/2671089333 conn(0x7f1510062690 msgr2=0x7f15100a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 -- 172.21.15.44:0/2671089333 shutdown_connections 2020-11-20T12:03:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.304+0000 7f1516baf700 1 -- 172.21.15.44:0/2671089333 wait complete. 2020-11-20T12:03:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.305+0000 7f1516baf700 1 Processor -- start 2020-11-20T12:03:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.305+0000 7f1516baf700 1 -- start start 2020-11-20T12:03:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1510060330 0x7f15100ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f151013a990 0x7f1510140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f151013d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151013c800 con 0x7f151013f800 2020-11-20T12:03:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151013b550 con 0x7f151013a990 2020-11-20T12:03:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151013b7f0 con 0x7f1510060330 2020-11-20T12:03:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f151013d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f151013d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36764/0 (socket says 172.21.15.44:36764) 2020-11-20T12:03:21.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 -- 172.21.15.44:0/1158673668 learned_addr learned my addr 172.21.15.44:0/1158673668 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:21.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f150ffff700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f151013a990 0x7f1510140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151494b700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1510060330 0x7f15100ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 -- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1510060330 msgr2=0x7f15100ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1510060330 0x7f15100ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 -- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f151013a990 msgr2=0x7f1510140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f151013a990 0x7f1510140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 -- 172.21.15.44:0/1158673668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100a8c30 con 0x7f151013f800 2020-11-20T12:03:21.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151494b700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1510060330 0x7f15100ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:21.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f151514c700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f151013d550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f15080092d0 tx=0x7f1508016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f150cff9700 1 -- 172.21.15.44:0/1158673668 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800bbc0 con 0x7f151013f800 2020-11-20T12:03:21.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f150cff9700 1 -- 172.21.15.44:0/1158673668 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150800be90 con 0x7f151013f800 2020-11-20T12:03:21.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15101410a0 con 0x7f151013f800 2020-11-20T12:03:21.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f150cff9700 1 -- 172.21.15.44:0/1158673668 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508010da0 con 0x7f151013f800 2020-11-20T12:03:21.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.306+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151013c0f0 con 0x7f151013f800 2020-11-20T12:03:21.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.307+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1510061650 con 0x7f151013f800 2020-11-20T12:03:21.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.308+0000 7f150cff9700 1 -- 172.21.15.44:0/1158673668 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1508017070 con 0x7f151013f800 2020-11-20T12:03:21.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.308+0000 7f150cff9700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14ec056d20 0x7f14ec0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.308+0000 7f150cff9700 1 -- 172.21.15.44:0/1158673668 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f15080722a0 con 0x7f151013f800 2020-11-20T12:03:21.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.308+0000 7f151494b700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14ec056d20 0x7f14ec0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.308+0000 7f151494b700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14ec056d20 0x7f14ec0591e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f14f8001550 tx=0x7f14f8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.312+0000 7f150cff9700 1 -- 172.21.15.44:0/1158673668 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1508047610 con 0x7f151013f800 2020-11-20T12:03:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.480+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 --> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15100446d0 con 0x7f14ec056d20 2020-11-20T12:03:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.484+0000 7f150cff9700 1 -- 172.21.15.44:0/1158673668 <== mgr.4105 v2:172.21.15.79:6832/24383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82195 (secure 0 0 0) 0x7f15100446d0 con 0x7f14ec056d20 2020-11-20T12:03:21.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.485+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14ec056d20 msgr2=0x7f14ec0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.485+0000 7f1516baf700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14ec056d20 0x7f14ec0591e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f14f8001550 tx=0x7f14f8009040).stop 2020-11-20T12:03:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 msgr2=0x7f151013d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f151013d550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f15080092d0 tx=0x7f1508016000).stop 2020-11-20T12:03:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 shutdown_connections 2020-11-20T12:03:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14ec056d20 0x7f14ec0591e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f151013f800 0x7f151013d550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1510060330 0x7f15100ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 --2- 172.21.15.44:0/1158673668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f151013a990 0x7f1510140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 >> 172.21.15.44:0/1158673668 conn(0x7f1510062690 msgr2=0x7f15100a4660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 shutdown_connections 2020-11-20T12:03:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.486+0000 7f1516baf700 1 -- 172.21.15.44:0/1158673668 wait complete. 2020-11-20T12:03:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2020-11-20T12:03:21.500 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2020-11-20T12:03:20.578628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2747,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41216,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":39040,"kb_avail":754933504,"statfs":{"total":773094113280,"available":773051908096,"internally_reserved":0,"allocated":2162688,"data_stored":837891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39976960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":443,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002981"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873331+0000","last_change":"2020-11-20T12:02:59.871769+0000","last_active":"2020-11-20T12:02:59.873331+0000","last_peered":"2020-11-20T12:02:59.873331+0000","last_clean":"2020-11-20T12:02:59.873331+0000","last_became_active":"2020-11-20T12:02:59.871151+0000","last_became_peered":"2020-11-20T12:02:59.871151+0000","last_unstale":"2020-11-20T12:02:59.873331+0000","last_undegraded":"2020-11-20T12:02:59.873331+0000","last_fullsized":"2020-11-20T12:02:59.873331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.774547+0000","last_change":"2020-11-20T12:02:53.744074+0000","last_active":"2020-11-20T12:02:59.774547+0000","last_peered":"2020-11-20T12:02:59.774547+0000","last_clean":"2020-11-20T12:02:59.774547+0000","last_became_active":"2020-11-20T12:02:53.743596+0000","last_became_peered":"2020-11-20T12:02:53.743596+0000","last_unstale":"2020-11-20T12:02:59.774547+0000","last_undegraded":"2020-11-20T12:02:59.774547+0000","last_fullsized":"2020-11-20T12:02:59.774547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.195451+0000","last_change":"2020-11-20T12:02:56.751652+0000","last_active":"2020-11-20T12:03:00.195451+0000","last_peered":"2020-11-20T12:03:00.195451+0000","last_clean":"2020-11-20T12:03:00.195451+0000","last_became_active":"2020-11-20T12:02:56.751344+0000","last_became_peered":"2020-11-20T12:02:56.751344+0000","last_unstale":"2020-11-20T12:03:00.195451+0000","last_undegraded":"2020-11-20T12:03:00.195451+0000","last_fullsized":"2020-11-20T12:03:00.195451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.780084+0000","last_change":"2020-11-20T12:02:59.779112+0000","last_active":"2020-11-20T12:02:59.780084+0000","last_peered":"2020-11-20T12:02:59.780084+0000","last_clean":"2020-11-20T12:02:59.780084+0000","last_became_active":"2020-11-20T12:02:59.778434+0000","last_became_peered":"2020-11-20T12:02:59.778434+0000","last_unstale":"2020-11-20T12:02:59.780084+0000","last_undegraded":"2020-11-20T12:02:59.780084+0000","last_fullsized":"2020-11-20T12:02:59.780084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.867987+0000","last_change":"2020-11-20T12:02:53.623419+0000","last_active":"2020-11-20T12:02:59.867987+0000","last_peered":"2020-11-20T12:02:59.867987+0000","last_clean":"2020-11-20T12:02:59.867987+0000","last_became_active":"2020-11-20T12:02:53.623135+0000","last_became_peered":"2020-11-20T12:02:53.623135+0000","last_unstale":"2020-11-20T12:02:59.867987+0000","last_undegraded":"2020-11-20T12:02:59.867987+0000","last_fullsized":"2020-11-20T12:02:59.867987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.191541+0000","last_change":"2020-11-20T12:02:56.750366+0000","last_active":"2020-11-20T12:03:00.191541+0000","last_peered":"2020-11-20T12:03:00.191541+0000","last_clean":"2020-11-20T12:03:00.191541+0000","last_became_active":"2020-11-20T12:02:56.749896+0000","last_became_peered":"2020-11-20T12:02:56.749896+0000","last_unstale":"2020-11-20T12:03:00.191541+0000","last_undegraded":"2020-11-20T12:03:00.191541+0000","last_fullsized":"2020-11-20T12:03:00.191541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873473+0000","last_change":"2020-11-20T12:02:59.871656+0000","last_active":"2020-11-20T12:02:59.873473+0000","last_peered":"2020-11-20T12:02:59.873473+0000","last_clean":"2020-11-20T12:02:59.873473+0000","last_became_active":"2020-11-20T12:02:59.871033+0000","last_became_peered":"2020-11-20T12:02:59.871033+0000","last_unstale":"2020-11-20T12:02:59.873473+0000","last_undegraded":"2020-11-20T12:02:59.873473+0000","last_fullsized":"2020-11-20T12:02:59.873473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769916+0000","last_change":"2020-11-20T12:02:53.620206+0000","last_active":"2020-11-20T12:02:59.769916+0000","last_peered":"2020-11-20T12:02:59.769916+0000","last_clean":"2020-11-20T12:02:59.769916+0000","last_became_active":"2020-11-20T12:02:53.619848+0000","last_became_peered":"2020-11-20T12:02:53.619848+0000","last_unstale":"2020-11-20T12:02:59.769916+0000","last_undegraded":"2020-11-20T12:02:59.769916+0000","last_fullsized":"2020-11-20T12:02:59.769916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":"20","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.196841+0000","last_change":"2020-11-20T12:02:56.751843+0000","last_active":"2020-11-20T12:03:00.196841+0000","last_peered":"2020-11-20T12:03:00.196841+0000","last_clean":"2020-11-20T12:03:00.196841+0000","last_became_active":"2020-11-20T12:02:56.751444+0000","last_became_peered":"2020-11-20T12:02:56.751444+0000","last_unstale":"2020-11-20T12:03:00.196841+0000","last_undegraded":"2020-11-20T12:03:00.196841+0000","last_fullsized":"2020-11-20T12:03:00.196841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871493+0000","last_change":"2020-11-20T12:02:59.870900+0000","last_active":"2020-11-20T12:02:59.871493+0000","last_peered":"2020-11-20T12:02:59.871493+0000","last_clean":"2020-11-20T12:02:59.871493+0000","last_became_active":"2020-11-20T12:02:59.870194+0000","last_became_peered":"2020-11-20T12:02:59.870194+0000","last_unstale":"2020-11-20T12:02:59.871493+0000","last_undegraded":"2020-11-20T12:02:59.871493+0000","last_fullsized":"2020-11-20T12:02:59.871493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868625+0000","last_change":"2020-11-20T12:02:53.621673+0000","last_active":"2020-11-20T12:02:59.868625+0000","last_peered":"2020-11-20T12:02:59.868625+0000","last_clean":"2020-11-20T12:02:59.868625+0000","last_became_active":"2020-11-20T12:02:53.621317+0000","last_became_peered":"2020-11-20T12:02:53.621317+0000","last_unstale":"2020-11-20T12:02:59.868625+0000","last_undegraded":"2020-11-20T12:02:59.868625+0000","last_fullsized":"2020-11-20T12:02:59.868625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868438+0000","last_change":"2020-11-20T12:02:49.777943+0000","last_active":"2020-11-20T12:02:59.868438+0000","last_peered":"2020-11-20T12:02:59.868438+0000","last_clean":"2020-11-20T12:02:59.868438+0000","last_became_active":"2020-11-20T12:02:49.777287+0000","last_became_peered":"2020-11-20T12:02:49.777287+0000","last_unstale":"2020-11-20T12:02:59.868438+0000","last_undegraded":"2020-11-20T12:02:59.868438+0000","last_fullsized":"2020-11-20T12:02:59.868438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:45.568237+0000","last_clean_scrub_stamp":"2020-11-20T12:02:45.568237+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":"13","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.182489+0000","last_change":"2020-11-20T12:02:56.751911+0000","last_active":"2020-11-20T12:03:00.182489+0000","last_peered":"2020-11-20T12:03:00.182489+0000","last_clean":"2020-11-20T12:03:00.182489+0000","last_became_active":"2020-11-20T12:02:56.751558+0000","last_became_peered":"2020-11-20T12:02:56.751558+0000","last_unstale":"2020-11-20T12:03:00.182489+0000","last_undegraded":"2020-11-20T12:03:00.182489+0000","last_fullsized":"2020-11-20T12:03:00.182489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.870381+0000","last_change":"2020-11-20T12:02:59.869783+0000","last_active":"2020-11-20T12:02:59.870381+0000","last_peered":"2020-11-20T12:02:59.870381+0000","last_clean":"2020-11-20T12:02:59.870381+0000","last_became_active":"2020-11-20T12:02:59.869158+0000","last_became_peered":"2020-11-20T12:02:59.869158+0000","last_unstale":"2020-11-20T12:02:59.870381+0000","last_undegraded":"2020-11-20T12:02:59.870381+0000","last_fullsized":"2020-11-20T12:02:59.870381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868367+0000","last_change":"2020-11-20T12:02:53.621849+0000","last_active":"2020-11-20T12:02:59.868367+0000","last_peered":"2020-11-20T12:02:59.868367+0000","last_clean":"2020-11-20T12:02:59.868367+0000","last_became_active":"2020-11-20T12:02:53.621479+0000","last_became_peered":"2020-11-20T12:02:53.621479+0000","last_unstale":"2020-11-20T12:02:59.868367+0000","last_undegraded":"2020-11-20T12:02:59.868367+0000","last_fullsized":"2020-11-20T12:02:59.868367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.770789+0000","last_change":"2020-11-20T12:02:56.750191+0000","last_active":"2020-11-20T12:02:59.770789+0000","last_peered":"2020-11-20T12:02:59.770789+0000","last_clean":"2020-11-20T12:02:59.770789+0000","last_became_active":"2020-11-20T12:02:56.749787+0000","last_became_peered":"2020-11-20T12:02:56.749787+0000","last_unstale":"2020-11-20T12:02:59.770789+0000","last_undegraded":"2020-11-20T12:02:59.770789+0000","last_fullsized":"2020-11-20T12:02:59.770789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873151+0000","last_change":"2020-11-20T12:02:59.871503+0000","last_active":"2020-11-20T12:02:59.873151+0000","last_peered":"2020-11-20T12:02:59.873151+0000","last_clean":"2020-11-20T12:02:59.873151+0000","last_became_active":"2020-11-20T12:02:59.870839+0000","last_became_peered":"2020-11-20T12:02:59.870839+0000","last_unstale":"2020-11-20T12:02:59.873151+0000","last_undegraded":"2020-11-20T12:02:59.873151+0000","last_fullsized":"2020-11-20T12:02:59.873151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.868440+0000","last_change":"2020-11-20T12:02:53.746269+0000","last_active":"2020-11-20T12:02:59.868440+0000","last_peered":"2020-11-20T12:02:59.868440+0000","last_clean":"2020-11-20T12:02:59.868440+0000","last_became_active":"2020-11-20T12:02:53.745919+0000","last_became_peered":"2020-11-20T12:02:53.745919+0000","last_unstale":"2020-11-20T12:02:59.868440+0000","last_undegraded":"2020-11-20T12:02:59.868440+0000","last_fullsized":"2020-11-20T12:02:59.868440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"25'10","reported_seq":"22","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:18.511508+0000","last_change":"2020-11-20T12:02:56.752631+0000","last_active":"2020-11-20T12:03:18.511508+0000","last_peered":"2020-11-20T12:03:18.511508+0000","last_clean":"2020-11-20T12:03:18.511508+0000","last_became_active":"2020-11-20T12:02:56.752287+0000","last_became_peered":"2020-11-20T12:02:56.752287+0000","last_unstale":"2020-11-20T12:03:18.511508+0000","last_undegraded":"2020-11-20T12:03:18.511508+0000","last_fullsized":"2020-11-20T12:03:18.511508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1978,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.871787+0000","last_change":"2020-11-20T12:02:59.869976+0000","last_active":"2020-11-20T12:02:59.871787+0000","last_peered":"2020-11-20T12:02:59.871787+0000","last_clean":"2020-11-20T12:02:59.871787+0000","last_became_active":"2020-11-20T12:02:59.869214+0000","last_became_peered":"2020-11-20T12:02:59.869214+0000","last_unstale":"2020-11-20T12:02:59.871787+0000","last_undegraded":"2020-11-20T12:02:59.871787+0000","last_fullsized":"2020-11-20T12:02:59.871787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'6","reported_seq":"18","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192866+0000","last_change":"2020-11-20T12:02:57.108914+0000","last_active":"2020-11-20T12:03:00.192866+0000","last_peered":"2020-11-20T12:03:00.192866+0000","last_clean":"2020-11-20T12:03:00.192866+0000","last_became_active":"2020-11-20T12:02:57.108538+0000","last_became_peered":"2020-11-20T12:02:57.108538+0000","last_unstale":"2020-11-20T12:03:00.192866+0000","last_undegraded":"2020-11-20T12:03:00.192866+0000","last_fullsized":"2020-11-20T12:03:00.192866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.771858+0000","last_change":"2020-11-20T12:02:53.618862+0000","last_active":"2020-11-20T12:02:59.771858+0000","last_peered":"2020-11-20T12:02:59.771858+0000","last_clean":"2020-11-20T12:02:59.771858+0000","last_became_active":"2020-11-20T12:02:53.618438+0000","last_became_peered":"2020-11-20T12:02:53.618438+0000","last_unstale":"2020-11-20T12:02:59.771858+0000","last_undegraded":"2020-11-20T12:02:59.771858+0000","last_fullsized":"2020-11-20T12:02:59.771858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.873226+0000","last_change":"2020-11-20T12:02:59.872290+0000","last_active":"2020-11-20T12:02:59.873226+0000","last_peered":"2020-11-20T12:02:59.873226+0000","last_clean":"2020-11-20T12:02:59.873226+0000","last_became_active":"2020-11-20T12:02:59.870620+0000","last_became_peered":"2020-11-20T12:02:59.870620+0000","last_unstale":"2020-11-20T12:02:59.873226+0000","last_undegraded":"2020-11-20T12:02:59.873226+0000","last_fullsized":"2020-11-20T12:02:59.873226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:58.755877+0000","last_clean_scrub_stamp":"2020-11-20T12:02:58.755877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T12:02:59.769189+0000","last_change":"2020-11-20T12:02:53.620514+0000","last_active":"2020-11-20T12:02:59.769189+0000","last_peered":"2020-11-20T12:02:59.769189+0000","last_clean":"2020-11-20T12:02:59.769189+0000","last_became_active":"2020-11-20T12:02:53.620221+0000","last_became_peered":"2020-11-20T12:02:53.620221+0000","last_unstale":"2020-11-20T12:02:59.769189+0000","last_undegraded":"2020-11-20T12:02:59.769189+0000","last_fullsized":"2020-11-20T12:02:59.769189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:52.606117+0000","last_clean_scrub_stamp":"2020-11-20T12:02:52.606117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"16","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T12:03:00.192738+0000","last_change":"2020-11-20T12:02:56.752071+0000","last_active":"2020-11-20T12:03:00.192738+0000","last_peered":"2020-11-20T12:03:00.192738+0000","last_clean":"2020-11-20T12:03:00.192738+0000","last_became_active":"2020-11-20T12:02:56.751653+0000","last_became_peered":"2020-11-20T12:02:56.751653+0000","last_unstale":"2020-11-20T12:03:00.192738+0000","last_undegraded":"2020-11-20T12:03:00.192738+0000","last_fullsized":"2020-11-20T12:03:00.192738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:02:55.739001+0000","last_clean_scrub_stamp":"2020-11-20T12:02:55.739001+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2747,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5128,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366712,"statfs":{"total":96636764160,"available":96631513088,"internally_reserved":0,"allocated":262144,"data_stored":104105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5196,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366644,"statfs":{"total":96636764160,"available":96631443456,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5140,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366700,"statfs":{"total":96636764160,"available":96631500800,"internally_reserved":0,"allocated":274432,"data_stored":105640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5204,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366636,"statfs":{"total":96636764160,"available":96631435264,"internally_reserved":0,"allocated":274432,"data_stored":106083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":278528,"data_stored":104840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5132,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366708,"statfs":{"total":96636764160,"available":96631508992,"internally_reserved":0,"allocated":266240,"data_stored":104139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":270336,"data_stored":104265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]}]}} 2020-11-20T12:03:21.502 INFO:tasks.mon_thrash.ceph_manager:clean! 2020-11-20T12:03:21.503 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:03:21.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.626+0000 7f5e77462700 1 Processor -- start 2020-11-20T12:03:21.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.626+0000 7f5e77462700 1 -- start start 2020-11-20T12:03:21.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.626+0000 7f5e77462700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5e700aa020 0x7f5e700a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.626+0000 7f5e77462700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5e7013ac60 0x7f5e700a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.626+0000 7f5e77462700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e700a9770 con 0x7f5e700aa400 2020-11-20T12:03:21.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.626+0000 7f5e77462700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e700a56c0 con 0x7f5e7013ac60 2020-11-20T12:03:21.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.626+0000 7f5e77462700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e700a6da0 con 0x7f5e700aa020 2020-11-20T12:03:21.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e759ff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5e7013ac60 0x7f5e700a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e749fd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5e700aa400 0x7f5e700a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40516/0 (socket says 172.21.15.44:40516) 2020-11-20T12:03:21.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e749fd700 1 -- 172.21.15.44:0/1615466792 learned_addr learned my addr 172.21.15.44:0/1615466792 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:21.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e751fe700 1 --2- 172.21.15.44:0/1615466792 >> v2:172.21.15.79:3300/0 conn(0x7f5e700aa020 0x7f5e700a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e759ff700 1 -- 172.21.15.44:0/1615466792 >> v2:172.21.15.79:3300/0 conn(0x7f5e700aa020 msgr2=0x7f5e700a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:21.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e759ff700 1 --2- 172.21.15.44:0/1615466792 >> v2:172.21.15.79:3300/0 conn(0x7f5e700aa020 0x7f5e700a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e759ff700 1 -- 172.21.15.44:0/1615466792 >> v1:172.21.15.44:6789/0 conn(0x7f5e700aa400 legacy=0x7f5e700a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e5ffff700 1 -- 172.21.15.44:0/1615466792 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395868198 0 0) 0x7f5e600010d0 con 0x7f5e700aa400 2020-11-20T12:03:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e759ff700 1 -- 172.21.15.44:0/1615466792 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e700a7040 con 0x7f5e7013ac60 2020-11-20T12:03:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e759ff700 1 --2- 172.21.15.44:0/1615466792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e7013ac60 0x7f5e700a3700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5e700adec0 tx=0x7f5e680091d0).ready entity=mon.0 client_cookie=a385d69d4185154e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e5ffff700 1 -- 172.21.15.44:0/1615466792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800ba70 con 0x7f5e7013ac60 2020-11-20T12:03:21.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e5ffff700 1 -- 172.21.15.44:0/1615466792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6800bbd0 con 0x7f5e7013ac60 2020-11-20T12:03:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.627+0000 7f5e5ffff700 1 -- 172.21.15.44:0/1615466792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68011400 con 0x7f5e7013ac60 2020-11-20T12:03:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 -- 172.21.15.44:0/1615466792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e7013ac60 msgr2=0x7f5e700a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 --2- 172.21.15.44:0/1615466792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e7013ac60 0x7f5e700a3700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5e700adec0 tx=0x7f5e680091d0).stop 2020-11-20T12:03:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 -- 172.21.15.44:0/1615466792 shutdown_connections 2020-11-20T12:03:21.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 --2- 172.21.15.44:0/1615466792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e7013ac60 0x7f5e700a3700 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 --2- 172.21.15.44:0/1615466792 >> v2:172.21.15.79:3300/0 conn(0x7f5e700aa020 0x7f5e700a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 -- 172.21.15.44:0/1615466792 >> 172.21.15.44:0/1615466792 conn(0x7f5e70062690 msgr2=0x7f5e700a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 -- 172.21.15.44:0/1615466792 shutdown_connections 2020-11-20T12:03:21.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.628+0000 7f5e77462700 1 -- 172.21.15.44:0/1615466792 wait complete. 2020-11-20T12:03:21.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.629+0000 7f5e77462700 1 Processor -- start 2020-11-20T12:03:21.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.629+0000 7f5e77462700 1 -- start start 2020-11-20T12:03:21.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e77462700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e700aa020 0x7f5e70066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e77462700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e700aa400 0x7f5e7014b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e77462700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e7013ac60 0x7f5e7014bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e77462700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e70065370 con 0x7f5e700aa400 2020-11-20T12:03:21.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e77462700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e700ad380 con 0x7f5e700aa020 2020-11-20T12:03:21.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e77462700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e700ad620 con 0x7f5e7013ac60 2020-11-20T12:03:21.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e700aa400 0x7f5e7014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e700aa400 0x7f5e7014b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36776/0 (socket says 172.21.15.44:36776) 2020-11-20T12:03:21.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 -- 172.21.15.44:0/2906245640 learned_addr learned my addr 172.21.15.44:0/2906245640 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:21.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e759ff700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e7013ac60 0x7f5e7014bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e751fe700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e700aa020 0x7f5e70066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 -- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e7013ac60 msgr2=0x7f5e7014bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e7013ac60 0x7f5e7014bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 -- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e700aa020 msgr2=0x7f5e70066ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e700aa020 0x7f5e70066ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 -- 172.21.15.44:0/2906245640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70066590 con 0x7f5e700aa400 2020-11-20T12:03:21.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e749fd700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e700aa400 0x7f5e7014b670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f5e6000c760 tx=0x7f5e60007920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/2906245640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e600072e0 con 0x7f5e700aa400 2020-11-20T12:03:21.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/2906245640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e60007440 con 0x7f5e700aa400 2020-11-20T12:03:21.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.630+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e700ace70 con 0x7f5e700aa400 2020-11-20T12:03:21.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.631+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/2906245640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60005800 con 0x7f5e700aa400 2020-11-20T12:03:21.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.631+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7009db60 con 0x7f5e700aa400 2020-11-20T12:03:21.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.632+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e700654e0 con 0x7f5e700aa400 2020-11-20T12:03:21.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.632+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/2906245640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f5e60010070 con 0x7f5e700aa400 2020-11-20T12:03:21.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.632+0000 7f5e5d7fa700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e48056d50 0x7f5e48059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.632+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/2906245640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5e6006f630 con 0x7f5e700aa400 2020-11-20T12:03:21.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.632+0000 7f5e751fe700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e48056d50 0x7f5e48059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.633+0000 7f5e751fe700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e48056d50 0x7f5e48059210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f5e70067eb0 tx=0x7f5e6c006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.636+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/2906245640 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5e700654e0 con 0x7f5e700aa400 2020-11-20T12:03:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.810+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e700a9770 con 0x7f5e700aa400 2020-11-20T12:03:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.810+0000 7f5e5d7fa700 1 -- 172.21.15.44:0/2906245640 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17039 (secure 0 0 0) 0x7f5e700a9770 con 0x7f5e700aa400 2020-11-20T12:03:21.814 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:21.815 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","created":"2020-11-20T12:02:41.040478+0000","modified":"2020-11-20T12:03:00.178525+0000","last_up_change":"2020-11-20T12:02:48.585859+0000","last_in_change":"2020-11-20T12:02:48.585859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:02:44.611733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:02:52.195125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:02:55.303534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:02:57.791405+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2856c45e-54ed-4130-bd9f-898ffadae60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":24728},{"type":"v1","addr":"172.21.15.44:6811","nonce":24728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":24728},{"type":"v1","addr":"172.21.15.44:6816","nonce":24728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":24728},{"type":"v1","addr":"172.21.15.44:6823","nonce":24728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":24728},{"type":"v1","addr":"172.21.15.44:6820","nonce":24728}]},"public_addr":"172.21.15.44:6811/24728","cluster_addr":"172.21.15.44:6816/24728","heartbeat_back_addr":"172.21.15.44:6823/24728","heartbeat_front_addr":"172.21.15.44:6820/24728","state":["exists","up"]},{"osd":1,"uuid":"9af002f0-d9d2-4362-8855-73c63682dc0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":24729},{"type":"v1","addr":"172.21.15.44:6810","nonce":24729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":24729},{"type":"v1","addr":"172.21.15.44:6814","nonce":24729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6819","nonce":24729},{"type":"v1","addr":"172.21.15.44:6821","nonce":24729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6815","nonce":24729},{"type":"v1","addr":"172.21.15.44:6817","nonce":24729}]},"public_addr":"172.21.15.44:6810/24729","cluster_addr":"172.21.15.44:6814/24729","heartbeat_back_addr":"172.21.15.44:6821/24729","heartbeat_front_addr":"172.21.15.44:6817/24729","state":["exists","up"]},{"osd":2,"uuid":"475fc90c-4230-4093-9438-34ba65bdfdde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":24727},{"type":"v1","addr":"172.21.15.44:6801","nonce":24727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":24727},{"type":"v1","addr":"172.21.15.44:6803","nonce":24727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":24727},{"type":"v1","addr":"172.21.15.44:6807","nonce":24727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":24727},{"type":"v1","addr":"172.21.15.44:6805","nonce":24727}]},"public_addr":"172.21.15.44:6801/24727","cluster_addr":"172.21.15.44:6803/24727","heartbeat_back_addr":"172.21.15.44:6807/24727","heartbeat_front_addr":"172.21.15.44:6805/24727","state":["exists","up"]},{"osd":3,"uuid":"f951d29a-4ead-4d15-8e19-178a8eef2b91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":24740},{"type":"v1","addr":"172.21.15.44:6825","nonce":24740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":24740},{"type":"v1","addr":"172.21.15.44:6827","nonce":24740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":24740},{"type":"v1","addr":"172.21.15.44:6831","nonce":24740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":24740},{"type":"v1","addr":"172.21.15.44:6829","nonce":24740}]},"public_addr":"172.21.15.44:6825/24740","cluster_addr":"172.21.15.44:6827/24740","heartbeat_back_addr":"172.21.15.44:6831/24740","heartbeat_front_addr":"172.21.15.44:6829/24740","state":["exists","up"]},{"osd":4,"uuid":"74c35433-a1af-4165-ba72-6935b43df900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":24937},{"type":"v1","addr":"172.21.15.79:6817","nonce":24937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":24937},{"type":"v1","addr":"172.21.15.79:6819","nonce":24937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":24937},{"type":"v1","addr":"172.21.15.79:6823","nonce":24937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":24937},{"type":"v1","addr":"172.21.15.79:6821","nonce":24937}]},"public_addr":"172.21.15.79:6817/24937","cluster_addr":"172.21.15.79:6819/24937","heartbeat_back_addr":"172.21.15.79:6823/24937","heartbeat_front_addr":"172.21.15.79:6821/24937","state":["exists","up"]},{"osd":5,"uuid":"9495ddb8-00b5-4d73-acb2-86bf2949700f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":24927},{"type":"v1","addr":"172.21.15.79:6801","nonce":24927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":24927},{"type":"v1","addr":"172.21.15.79:6803","nonce":24927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":24927},{"type":"v1","addr":"172.21.15.79:6807","nonce":24927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":24927},{"type":"v1","addr":"172.21.15.79:6805","nonce":24927}]},"public_addr":"172.21.15.79:6801/24927","cluster_addr":"172.21.15.79:6803/24927","heartbeat_back_addr":"172.21.15.79:6807/24927","heartbeat_front_addr":"172.21.15.79:6805/24927","state":["exists","up"]},{"osd":6,"uuid":"6502bfc3-d91f-407c-a2e7-83bf8472704a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":24928},{"type":"v1","addr":"172.21.15.79:6809","nonce":24928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":24928},{"type":"v1","addr":"172.21.15.79:6811","nonce":24928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":24928},{"type":"v1","addr":"172.21.15.79:6815","nonce":24928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":24928},{"type":"v1","addr":"172.21.15.79:6813","nonce":24928}]},"public_addr":"172.21.15.79:6809/24928","cluster_addr":"172.21.15.79:6811/24928","heartbeat_back_addr":"172.21.15.79:6815/24928","heartbeat_front_addr":"172.21.15.79:6813/24928","state":["exists","up"]},{"osd":7,"uuid":"7d22fe22-d60a-468f-b9c6-bfe4d9f50c37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":24960},{"type":"v1","addr":"172.21.15.79:6825","nonce":24960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":24960},{"type":"v1","addr":"172.21.15.79:6827","nonce":24960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":24960},{"type":"v1","addr":"172.21.15.79:6831","nonce":24960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":24960},{"type":"v1","addr":"172.21.15.79:6829","nonce":24960}]},"public_addr":"172.21.15.79:6825/24960","cluster_addr":"172.21.15.79:6827/24960","heartbeat_back_addr":"172.21.15.79:6831/24960","heartbeat_front_addr":"172.21.15.79:6829/24960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.617263+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.610559+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.390685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.614579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.556883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.631167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:02:46.619946+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:03:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e48056d50 msgr2=0x7f5e48059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e48056d50 0x7f5e48059210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f5e70067eb0 tx=0x7f5e6c006df0).stop 2020-11-20T12:03:21.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e700aa400 msgr2=0x7f5e7014b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e700aa400 0x7f5e7014b670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f5e6000c760 tx=0x7f5e60007920).stop 2020-11-20T12:03:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 shutdown_connections 2020-11-20T12:03:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e48056d50 0x7f5e48059210 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e7013ac60 0x7f5e7014bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e700aa020 0x7f5e70066ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 --2- 172.21.15.44:0/2906245640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e700aa400 0x7f5e7014b670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 >> 172.21.15.44:0/2906245640 conn(0x7f5e70062690 msgr2=0x7f5e700a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 shutdown_connections 2020-11-20T12:03:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.812+0000 7f5e77462700 1 -- 172.21.15.44:0/2906245640 wait complete. 2020-11-20T12:03:21.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:02:44.611733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:02:52.195125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:02:55.303534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:02:57.791405+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:03:21.828 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:03:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 Processor -- start 2020-11-20T12:03:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 -- start start 2020-11-20T12:03:21.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f60fc060a40 0x7f60fc064350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a45f0 0x7f60fc0a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f60fc0a96a0 0x7f60fc0a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc0a64d0 con 0x7f60fc060a40 2020-11-20T12:03:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc0a5860 con 0x7f60fc0a96a0 2020-11-20T12:03:21.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610199f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0a4150 con 0x7f60fc0a45f0 2020-11-20T12:03:21.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.954+0000 7f610099d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f60fc060a40 0x7f60fc064350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f60fc060a40 0x7f60fc064350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36782/0 (socket says 172.21.15.44:36782) 2020-11-20T12:03:21.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 -- 172.21.15.44:0/3729515356 learned_addr learned my addr 172.21.15.44:0/3729515356 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610119e700 1 --2- 172.21.15.44:0/3729515356 >> v2:172.21.15.79:3300/0 conn(0x7f60fc0a96a0 0x7f60fc0a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f60fbfff700 1 --2- 172.21.15.44:0/3729515356 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a45f0 0x7f60fc0a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 -- 172.21.15.44:0/3729515356 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a45f0 msgr2=0x7f60fc0a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 --2- 172.21.15.44:0/3729515356 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a45f0 0x7f60fc0a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 -- 172.21.15.44:0/3729515356 >> v2:172.21.15.79:3300/0 conn(0x7f60fc0a96a0 msgr2=0x7f60fc0a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 --2- 172.21.15.44:0/3729515356 >> v2:172.21.15.79:3300/0 conn(0x7f60fc0a96a0 0x7f60fc0a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 -- 172.21.15.44:0/3729515356 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc060e20 con 0x7f60fc060a40 2020-11-20T12:03:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f610099d700 1 --2- 172.21.15.44:0/3729515356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc060a40 0x7f60fc064350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f60fc062db0 tx=0x7f60ec006eb0).ready entity=mon.0 client_cookie=8995038385e1a72e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f60fb7fe700 1 -- 172.21.15.44:0/3729515356 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0098d0 con 0x7f60fc060a40 2020-11-20T12:03:21.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f60fb7fe700 1 -- 172.21.15.44:0/3729515356 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec009a30 con 0x7f60fc060a40 2020-11-20T12:03:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.955+0000 7f60fb7fe700 1 -- 172.21.15.44:0/3729515356 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec009db0 con 0x7f60fc060a40 2020-11-20T12:03:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 -- 172.21.15.44:0/3729515356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc060a40 msgr2=0x7f60fc064350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 --2- 172.21.15.44:0/3729515356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc060a40 0x7f60fc064350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f60fc062db0 tx=0x7f60ec006eb0).stop 2020-11-20T12:03:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 -- 172.21.15.44:0/3729515356 shutdown_connections 2020-11-20T12:03:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 --2- 172.21.15.44:0/3729515356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc060a40 0x7f60fc064350 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 --2- 172.21.15.44:0/3729515356 >> v2:172.21.15.79:3300/0 conn(0x7f60fc0a96a0 0x7f60fc0a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 --2- 172.21.15.44:0/3729515356 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a45f0 0x7f60fc0a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 -- 172.21.15.44:0/3729515356 >> 172.21.15.44:0/3729515356 conn(0x7f60fc061d80 msgr2=0x7f60fc060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:21.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 -- 172.21.15.44:0/3729515356 shutdown_connections 2020-11-20T12:03:21.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.956+0000 7f610199f700 1 -- 172.21.15.44:0/3729515356 wait complete. 2020-11-20T12:03:21.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 Processor -- start 2020-11-20T12:03:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 -- start start 2020-11-20T12:03:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60fc060a40 0x7f60fc0a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc0a45f0 0x7f60fc0a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a96a0 0x7f60fc0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc14d1b0 con 0x7f60fc0a45f0 2020-11-20T12:03:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0643f0 con 0x7f60fc060a40 2020-11-20T12:03:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f610199f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc064690 con 0x7f60fc0a96a0 2020-11-20T12:03:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f60fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc0a45f0 0x7f60fc0a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.957+0000 7f60fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc0a45f0 0x7f60fc0a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36788/0 (socket says 172.21.15.44:36788) 2020-11-20T12:03:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60fbfff700 1 -- 172.21.15.44:0/240558742 learned_addr learned my addr 172.21.15.44:0/240558742 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f610099d700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60fc060a40 0x7f60fc0a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f610119e700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a96a0 0x7f60fc0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60fbfff700 1 -- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a96a0 msgr2=0x7f60fc0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60fbfff700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a96a0 0x7f60fc0ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60fbfff700 1 -- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60fc060a40 msgr2=0x7f60fc0a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60fbfff700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60fc060a40 0x7f60fc0a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60fbfff700 1 -- 172.21.15.44:0/240558742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc0ac5e0 con 0x7f60fc0a45f0 2020-11-20T12:03:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60fbfff700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc0a45f0 0x7f60fc0a3090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f60f000db60 tx=0x7f60f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60f8ff9700 1 -- 172.21.15.44:0/240558742 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0009790 con 0x7f60fc0a45f0 2020-11-20T12:03:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60f8ff9700 1 -- 172.21.15.44:0/240558742 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f00098f0 con 0x7f60fc0a45f0 2020-11-20T12:03:21.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc0a3580 con 0x7f60fc0a45f0 2020-11-20T12:03:21.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f60f8ff9700 1 -- 172.21.15.44:0/240558742 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0009c90 con 0x7f60fc0a45f0 2020-11-20T12:03:21.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.958+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc0abd50 con 0x7f60fc0a45f0 2020-11-20T12:03:21.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.959+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc0ac7d0 con 0x7f60fc0a45f0 2020-11-20T12:03:21.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.959+0000 7f60f8ff9700 1 -- 172.21.15.44:0/240558742 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f60f0013480 con 0x7f60fc0a45f0 2020-11-20T12:03:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.960+0000 7f60f8ff9700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f60d4056d20 0x7f60d40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.960+0000 7f60f8ff9700 1 -- 172.21.15.44:0/240558742 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f60f00706b0 con 0x7f60fc0a45f0 2020-11-20T12:03:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.960+0000 7f610099d700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f60d4056d20 0x7f60d40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.960+0000 7f610099d700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f60d4056d20 0x7f60d40591e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f60ec006ef0 tx=0x7f60ec015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:21.963+0000 7f60f8ff9700 1 -- 172.21.15.44:0/240558742 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f60fc0ac7d0 con 0x7f60fc0a45f0 2020-11-20T12:03:22.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.138+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f60fc0ac7d0 con 0x7f60fc0a45f0 2020-11-20T12:03:22.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.138+0000 7f60f8ff9700 1 -- 172.21.15.44:0/240558742 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f60fc0ac7d0 con 0x7f60fc0a45f0 2020-11-20T12:03:22.142 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2020-11-20T12:03:22.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f60d4056d20 msgr2=0x7f60d40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f60d4056d20 0x7f60d40591e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f60ec006ef0 tx=0x7f60ec015000).stop 2020-11-20T12:03:22.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc0a45f0 msgr2=0x7f60fc0a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc0a45f0 0x7f60fc0a3090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f60f000db60 tx=0x7f60f0008040).stop 2020-11-20T12:03:22.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 shutdown_connections 2020-11-20T12:03:22.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f60d4056d20 0x7f60d40591e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60fc0a96a0 0x7f60fc0ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60fc060a40 0x7f60fc0a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 --2- 172.21.15.44:0/240558742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60fc0a45f0 0x7f60fc0a3090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 >> 172.21.15.44:0/240558742 conn(0x7f60fc061d80 msgr2=0x7f60fc1390d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:22.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 shutdown_connections 2020-11-20T12:03:22.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.140+0000 7f610199f700 1 -- 172.21.15.44:0/240558742 wait complete. 2020-11-20T12:03:22.155 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:03:22.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.279+0000 7f7dbf343700 1 Processor -- start 2020-11-20T12:03:22.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.279+0000 7f7dbf343700 1 -- start start 2020-11-20T12:03:22.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbf343700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad930 0x7f7db80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbf343700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7db80a8ea0 0x7f7db80a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbf343700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db80a7200 con 0x7f7db80ad550 2020-11-20T12:03:22.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbf343700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db8135240 con 0x7f7db80a8ea0 2020-11-20T12:03:22.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbf343700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80a6cc0 con 0x7f7db80ad930 2020-11-20T12:03:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbd8e0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7db80a8ea0 0x7f7db80a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbd8e0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7db80a8ea0 0x7f7db80a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36798/0 (socket says 172.21.15.44:36798) 2020-11-20T12:03:22.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3294849366 learned_addr learned my addr 172.21.15.44:0/3294849366 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:22.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbc8de700 1 --2- 172.21.15.44:0/3294849366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad930 0x7f7db80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3294849366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad930 msgr2=0x7f7db80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:22.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbd8e0700 1 --2- 172.21.15.44:0/3294849366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad930 0x7f7db80a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3294849366 >> v1:172.21.15.79:6789/0 conn(0x7f7db80ad550 legacy=0x7f7db80a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3294849366 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db80a6f60 con 0x7f7db80a8ea0 2020-11-20T12:03:22.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.280+0000 7f7dbc8de700 1 --2- 172.21.15.44:0/3294849366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad930 0x7f7db80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbd8e0700 1 --2- 172.21.15.44:0/3294849366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80a8ea0 0x7f7db80a3bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f7db0009d80 tx=0x7f7db0009290).ready entity=mon.0 client_cookie=b6057828ea147ec0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7da7fff700 1 -- 172.21.15.44:0/3294849366 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db000bc40 con 0x7f7db80a8ea0 2020-11-20T12:03:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7da7fff700 1 -- 172.21.15.44:0/3294849366 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db000bda0 con 0x7f7db80a8ea0 2020-11-20T12:03:22.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7da7fff700 1 -- 172.21.15.44:0/3294849366 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0011610 con 0x7f7db80a8ea0 2020-11-20T12:03:22.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 -- 172.21.15.44:0/3294849366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80a8ea0 msgr2=0x7f7db80a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3294849366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80a8ea0 0x7f7db80a3bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f7db0009d80 tx=0x7f7db0009290).stop 2020-11-20T12:03:22.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 -- 172.21.15.44:0/3294849366 shutdown_connections 2020-11-20T12:03:22.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3294849366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80a8ea0 0x7f7db80a3bf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3294849366 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad930 0x7f7db80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 -- 172.21.15.44:0/3294849366 >> 172.21.15.44:0/3294849366 conn(0x7f7db80624b0 msgr2=0x7f7db80ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:22.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 -- 172.21.15.44:0/3294849366 shutdown_connections 2020-11-20T12:03:22.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.281+0000 7f7dbf343700 1 -- 172.21.15.44:0/3294849366 wait complete. 2020-11-20T12:03:22.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.282+0000 7f7dbf343700 1 Processor -- start 2020-11-20T12:03:22.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.282+0000 7f7dbf343700 1 -- start start 2020-11-20T12:03:22.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbf343700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80a8ea0 0x7f7db813d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbf343700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad550 0x7f7db81371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbf343700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80ad930 0x7f7db8138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbf343700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db813b490 con 0x7f7db80ad930 2020-11-20T12:03:22.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbf343700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db813b730 con 0x7f7db80a8ea0 2020-11-20T12:03:22.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbf343700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db813b9d0 con 0x7f7db80ad550 2020-11-20T12:03:22.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80ad930 0x7f7db8138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80ad930 0x7f7db8138dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36804/0 (socket says 172.21.15.44:36804) 2020-11-20T12:03:22.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3631063711 learned_addr learned my addr 172.21.15.44:0/3631063711 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:22.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbc8de700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad550 0x7f7db81371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd0df700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80a8ea0 0x7f7db813d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad550 msgr2=0x7f7db81371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad550 0x7f7db81371c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80a8ea0 msgr2=0x7f7db813d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80a8ea0 0x7f7db813d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.283+0000 7f7dbd8e0700 1 -- 172.21.15.44:0/3631063711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db813b110 con 0x7f7db80ad930 2020-11-20T12:03:22.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.284+0000 7f7dbd8e0700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80ad930 0x7f7db8138dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f7db00090f0 tx=0x7f7db0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.284+0000 7f7da57fa700 1 -- 172.21.15.44:0/3631063711 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db000bc40 con 0x7f7db80ad930 2020-11-20T12:03:22.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.284+0000 7f7da57fa700 1 -- 172.21.15.44:0/3631063711 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db00032a0 con 0x7f7db80ad930 2020-11-20T12:03:22.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.284+0000 7f7da57fa700 1 -- 172.21.15.44:0/3631063711 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0010cf0 con 0x7f7db80ad930 2020-11-20T12:03:22.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.284+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db80a2d10 con 0x7f7db80ad930 2020-11-20T12:03:22.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.284+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db8139500 con 0x7f7db80ad930 2020-11-20T12:03:22.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.285+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db813b300 con 0x7f7db80ad930 2020-11-20T12:03:22.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.285+0000 7f7da57fa700 1 -- 172.21.15.44:0/3631063711 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7db0017070 con 0x7f7db80ad930 2020-11-20T12:03:22.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.285+0000 7f7da57fa700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7d90056d20 0x7f7d900591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.285+0000 7f7da57fa700 1 -- 172.21.15.44:0/3631063711 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f7db0072600 con 0x7f7db80ad930 2020-11-20T12:03:22.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.285+0000 7f7dbd0df700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7d90056d20 0x7f7d900591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.286+0000 7f7dbd0df700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7d90056d20 0x7f7d900591e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7da8003610 tx=0x7f7da8000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.289+0000 7f7da57fa700 1 -- 172.21.15.44:0/3631063711 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7db00478f0 con 0x7f7db80ad930 2020-11-20T12:03:22.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.463+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7db80a2ea0 con 0x7f7db80ad930 2020-11-20T12:03:22.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.463+0000 7f7da57fa700 1 -- 172.21.15.44:0/3631063711 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7db0015020 con 0x7f7db80ad930 2020-11-20T12:03:22.468 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:22.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7d90056d20 msgr2=0x7f7d900591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7d90056d20 0x7f7d900591e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7da8003610 tx=0x7f7da8000b80).stop 2020-11-20T12:03:22.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80ad930 msgr2=0x7f7db8138dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80ad930 0x7f7db8138dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f7db00090f0 tx=0x7f7db0016000).stop 2020-11-20T12:03:22.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 shutdown_connections 2020-11-20T12:03:22.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7d90056d20 0x7f7d900591e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db80ad930 0x7f7db8138dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80a8ea0 0x7f7db813d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 --2- 172.21.15.44:0/3631063711 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7db80ad550 0x7f7db81371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 >> 172.21.15.44:0/3631063711 conn(0x7f7db80624b0 msgr2=0x7f7db805ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 shutdown_connections 2020-11-20T12:03:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.465+0000 7f7dbf343700 1 -- 172.21.15.44:0/3631063711 wait complete. 2020-11-20T12:03:22.480 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:03:22.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.620+0000 7f93dc120700 1 Processor -- start 2020-11-20T12:03:22.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93dc120700 1 -- start start 2020-11-20T12:03:22.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93dc120700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f93d4137cc0 0x7f93d41428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93dc120700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d40a4280 con 0x7f93d4060170 2020-11-20T12:03:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93dc120700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d40a5d10 con 0x7f93d4060550 2020-11-20T12:03:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93dc120700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d40a4dc0 con 0x7f93d4137cc0 2020-11-20T12:03:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93da6bd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f93d4137cc0 0x7f93d41428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93d9ebc700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f93d4060170 0x7f93d40a46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40556/0 (socket says 172.21.15.44:40556) 2020-11-20T12:03:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93da6bd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f93d4137cc0 0x7f93d41428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36808/0 (socket says 172.21.15.44:36808) 2020-11-20T12:03:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.621+0000 7f93d9ebc700 1 -- 172.21.15.44:0/2891380618 learned_addr learned my addr 172.21.15.44:0/2891380618 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93da6bd700 1 -- 172.21.15.44:0/2891380618 >> v1:172.21.15.79:6789/0 conn(0x7f93d4060550 legacy=0x7f93d41375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93da6bd700 1 -- 172.21.15.44:0/2891380618 >> v1:172.21.15.44:6789/0 conn(0x7f93d4060170 legacy=0x7f93d40a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93da6bd700 1 -- 172.21.15.44:0/2891380618 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d40a6840 con 0x7f93d4137cc0 2020-11-20T12:03:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93da6bd700 1 --2- 172.21.15.44:0/2891380618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4137cc0 0x7f93d41428a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f93cc009c90 tx=0x7f93cc0091d0).ready entity=mon.0 client_cookie=96536c8a24aa63a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93d8eba700 1 -- 172.21.15.44:0/2891380618 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00bbc0 con 0x7f93d4137cc0 2020-11-20T12:03:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93d8eba700 1 -- 172.21.15.44:0/2891380618 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc00bd20 con 0x7f93d4137cc0 2020-11-20T12:03:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93d8eba700 1 -- 172.21.15.44:0/2891380618 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc010680 con 0x7f93d4137cc0 2020-11-20T12:03:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93dc120700 1 -- 172.21.15.44:0/2891380618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4137cc0 msgr2=0x7f93d41428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93dc120700 1 --2- 172.21.15.44:0/2891380618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4137cc0 0x7f93d41428a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f93cc009c90 tx=0x7f93cc0091d0).stop 2020-11-20T12:03:22.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93dc120700 1 -- 172.21.15.44:0/2891380618 shutdown_connections 2020-11-20T12:03:22.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93dc120700 1 --2- 172.21.15.44:0/2891380618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4137cc0 0x7f93d41428a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.622+0000 7f93dc120700 1 -- 172.21.15.44:0/2891380618 >> 172.21.15.44:0/2891380618 conn(0x7f93d40624b0 msgr2=0x7f93d40ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:22.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.623+0000 7f93dc120700 1 -- 172.21.15.44:0/2891380618 shutdown_connections 2020-11-20T12:03:22.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.623+0000 7f93dc120700 1 -- 172.21.15.44:0/2891380618 wait complete. 2020-11-20T12:03:22.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.623+0000 7f93dc120700 1 Processor -- start 2020-11-20T12:03:22.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93dc120700 1 -- start start 2020-11-20T12:03:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93dc120700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4060170 0x7f93d40abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93dc120700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d4060550 0x7f93d4066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93dc120700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93d4137cc0 0x7f93d4149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93dc120700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d414ac10 con 0x7f93d4060170 2020-11-20T12:03:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93dc120700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d4066c10 con 0x7f93d4060550 2020-11-20T12:03:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93dc120700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d40ac810 con 0x7f93d4137cc0 2020-11-20T12:03:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93d9ebc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4060170 0x7f93d40abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93d9ebc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4060170 0x7f93d40abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36812/0 (socket says 172.21.15.44:36812) 2020-11-20T12:03:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93d9ebc700 1 -- 172.21.15.44:0/2497796291 learned_addr learned my addr 172.21.15.44:0/2497796291 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93d96bb700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d4060550 0x7f93d4066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.624+0000 7f93da6bd700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93d4137cc0 0x7f93d4149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93d9ebc700 1 -- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93d4137cc0 msgr2=0x7f93d4149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93d9ebc700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93d4137cc0 0x7f93d4149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93d9ebc700 1 -- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d4060550 msgr2=0x7f93d4066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93d9ebc700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d4060550 0x7f93d4066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93d9ebc700 1 -- 172.21.15.44:0/2497796291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d40acab0 con 0x7f93d4060170 2020-11-20T12:03:22.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93d9ebc700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4060170 0x7f93d40abe90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f93c400c710 tx=0x7f93c4007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93c1ffb700 1 -- 172.21.15.44:0/2497796291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c40072e0 con 0x7f93d4060170 2020-11-20T12:03:22.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93c1ffb700 1 -- 172.21.15.44:0/2497796291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c4007440 con 0x7f93d4060170 2020-11-20T12:03:22.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d4065070 con 0x7f93d4060170 2020-11-20T12:03:22.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93c1ffb700 1 -- 172.21.15.44:0/2497796291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4005800 con 0x7f93d4060170 2020-11-20T12:03:22.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.625+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d40acfd0 con 0x7f93d4060170 2020-11-20T12:03:22.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.626+0000 7f93c1ffb700 1 -- 172.21.15.44:0/2497796291 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f93c401a060 con 0x7f93d4060170 2020-11-20T12:03:22.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.626+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d4134d50 con 0x7f93d4060170 2020-11-20T12:03:22.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.626+0000 7f93c1ffb700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93ac056d20 0x7f93ac0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.626+0000 7f93c1ffb700 1 -- 172.21.15.44:0/2497796291 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f93c406f930 con 0x7f93d4060170 2020-11-20T12:03:22.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.627+0000 7f93d96bb700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93ac056d20 0x7f93ac0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.627+0000 7f93d96bb700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93ac056d20 0x7f93ac0591e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f93d40abc80 tx=0x7f93d0007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.630+0000 7f93c1ffb700 1 -- 172.21.15.44:0/2497796291 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f93c4044b10 con 0x7f93d4060170 2020-11-20T12:03:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.804+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93d40615e0 con 0x7f93d4060170 2020-11-20T12:03:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.804+0000 7f93c1ffb700 1 -- 172.21.15.44:0/2497796291 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f93c40180a0 con 0x7f93d4060170 2020-11-20T12:03:22.809 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:22.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93ac056d20 msgr2=0x7f93ac0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93ac056d20 0x7f93ac0591e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f93d40abc80 tx=0x7f93d0007b90).stop 2020-11-20T12:03:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4060170 msgr2=0x7f93d40abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4060170 0x7f93d40abe90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f93c400c710 tx=0x7f93c4007c40).stop 2020-11-20T12:03:22.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 shutdown_connections 2020-11-20T12:03:22.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93ac056d20 0x7f93ac0591e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93d4137cc0 0x7f93d4149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f93d4060170 0x7f93d40abe90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 --2- 172.21.15.44:0/2497796291 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d4060550 0x7f93d4066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 >> 172.21.15.44:0/2497796291 conn(0x7f93d40624b0 msgr2=0x7f93d4138450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:22.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 shutdown_connections 2020-11-20T12:03:22.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.806+0000 7f93dc120700 1 -- 172.21.15.44:0/2497796291 wait complete. 2020-11-20T12:03:22.821 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:03:22.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd182706700 1 Processor -- start 2020-11-20T12:03:22.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd182706700 1 -- start start 2020-11-20T12:03:22.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd182706700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd17c0a3ad0 0x7fd17c1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd182706700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 0x7fd17c149770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd182706700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c142d40 con 0x7fd17c1428c0 2020-11-20T12:03:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd182706700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c0a57f0 con 0x7fd17c0a3ad0 2020-11-20T12:03:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd182706700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0a5a90 con 0x7fd17c13c580 2020-11-20T12:03:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd17b7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd17c1428c0 0x7fd17c13be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40568/0 (socket says 172.21.15.44:40568) 2020-11-20T12:03:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.947+0000 7fd17b7fe700 1 -- 172.21.15.44:0/1918796617 learned_addr learned my addr 172.21.15.44:0/1918796617 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:22.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd180ca3700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 0x7fd17c149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17bfff700 1 --2- 172.21.15.44:0/1918796617 >> v2:172.21.15.79:3300/0 conn(0x7fd17c0a3ad0 0x7fd17c1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393764893 0 0) 0x7fd17c142d40 con 0x7fd17c1428c0 2020-11-20T12:03:22.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168003240 con 0x7fd17c1428c0 2020-11-20T12:03:22.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4154822076 0 0) 0x7fd168003240 con 0x7fd17c1428c0 2020-11-20T12:03:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 msgr2=0x7fd17c149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 0x7fd17c149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 >> v2:172.21.15.79:3300/0 conn(0x7fd17c0a3ad0 msgr2=0x7fd17c1423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 --2- 172.21.15.44:0/1918796617 >> v2:172.21.15.79:3300/0 conn(0x7fd17c0a3ad0 0x7fd17c1423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0a5d30 con 0x7fd17c1428c0 2020-11-20T12:03:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fd1700014e0 con 0x7fd17c1428c0 2020-11-20T12:03:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 >> v1:172.21.15.44:6789/0 conn(0x7fd17c1428c0 legacy=0x7fd17c13be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168003730 0x7fd168003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 0x7fd1680044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1680049d0 0x7fd168004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0a57f0 con 0x7fd168003730 2020-11-20T12:03:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0a5a90 con 0x7fd1680049d0 2020-11-20T12:03:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680041d0 con 0x7fd17c13c580 2020-11-20T12:03:22.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17bfff700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168003730 0x7fd168003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd180ca3700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1680049d0 0x7fd168004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.948+0000 7fd17b7fe700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 0x7fd1680044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17bfff700 1 -- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 msgr2=0x7fd1680044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:22.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17bfff700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13c580 0x7fd1680044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17bfff700 1 -- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1680049d0 msgr2=0x7fd168004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17bfff700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1680049d0 0x7fd168004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17bfff700 1 -- 172.21.15.44:0/1918796617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168003240 con 0x7fd168003730 2020-11-20T12:03:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17bfff700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168003730 0x7fd168003b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd16c00bc00 tx=0x7fd16c006df0).ready entity=mon.0 client_cookie=359b04ebd254ea81 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00b310 con 0x7fd168003730 2020-11-20T12:03:22.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c00ea30 con 0x7fd168003730 2020-11-20T12:03:22.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd17affd700 1 -- 172.21.15.44:0/1918796617 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00edb0 con 0x7fd168003730 2020-11-20T12:03:22.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd182706700 1 -- 172.21.15.44:0/1918796617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168003730 msgr2=0x7fd168003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd182706700 1 --2- 172.21.15.44:0/1918796617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd168003730 0x7fd168003b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd16c00bc00 tx=0x7fd16c006df0).stop 2020-11-20T12:03:22.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd180ca3700 1 -- 172.21.15.44:0/1918796617 reap_dead start 2020-11-20T12:03:22.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd182706700 1 -- 172.21.15.44:0/1918796617 shutdown_connections 2020-11-20T12:03:22.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd182706700 1 -- 172.21.15.44:0/1918796617 >> 172.21.15.44:0/1918796617 conn(0x7fd17c0624b0 msgr2=0x7fd17c0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:22.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd182706700 1 -- 172.21.15.44:0/1918796617 shutdown_connections 2020-11-20T12:03:22.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.949+0000 7fd182706700 1 -- 172.21.15.44:0/1918796617 wait complete. 2020-11-20T12:03:22.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.950+0000 7fd182706700 1 Processor -- start 2020-11-20T12:03:22.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.950+0000 7fd182706700 1 -- start start 2020-11-20T12:03:22.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd182706700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd17c140690 0x7fd17c152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd182706700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c140a70 0x7fd17c0ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd182706700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13cc20 0x7fd17c13f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd182706700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0abf40 con 0x7fd17c140a70 2020-11-20T12:03:22.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd182706700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0ac1e0 con 0x7fd17c140690 2020-11-20T12:03:22.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd182706700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0ab200 con 0x7fd17c13cc20 2020-11-20T12:03:22.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c140a70 0x7fd17c0ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c140a70 0x7fd17c0ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36832/0 (socket says 172.21.15.44:36832) 2020-11-20T12:03:22.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 -- 172.21.15.44:0/3229416250 learned_addr learned my addr 172.21.15.44:0/3229416250 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:22.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd180ca3700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13cc20 0x7fd17c13f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17bfff700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd17c140690 0x7fd17c152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 -- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13cc20 msgr2=0x7fd17c13f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13cc20 0x7fd17c13f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 -- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd17c140690 msgr2=0x7fd17c152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:22.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd17c140690 0x7fd17c152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:22.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.951+0000 7fd17b7fe700 1 -- 172.21.15.44:0/3229416250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0ab3c0 con 0x7fd17c140a70 2020-11-20T12:03:22.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.952+0000 7fd17b7fe700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c140a70 0x7fd17c0ac680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd170000f90 tx=0x7fd170007800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.952+0000 7fd15bfff700 1 -- 172.21.15.44:0/3229416250 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000f960 con 0x7fd17c140a70 2020-11-20T12:03:22.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.952+0000 7fd15bfff700 1 -- 172.21.15.44:0/3229416250 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17000fac0 con 0x7fd17c140a70 2020-11-20T12:03:22.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.952+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c140200 con 0x7fd17c140a70 2020-11-20T12:03:22.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.952+0000 7fd15bfff700 1 -- 172.21.15.44:0/3229416250 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000fe80 con 0x7fd17c140a70 2020-11-20T12:03:22.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.952+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c13f8f0 con 0x7fd17c140a70 2020-11-20T12:03:22.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.953+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c13d000 con 0x7fd17c140a70 2020-11-20T12:03:22.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.953+0000 7fd15bfff700 1 -- 172.21.15.44:0/3229416250 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd17001b030 con 0x7fd17c140a70 2020-11-20T12:03:22.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.953+0000 7fd15bfff700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd154056d20 0x7fd1540591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:22.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.953+0000 7fd15bfff700 1 -- 172.21.15.44:0/3229416250 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd17006f9a0 con 0x7fd17c140a70 2020-11-20T12:03:22.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.953+0000 7fd17bfff700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd154056d20 0x7fd1540591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:22.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.954+0000 7fd17bfff700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd154056d20 0x7fd1540591e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd17c045400 tx=0x7fd16c019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:22.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:22.957+0000 7fd15bfff700 1 -- 172.21.15.44:0/3229416250 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd17c13d000 con 0x7fd17c140a70 2020-11-20T12:03:23.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.131+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd17c13d000 con 0x7fd17c140a70 2020-11-20T12:03:23.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.131+0000 7fd15bfff700 1 -- 172.21.15.44:0/3229416250 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd17c13d000 con 0x7fd17c140a70 2020-11-20T12:03:23.136 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 8 2020-11-20T12:03:23.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd154056d20 msgr2=0x7fd1540591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd154056d20 0x7fd1540591e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd17c045400 tx=0x7fd16c019000).stop 2020-11-20T12:03:23.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c140a70 msgr2=0x7fd17c0ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c140a70 0x7fd17c0ac680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd170000f90 tx=0x7fd170007800).stop 2020-11-20T12:03:23.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 shutdown_connections 2020-11-20T12:03:23.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd154056d20 0x7fd1540591e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd17c13cc20 0x7fd17c13f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd17c140690 0x7fd17c152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 --2- 172.21.15.44:0/3229416250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd17c140a70 0x7fd17c0ac680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 >> 172.21.15.44:0/3229416250 conn(0x7fd17c0624b0 msgr2=0x7fd17c135cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 shutdown_connections 2020-11-20T12:03:23.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.133+0000 7fd182706700 1 -- 172.21.15.44:0/3229416250 wait complete. 2020-11-20T12:03:23.148 DEBUG:tasks.mon_thrash:Yielding 2020-11-20T12:03:23.148 INFO:teuthology.run_tasks:Running task workunit... 2020-11-20T12:03:23.163 INFO:tasks.workunit:Pulling workunits from ref b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:03:23.164 INFO:tasks.workunit:Making a separate scratch dir for every client... 2020-11-20T12:03:23.165 INFO:tasks.workunit:timeout=3h 2020-11-20T12:03:23.165 INFO:tasks.workunit:cleanup=True 2020-11-20T12:03:23.166 INFO:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.0 2020-11-20T12:03:23.166 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:03:23.182 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.0 2020-11-20T12:03:23.182 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2020-11-20T12:03:23.183 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 2fh/47d Inode: 1 Links: 2 2020-11-20T12:03:23.183 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:03:23.183 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:fusefs_t:s0 2020-11-20T12:03:23.183 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2020-11-20T12:03:23.183 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2020-11-20 12:03:00.180576626 +0000 2020-11-20T12:03:23.184 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2020-11-20 12:03:19.100134644 +0000 2020-11-20T12:03:23.184 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2020-11-20T12:03:23.184 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2020-11-20T12:03:23.185 INFO:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2020-11-20T12:03:23.260 INFO:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:03:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.291+0000 7f48fd853700 1 Processor -- start 2020-11-20T12:03:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.291+0000 7f48fd853700 1 -- start start 2020-11-20T12:03:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48fd853700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f8140810 0x7f48f8142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48fd853700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f80a6260 con 0x7f48f813a990 2020-11-20T12:03:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48fd853700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f80a6490 con 0x7f48f8060330 2020-11-20T12:03:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48fd853700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80a52e0 con 0x7f48f8140810 2020-11-20T12:03:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f67fc700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f48f813a990 0x7f48f813ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40588/0 (socket says 172.21.15.44:40588) 2020-11-20T12:03:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f67fc700 1 -- 172.21.15.44:0/3531085806 learned_addr learned my addr 172.21.15.44:0/3531085806 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f77fe700 1 --2- 172.21.15.44:0/3531085806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f8140810 0x7f48f8142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f5ffb700 1 -- 172.21.15.44:0/3531085806 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371457916 0 0) 0x7f48f80a6260 con 0x7f48f813a990 2020-11-20T12:03:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f5ffb700 1 -- 172.21.15.44:0/3531085806 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e4003240 con 0x7f48f813a990 2020-11-20T12:03:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f5ffb700 1 -- 172.21.15.44:0/3531085806 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569650039 0 0) 0x7f48f80a6490 con 0x7f48f8060330 2020-11-20T12:03:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f5ffb700 1 -- 172.21.15.44:0/3531085806 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f80a6260 con 0x7f48f8060330 2020-11-20T12:03:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f77fe700 1 -- 172.21.15.44:0/3531085806 >> v1:172.21.15.79:6789/0 conn(0x7f48f8060330 legacy=0x7f48f80a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f77fe700 1 -- 172.21.15.44:0/3531085806 >> v1:172.21.15.44:6789/0 conn(0x7f48f813a990 legacy=0x7f48f813ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.292+0000 7f48f77fe700 1 -- 172.21.15.44:0/3531085806 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80a6730 con 0x7f48f8140810 2020-11-20T12:03:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48f77fe700 1 --2- 172.21.15.44:0/3531085806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f8140810 0x7f48f8142c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f48f0009cc0 tx=0x7f48f00091d0).ready entity=mon.2 client_cookie=170b491d6fad9451 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48f5ffb700 1 -- 172.21.15.44:0/3531085806 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000ba20 con 0x7f48f8140810 2020-11-20T12:03:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48f5ffb700 1 -- 172.21.15.44:0/3531085806 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f000bb80 con 0x7f48f8140810 2020-11-20T12:03:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48f5ffb700 1 -- 172.21.15.44:0/3531085806 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0011400 con 0x7f48f8140810 2020-11-20T12:03:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48fd853700 1 -- 172.21.15.44:0/3531085806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f8140810 msgr2=0x7f48f8142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48fd853700 1 --2- 172.21.15.44:0/3531085806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f8140810 0x7f48f8142c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f48f0009cc0 tx=0x7f48f00091d0).stop 2020-11-20T12:03:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48fd853700 1 -- 172.21.15.44:0/3531085806 shutdown_connections 2020-11-20T12:03:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48fd853700 1 --2- 172.21.15.44:0/3531085806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f8140810 0x7f48f8142c00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48fd853700 1 -- 172.21.15.44:0/3531085806 >> 172.21.15.44:0/3531085806 conn(0x7f48f8062690 msgr2=0x7f48f80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48fd853700 1 -- 172.21.15.44:0/3531085806 shutdown_connections 2020-11-20T12:03:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.293+0000 7f48fd853700 1 -- 172.21.15.44:0/3531085806 wait complete. 2020-11-20T12:03:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.295+0000 7f48fd853700 1 Processor -- start 2020-11-20T12:03:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.295+0000 7f48fd853700 1 -- start start 2020-11-20T12:03:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48fd853700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48f8060330 0x7f48f80ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48fd853700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f813a990 0x7f48f81365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48fd853700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f8140810 0x7f48f8136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48fd853700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8136fd0 con 0x7f48f8140810 2020-11-20T12:03:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48fd853700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f813c430 con 0x7f48f8060330 2020-11-20T12:03:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48fd853700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f813c6d0 con 0x7f48f813a990 2020-11-20T12:03:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f8140810 0x7f48f8136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f8140810 0x7f48f8136ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36848/0 (socket says 172.21.15.44:36848) 2020-11-20T12:03:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 -- 172.21.15.44:0/3880199653 learned_addr learned my addr 172.21.15.44:0/3880199653 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f6ffd700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48f8060330 0x7f48f80ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f67fc700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f813a990 0x7f48f81365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 -- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f813a990 msgr2=0x7f48f81365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f813a990 0x7f48f81365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 -- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48f8060330 msgr2=0x7f48f80ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48f8060330 0x7f48f80ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.296+0000 7f48f77fe700 1 -- 172.21.15.44:0/3880199653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f813c970 con 0x7f48f8140810 2020-11-20T12:03:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.297+0000 7f48f77fe700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f8140810 0x7f48f8136ae0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f48f0011ed0 tx=0x7f48f0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.297+0000 7f48deffd700 1 -- 172.21.15.44:0/3880199653 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0006ea0 con 0x7f48f8140810 2020-11-20T12:03:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.297+0000 7f48deffd700 1 -- 172.21.15.44:0/3880199653 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f001c5f0 con 0x7f48f8140810 2020-11-20T12:03:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.297+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f80a3560 con 0x7f48f8140810 2020-11-20T12:03:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.297+0000 7f48deffd700 1 -- 172.21.15.44:0/3880199653 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f001c970 con 0x7f48f8140810 2020-11-20T12:03:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.297+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f80ad6c0 con 0x7f48f8140810 2020-11-20T12:03:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.298+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f813cb00 con 0x7f48f8140810 2020-11-20T12:03:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.298+0000 7f48deffd700 1 -- 172.21.15.44:0/3880199653 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f48f0024030 con 0x7f48f8140810 2020-11-20T12:03:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.298+0000 7f48deffd700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f48d4056d20 0x7f48d40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.298+0000 7f48deffd700 1 -- 172.21.15.44:0/3880199653 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f48f0071970 con 0x7f48f8140810 2020-11-20T12:03:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.299+0000 7f48f6ffd700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f48d4056d20 0x7f48d40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.299+0000 7f48f6ffd700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f48d4056d20 0x7f48d40591e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f48ec000bc0 tx=0x7f48ec007a00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.302+0000 7f48deffd700 1 -- 172.21.15.44:0/3880199653 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f48f0046d10 con 0x7f48f8140810 2020-11-20T12:03:23.372 INFO:tasks.workunit.client.0.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2020-11-20T12:03:23.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.536+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48f8061650 con 0x7f48f8140810 2020-11-20T12:03:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.536+0000 7f48deffd700 1 -- 172.21.15.44:0/3880199653 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2131 (secure 0 0 0) 0x7f48f0022020 con 0x7f48f8140810 2020-11-20T12:03:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: 2020-11-20T12:03:23.541 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/4254760701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":4254760701},{"type":"v1","addr":"172.21.15.79:6835","nonce":4254760701}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:03:00.178663+0000","modified":"2020-11-20T12:03:03.184476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6833/2557888073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2557888073},{"type":"v1","addr":"172.21.15.44:6833","nonce":2557888073}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:03:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f48d4056d20 msgr2=0x7f48d40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f48d4056d20 0x7f48d40591e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f48ec000bc0 tx=0x7f48ec007a00).stop 2020-11-20T12:03:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f8140810 msgr2=0x7f48f8136ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f8140810 0x7f48f8136ae0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f48f0011ed0 tx=0x7f48f0016000).stop 2020-11-20T12:03:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 shutdown_connections 2020-11-20T12:03:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f48d4056d20 0x7f48d40591e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48f8140810 0x7f48f8136ae0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f48f8060330 0x7f48f80ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 --2- 172.21.15.44:0/3880199653 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f48f813a990 0x7f48f81365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 >> 172.21.15.44:0/3880199653 conn(0x7f48f8062690 msgr2=0x7f48f8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 shutdown_connections 2020-11-20T12:03:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.538+0000 7f48fd853700 1 -- 172.21.15.44:0/3880199653 wait complete. 2020-11-20T12:03:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 7 2020-11-20T12:03:23.553 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2020-11-20T12:03:23.553 INFO:tasks.mon_thrash.mon_thrasher:seed: 1605873801, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2020-11-20T12:03:23.554 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:03:23.554 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:03:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.714+0000 7f387c188700 1 Processor -- start 2020-11-20T12:03:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.714+0000 7f387c188700 1 -- start start 2020-11-20T12:03:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387c188700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3874060330 0x7f38740a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387c188700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f38740a4c30 0x7f387413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387c188700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f387413ae80 0x7f3874136f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387c188700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38740a52e0 con 0x7f3874060330 2020-11-20T12:03:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387c188700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38740a6260 con 0x7f387413ae80 2020-11-20T12:03:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387c188700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740a6490 con 0x7f38740a4c30 2020-11-20T12:03:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3874060330 0x7f38740a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3874060330 0x7f38740a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36854/0 (socket says 172.21.15.44:36854) 2020-11-20T12:03:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 -- 172.21.15.44:0/2872165517 learned_addr learned my addr 172.21.15.44:0/2872165517 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387a725700 1 --2- 172.21.15.44:0/2872165517 >> v2:172.21.15.79:3300/0 conn(0x7f387413ae80 0x7f3874136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879723700 1 --2- 172.21.15.44:0/2872165517 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f38740a4c30 0x7f387413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 -- 172.21.15.44:0/2872165517 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f38740a4c30 msgr2=0x7f387413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 --2- 172.21.15.44:0/2872165517 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f38740a4c30 0x7f387413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 -- 172.21.15.44:0/2872165517 >> v2:172.21.15.79:3300/0 conn(0x7f387413ae80 msgr2=0x7f3874136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 --2- 172.21.15.44:0/2872165517 >> v2:172.21.15.79:3300/0 conn(0x7f387413ae80 0x7f3874136f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879f24700 1 -- 172.21.15.44:0/2872165517 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740a6730 con 0x7f3874060330 2020-11-20T12:03:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f387a725700 1 --2- 172.21.15.44:0/2872165517 >> v2:172.21.15.79:3300/0 conn(0x7f387413ae80 0x7f3874136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.715+0000 7f3879723700 1 --2- 172.21.15.44:0/2872165517 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f38740a4c30 0x7f387413a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f3879f24700 1 --2- 172.21.15.44:0/2872165517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3874060330 0x7f38740a4740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f3874061220 tx=0x7f3864006eb0).ready entity=mon.0 client_cookie=834f92afad128777 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f3878f22700 1 -- 172.21.15.44:0/2872165517 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640098d0 con 0x7f3874060330 2020-11-20T12:03:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f3878f22700 1 -- 172.21.15.44:0/2872165517 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864009a30 con 0x7f3874060330 2020-11-20T12:03:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f3878f22700 1 -- 172.21.15.44:0/2872165517 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864009db0 con 0x7f3874060330 2020-11-20T12:03:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 -- 172.21.15.44:0/2872165517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3874060330 msgr2=0x7f38740a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 --2- 172.21.15.44:0/2872165517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3874060330 0x7f38740a4740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f3874061220 tx=0x7f3864006eb0).stop 2020-11-20T12:03:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 -- 172.21.15.44:0/2872165517 shutdown_connections 2020-11-20T12:03:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 --2- 172.21.15.44:0/2872165517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3874060330 0x7f38740a4740 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 --2- 172.21.15.44:0/2872165517 >> v2:172.21.15.79:3300/0 conn(0x7f387413ae80 0x7f3874136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 --2- 172.21.15.44:0/2872165517 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f38740a4c30 0x7f387413a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 -- 172.21.15.44:0/2872165517 >> 172.21.15.44:0/2872165517 conn(0x7f3874062690 msgr2=0x7f38740a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 -- 172.21.15.44:0/2872165517 shutdown_connections 2020-11-20T12:03:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.716+0000 7f387c188700 1 -- 172.21.15.44:0/2872165517 wait complete. 2020-11-20T12:03:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.717+0000 7f387c188700 1 Processor -- start 2020-11-20T12:03:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.717+0000 7f387c188700 1 -- start start 2020-11-20T12:03:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f387c188700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3874060330 0x7f3874137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f387c188700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740a4c30 0x7f3874131ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f387c188700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f387413ae80 0x7f38741323b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f387c188700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741340a0 con 0x7f38740a4c30 2020-11-20T12:03:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f387c188700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3874134340 con 0x7f3874060330 2020-11-20T12:03:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f387c188700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3874131a20 con 0x7f387413ae80 2020-11-20T12:03:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740a4c30 0x7f3874131ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740a4c30 0x7f3874131ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36862/0 (socket says 172.21.15.44:36862) 2020-11-20T12:03:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 -- 172.21.15.44:0/674730701 learned_addr learned my addr 172.21.15.44:0/674730701 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879f24700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3874060330 0x7f3874137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f387a725700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f387413ae80 0x7f38741323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 -- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f387413ae80 msgr2=0x7f38741323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f387413ae80 0x7f38741323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 -- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3874060330 msgr2=0x7f3874137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3874060330 0x7f3874137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 -- 172.21.15.44:0/674730701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874136d90 con 0x7f38740a4c30 2020-11-20T12:03:23.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.718+0000 7f3879723700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740a4c30 0x7f3874131ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f387000db60 tx=0x7f3870008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:23.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.719+0000 7f3861ffb700 1 -- 172.21.15.44:0/674730701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870009790 con 0x7f38740a4c30 2020-11-20T12:03:23.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.719+0000 7f3861ffb700 1 -- 172.21.15.44:0/674730701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38700098f0 con 0x7f38740a4c30 2020-11-20T12:03:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.719+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38741314d0 con 0x7f38740a4c30 2020-11-20T12:03:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.719+0000 7f3861ffb700 1 -- 172.21.15.44:0/674730701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870009c90 con 0x7f38740a4c30 2020-11-20T12:03:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.719+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3874133d60 con 0x7f38740a4c30 2020-11-20T12:03:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.720+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3874131be0 con 0x7f38740a4c30 2020-11-20T12:03:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.720+0000 7f3861ffb700 1 -- 172.21.15.44:0/674730701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3870013480 con 0x7f38740a4c30 2020-11-20T12:03:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.720+0000 7f3861ffb700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f384c056d20 0x7f384c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.720+0000 7f3861ffb700 1 -- 172.21.15.44:0/674730701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f387006ffa0 con 0x7f38740a4c30 2020-11-20T12:03:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.720+0000 7f3879f24700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f384c056d20 0x7f384c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.721+0000 7f3879f24700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f384c056d20 0x7f384c0591e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3864006ef0 tx=0x7f3864015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:23.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.724+0000 7f3861ffb700 1 -- 172.21.15.44:0/674730701 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3870045230 con 0x7f38740a4c30 2020-11-20T12:03:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.968+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3874061650 con 0x7f38740a4c30 2020-11-20T12:03:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.968+0000 7f3861ffb700 1 -- 172.21.15.44:0/674730701 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f3870048850 con 0x7f38740a4c30 2020-11-20T12:03:23.972 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":42,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:03:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f384c056d20 msgr2=0x7f384c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f384c056d20 0x7f384c0591e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3864006ef0 tx=0x7f3864015000).stop 2020-11-20T12:03:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740a4c30 msgr2=0x7f3874131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740a4c30 0x7f3874131ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f387000db60 tx=0x7f3870008040).stop 2020-11-20T12:03:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 shutdown_connections 2020-11-20T12:03:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f384c056d20 0x7f384c0591e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f387413ae80 0x7f38741323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3874060330 0x7f3874137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 --2- 172.21.15.44:0/674730701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f38740a4c30 0x7f3874131ec0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 >> 172.21.15.44:0/674730701 conn(0x7f3874062690 msgr2=0x7f38740a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 shutdown_connections 2020-11-20T12:03:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:23.970+0000 7f387c188700 1 -- 172.21.15.44:0/674730701 wait complete. 2020-11-20T12:03:23.985 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:03:23.985 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:03:23.985 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:03:24.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.128+0000 7faf1a5fa700 1 Processor -- start 2020-11-20T12:03:24.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.128+0000 7faf1a5fa700 1 -- start start 2020-11-20T12:03:24.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf1a5fa700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 0x7faf140a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf1a5fa700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faf140ab0e0 0x7faf1413ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf1a5fa700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf141385f0 con 0x7faf140a86d0 2020-11-20T12:03:24.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf1a5fa700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1413e250 con 0x7faf140ab0e0 2020-11-20T12:03:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf1a5fa700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1413e4f0 con 0x7faf140aad00 2020-11-20T12:03:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf18b97700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7faf140a86d0 0x7faf140abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40616/0 (socket says 172.21.15.44:40616) 2020-11-20T12:03:24.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf18b97700 1 -- 172.21.15.44:0/974757955 learned_addr learned my addr 172.21.15.44:0/974757955 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:24.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf13fff700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 0x7faf140a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf0bfff700 1 --2- 172.21.15.44:0/974757955 >> v2:172.21.15.79:3300/0 conn(0x7faf140ab0e0 0x7faf1413ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015301246 0 0) 0x7faf141385f0 con 0x7faf140a86d0 2020-11-20T12:03:24.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.129+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef4003240 con 0x7faf140a86d0 2020-11-20T12:03:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4075543152 0 0) 0x7faef4003240 con 0x7faf140a86d0 2020-11-20T12:03:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 msgr2=0x7faf140a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:24.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 0x7faf140a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 >> v2:172.21.15.79:3300/0 conn(0x7faf140ab0e0 msgr2=0x7faf1413ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 --2- 172.21.15.44:0/974757955 >> v2:172.21.15.79:3300/0 conn(0x7faf140ab0e0 0x7faf1413ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140ac340 con 0x7faf140a86d0 2020-11-20T12:03:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7faf0c001860 con 0x7faf140a86d0 2020-11-20T12:03:24.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 >> v1:172.21.15.44:6789/0 conn(0x7faf140a86d0 legacy=0x7faf140abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faef4003730 0x7faef4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 0x7faef40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef40049d0 0x7faef4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1413e250 con 0x7faef40049d0 2020-11-20T12:03:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1413e4f0 con 0x7faef4003730 2020-11-20T12:03:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef40041d0 con 0x7faf140aad00 2020-11-20T12:03:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf18b97700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef40049d0 0x7faef4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf13fff700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faef4003730 0x7faef4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf0bfff700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 0x7faef40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf18b97700 1 -- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 msgr2=0x7faef40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf18b97700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140aad00 0x7faef40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf18b97700 1 -- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faef4003730 msgr2=0x7faef4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf18b97700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faef4003730 0x7faef4003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf18b97700 1 -- 172.21.15.44:0/974757955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4003240 con 0x7faef40049d0 2020-11-20T12:03:24.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf18b97700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef40049d0 0x7faef4004e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7faf0c00c840 tx=0x7faf0c007b80).ready entity=mon.0 client_cookie=21711bd58bf0f1d1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c005b80 con 0x7faef40049d0 2020-11-20T12:03:24.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c005ce0 con 0x7faef40049d0 2020-11-20T12:03:24.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.130+0000 7faf137fe700 1 -- 172.21.15.44:0/974757955 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007300 con 0x7faef40049d0 2020-11-20T12:03:24.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.131+0000 7faf1a5fa700 1 -- 172.21.15.44:0/974757955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef40049d0 msgr2=0x7faef4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.131+0000 7faf1a5fa700 1 --2- 172.21.15.44:0/974757955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faef40049d0 0x7faef4004e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7faf0c00c840 tx=0x7faf0c007b80).stop 2020-11-20T12:03:24.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.131+0000 7faf18b97700 1 -- 172.21.15.44:0/974757955 reap_dead start 2020-11-20T12:03:24.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.131+0000 7faf1a5fa700 1 -- 172.21.15.44:0/974757955 shutdown_connections 2020-11-20T12:03:24.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.131+0000 7faf1a5fa700 1 -- 172.21.15.44:0/974757955 >> 172.21.15.44:0/974757955 conn(0x7faf14060170 msgr2=0x7faf14060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:24.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.131+0000 7faf1a5fa700 1 -- 172.21.15.44:0/974757955 shutdown_connections 2020-11-20T12:03:24.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.131+0000 7faf1a5fa700 1 -- 172.21.15.44:0/974757955 wait complete. 2020-11-20T12:03:24.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.132+0000 7faf1a5fa700 1 Processor -- start 2020-11-20T12:03:24.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.132+0000 7faf1a5fa700 1 -- start start 2020-11-20T12:03:24.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf1a5fa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf14140ea0 0x7faf1414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf1a5fa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf140a7dd0 0x7faf140a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf1a5fa700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140a5590 0x7faf140a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf1a5fa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140a72e0 con 0x7faf140a7dd0 2020-11-20T12:03:24.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf1a5fa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140a5f00 con 0x7faf14140ea0 2020-11-20T12:03:24.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf1a5fa700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140a64e0 con 0x7faf140a5590 2020-11-20T12:03:24.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf140a7dd0 0x7faf140a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf140a7dd0 0x7faf140a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36880/0 (socket says 172.21.15.44:36880) 2020-11-20T12:03:24.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 -- 172.21.15.44:0/2825802584 learned_addr learned my addr 172.21.15.44:0/2825802584 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:24.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf13fff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf14140ea0 0x7faf1414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf18b97700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140a5590 0x7faf140a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140a5590 msgr2=0x7faf140a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7faf140a5590 0x7faf140a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf14140ea0 msgr2=0x7faf1414d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faf14140ea0 0x7faf1414d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 -- 172.21.15.44:0/2825802584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140a6780 con 0x7faf140a7dd0 2020-11-20T12:03:24.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf0bfff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf140a7dd0 0x7faf140a50a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7faefc011a90 tx=0x7faefc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc018070 con 0x7faf140a7dd0 2020-11-20T12:03:24.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc00f910 con 0x7faf140a7dd0 2020-11-20T12:03:24.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf140a6a20 con 0x7faf140a7dd0 2020-11-20T12:03:24.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.133+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00fc90 con 0x7faf140a7dd0 2020-11-20T12:03:24.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.134+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf140a76d0 con 0x7faf140a7dd0 2020-11-20T12:03:24.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.134+0000 7faf1a5fa700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf141370c0 0x7faf141374a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.134+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1405c5f0 con 0x7faf141370c0 2020-11-20T12:03:24.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.134+0000 7faf13fff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf141370c0 0x7faf141374a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf13fff700 1 -- 172.21.15.44:0/2825802584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf140a6fe0 con 0x7faf141370c0 2020-11-20T12:03:24.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7faefc014440 con 0x7faf140a7dd0 2020-11-20T12:03:24.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf13fff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf141370c0 0x7faf141374a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7faf04006860 tx=0x7faf04006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7faef0056d20 0x7faef00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7faefc0706e0 con 0x7faf140a7dd0 2020-11-20T12:03:24.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04012a90 con 0x7faf141370c0 2020-11-20T12:03:24.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7faf04012bf0 con 0x7faf141370c0 2020-11-20T12:03:24.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf18b97700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7faef0056d20 0x7faef00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf141370c0 msgr2=0x7faf141374a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf141370c0 0x7faf141374a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7faf04006860 tx=0x7faf04006df0).stop 2020-11-20T12:03:24.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf141370c0 msgr2=0x7faf141374a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:24.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.135+0000 7faf10ff9700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf141370c0 0x7faf141374a0 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0x7faf04006860 tx=0x7faf04006df0).stop 2020-11-20T12:03:24.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.136+0000 7faf18b97700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7faef0056d20 0x7faef00591e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7faf14061040 tx=0x7faf0c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.143+0000 7faf1a5fa700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14137990 0x7faf140ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.143+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140a6bb0 con 0x7faf14137990 2020-11-20T12:03:24.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.143+0000 7faf13fff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14137990 0x7faf140ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.143+0000 7faf13fff700 1 -- 172.21.15.44:0/2825802584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faf14069680 con 0x7faf14137990 2020-11-20T12:03:24.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf13fff700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14137990 0x7faf140ad1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7faf0400bfd0 tx=0x7faf0401c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04015630 con 0x7faf14137990 2020-11-20T12:03:24.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7faf04012880 con 0x7faf14137990 2020-11-20T12:03:24.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14137990 msgr2=0x7faf140ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf10ff9700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14137990 0x7faf140ad1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7faf0400bfd0 tx=0x7faf0401c000).stop 2020-11-20T12:03:24.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf10ff9700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14137990 msgr2=0x7faf140ad1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:24.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf10ff9700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14137990 0x7faf140ad1d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7faef0056d20 msgr2=0x7faef00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7faef0056d20 0x7faef00591e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7faf14061040 tx=0x7faf0c015000).stop 2020-11-20T12:03:24.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf140a7dd0 msgr2=0x7faf140a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf140a7dd0 0x7faf140a50a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7faefc011a90 tx=0x7faefc006df0).stop 2020-11-20T12:03:24.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf18b97700 1 -- 172.21.15.44:0/2825802584 reap_dead start 2020-11-20T12:03:24.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 shutdown_connections 2020-11-20T12:03:24.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 --2- 172.21.15.44:0/2825802584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf140a7dd0 0x7faf140a50a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 >> 172.21.15.44:0/2825802584 conn(0x7faf14060170 msgr2=0x7faf14060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:24.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 shutdown_connections 2020-11-20T12:03:24.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.144+0000 7faf1a5fa700 1 -- 172.21.15.44:0/2825802584 wait complete. 2020-11-20T12:03:24.166 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:03:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:03:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:03:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:03:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2020-11-20T12:03:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:03:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:03:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:03:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:03:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 43, 2020-11-20T12:03:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:03:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:03:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:03:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:03:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:03:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:03:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:03:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:03:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:03:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:03:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:03:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:03:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:03:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:03:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:03:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:03:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:03:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:03:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:03:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:03:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:03:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:03:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:03:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:03:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:03:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:03:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:03:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:03:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:03:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:03:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:03:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:03:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:03:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:03:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:03:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:03:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:03:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:03:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:03:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:03:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:03:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:03:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:03:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:03:24.199 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:03:24.200 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:03:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.339+0000 7f4dd4a1d700 1 Processor -- start 2020-11-20T12:03:24.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.339+0000 7f4dd4a1d700 1 -- start start 2020-11-20T12:03:24.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dd4a1d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 0x7f4dd00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dd4a1d700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00a9770 con 0x7f4dd013ac60 2020-11-20T12:03:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dd4a1d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd00a6da0 con 0x7f4dd00aa400 2020-11-20T12:03:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dd4a1d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00a56c0 con 0x7f4dd00aa020 2020-11-20T12:03:24.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dced9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f4dd013ac60 0x7f4dd00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40638/0 (socket says 172.21.15.44:40638) 2020-11-20T12:03:24.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dced9d700 1 -- 172.21.15.44:0/1156515732 learned_addr learned my addr 172.21.15.44:0/1156515732 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:24.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 0x7f4dd00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043584169 0 0) 0x7f4dc8001450 con 0x7f4dd013ac60 2020-11-20T12:03:24.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db4003240 con 0x7f4dd013ac60 2020-11-20T12:03:24.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 729543464 0 0) 0x7f4db4003240 con 0x7f4dd013ac60 2020-11-20T12:03:24.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 msgr2=0x7f4dd00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:24.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 0x7f4dd00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 >> v1:172.21.15.79:6789/0 conn(0x7f4dd00aa400 legacy=0x7f4dd00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00a7040 con 0x7f4dd013ac60 2020-11-20T12:03:24.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f4dc80019f0 con 0x7f4dd013ac60 2020-11-20T12:03:24.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 >> v1:172.21.15.44:6789/0 conn(0x7f4dd013ac60 legacy=0x7f4dd00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db4003760 0x7f4db4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 0x7f4db40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.340+0000 7f4dcd59a700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4db40049d0 0x7f4db4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00a56c0 con 0x7f4db40049d0 2020-11-20T12:03:24.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4004170 con 0x7f4db4003760 2020-11-20T12:03:24.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4004330 con 0x7f4dd00aa020 2020-11-20T12:03:24.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db4003760 0x7f4db4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dced9d700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4db40049d0 0x7f4db4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dcdd9b700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 0x7f4db40044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dced9d700 1 -- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 msgr2=0x7f4db40044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:24.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dced9d700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00aa020 0x7f4db40044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dced9d700 1 -- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db4003760 msgr2=0x7f4db4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:24.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dced9d700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db4003760 0x7f4db4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dced9d700 1 -- 172.21.15.44:0/1156515732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db4003240 con 0x7f4db40049d0 2020-11-20T12:03:24.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db4003760 0x7f4db4003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:24.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dced9d700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4db40049d0 0x7f4db4004e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f4dc800c840 tx=0x7f4dd00a5f30).ready entity=mon.0 client_cookie=be7f58233aec755b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8005b80 con 0x7f4db40049d0 2020-11-20T12:03:24.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8005ce0 con 0x7f4db40049d0 2020-11-20T12:03:24.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.341+0000 7f4dcd59a700 1 -- 172.21.15.44:0/1156515732 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8007360 con 0x7f4db40049d0 2020-11-20T12:03:24.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.342+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1156515732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4db40049d0 msgr2=0x7f4db4004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.342+0000 7f4dd4a1d700 1 --2- 172.21.15.44:0/1156515732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4db40049d0 0x7f4db4004e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f4dc800c840 tx=0x7f4dd00a5f30).stop 2020-11-20T12:03:24.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.342+0000 7f4dced9d700 1 -- 172.21.15.44:0/1156515732 reap_dead start 2020-11-20T12:03:24.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.342+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1156515732 shutdown_connections 2020-11-20T12:03:24.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.342+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1156515732 >> 172.21.15.44:0/1156515732 conn(0x7f4dd0062690 msgr2=0x7f4dd00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:24.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.342+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1156515732 shutdown_connections 2020-11-20T12:03:24.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.342+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1156515732 wait complete. 2020-11-20T12:03:24.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 Processor -- start 2020-11-20T12:03:24.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 -- start start 2020-11-20T12:03:24.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db40049d0 0x7f4dd00ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0060170 0x7f4dd00acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00a3070 0x7f4dd00ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01318f0 con 0x7f4dd0060170 2020-11-20T12:03:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00ad920 con 0x7f4db40049d0 2020-11-20T12:03:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dd4a1d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd0060b50 con 0x7f4dd00a3070 2020-11-20T12:03:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dcdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0060170 0x7f4dd00acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dcdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0060170 0x7f4dd00acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36904/0 (socket says 172.21.15.44:36904) 2020-11-20T12:03:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dcdd9b700 1 -- 172.21.15.44:0/1697100624 learned_addr learned my addr 172.21.15.44:0/1697100624 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dced9d700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00a3070 0x7f4dd00ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.343+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db40049d0 0x7f4dd00ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dcdd9b700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00a3070 msgr2=0x7f4dd00ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dcdd9b700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4dd00a3070 0x7f4dd00ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dcdd9b700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db40049d0 msgr2=0x7f4dd00ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dcdd9b700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4db40049d0 0x7f4dd00ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dcdd9b700 1 -- 172.21.15.44:0/1697100624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd0060610 con 0x7f4dd0060170 2020-11-20T12:03:24.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dcdd9b700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0060170 0x7f4dd00acf40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f4dc4002cb0 tx=0x7f4dc4007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc400f960 con 0x7f4dd0060170 2020-11-20T12:03:24.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc400fac0 con 0x7f4dd0060170 2020-11-20T12:03:24.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc400fe40 con 0x7f4dd0060170 2020-11-20T12:03:24.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd00608b0 con 0x7f4dd0060170 2020-11-20T12:03:24.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.344+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd0130a80 con 0x7f4dd0060170 2020-11-20T12:03:24.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.345+0000 7f4dd4a1d700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd0130d30 0x7f4dd0131110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.345+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00aae50 con 0x7f4dd0130d30 2020-11-20T12:03:24.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.345+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd0130d30 0x7f4dd0131110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.345+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4dc401b030 con 0x7f4dd0060170 2020-11-20T12:03:24.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.345+0000 7f4dce59c700 1 -- 172.21.15.44:0/1697100624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4dd0061650 con 0x7f4dd0130d30 2020-11-20T12:03:24.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.345+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd0130d30 0x7f4dd0131110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4dc0014f30 tx=0x7f4dc0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.345+0000 7f4dbe7fc700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4da4056d50 0x7f4da4059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4dc406fd80 con 0x7f4dd0060170 2020-11-20T12:03:24.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dced9d700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4da4056d50 0x7f4da4059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0012d80 con 0x7f4dd0130d30 2020-11-20T12:03:24.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4dc001e470 con 0x7f4dd0130d30 2020-11-20T12:03:24.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd0130d30 msgr2=0x7f4dd0131110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dbe7fc700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd0130d30 0x7f4dd0131110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4dc0014f30 tx=0x7f4dc0006df0).stop 2020-11-20T12:03:24.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd0130d30 msgr2=0x7f4dd0131110 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:24.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dbe7fc700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd0130d30 0x7f4dd0131110 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0x7f4dc0014f30 tx=0x7f4dc0006df0).stop 2020-11-20T12:03:24.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.346+0000 7f4dced9d700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4da4056d50 0x7f4da4059210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f4dd013b040 tx=0x7f4dc8014000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.354+0000 7f4dd4a1d700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd014a360 0x7f4dd014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.354+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00446d0 con 0x7f4dd014a360 2020-11-20T12:03:24.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.354+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd014a360 0x7f4dd014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.354+0000 7f4dce59c700 1 -- 172.21.15.44:0/1697100624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4dd00613b0 con 0x7f4dd014a360 2020-11-20T12:03:24.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dce59c700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd014a360 0x7f4dd014a770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4dc001b000 tx=0x7f4dc001c000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0016920 con 0x7f4dd014a360 2020-11-20T12:03:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4605 (secure 0 0 0) 0x7f4dc0012b70 con 0x7f4dd014a360 2020-11-20T12:03:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd014a360 msgr2=0x7f4dd014a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dbe7fc700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd014a360 0x7f4dd014a770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4dc001b000 tx=0x7f4dc001c000).stop 2020-11-20T12:03:24.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dbe7fc700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd014a360 msgr2=0x7f4dd014a770 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dbe7fc700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4dd014a360 0x7f4dd014a770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4da4056d50 msgr2=0x7f4da4059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4da4056d50 0x7f4da4059210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f4dd013b040 tx=0x7f4dc8014000).stop 2020-11-20T12:03:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0060170 msgr2=0x7f4dd00acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 --2- 172.21.15.44:0/1697100624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd0060170 0x7f4dd00acf40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f4dc4002cb0 tx=0x7f4dc4007b90).stop 2020-11-20T12:03:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dced9d700 1 -- 172.21.15.44:0/1697100624 reap_dead start 2020-11-20T12:03:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dced9d700 1 -- 172.21.15.44:0/1697100624 reap_dead start 2020-11-20T12:03:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 shutdown_connections 2020-11-20T12:03:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 >> 172.21.15.44:0/1697100624 conn(0x7f4dd0062690 msgr2=0x7f4dd00a8030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 shutdown_connections 2020-11-20T12:03:24.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.355+0000 7f4dd4a1d700 1 -- 172.21.15.44:0/1697100624 wait complete. 2020-11-20T12:03:24.383 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:03:24.383 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:03:24.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:03:24.384 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:03:24.384 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2020-11-20T12:03:24.384 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:03:24.384 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:03:24.384 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:03:24.385 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:03:24.385 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.385 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 43, 2020-11-20T12:03:24.385 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:24.385 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:03:24.386 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:03:24.386 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.386 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.386 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.386 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.387 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.387 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.387 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.387 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.387 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.388 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:03:24.388 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:03:24.388 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.388 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.388 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.389 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.389 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.389 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.389 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.389 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.390 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.390 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:03:24.390 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:03:24.390 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:03:24.391 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:03:24.391 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:03:24.391 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:03:24.391 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:24.392 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:24.392 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:03:24.392 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:03:24.392 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:03:24.392 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:03:24.393 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:03:24.393 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:24.393 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:03:24.393 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.394 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.394 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.394 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.394 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.394 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.395 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.395 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.395 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.395 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:03:24.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.396 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:03:24.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.397 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:03:24.397 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:03:24.398 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.398 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.399 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.400 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:03:24.400 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.402 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:03:24.402 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.403 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.403 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.404 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:03:24.404 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:03:24.405 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.405 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.406 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.407 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:03:24.407 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:03:24.408 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.408 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.409 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:03:24.410 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.411 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:03:24.412 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.412 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.412 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.413 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:03:24.413 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:03:24.414 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.414 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.415 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.416 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:03:24.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:03:24.417 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.417 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.418 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:03:24.418 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.420 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:03:24.420 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.420 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.420 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.421 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:03:24.421 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:03:24.421 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.421 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.421 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.422 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.422 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.422 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:03:24.422 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:03:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.424 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.424 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:03:24.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.424 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.424 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.425 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:24.425 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.425 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.425 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:03:24.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.426 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.426 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.426 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:03:24.426 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.426 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.427 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:03:24.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.427 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.427 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.427 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:03:24.428 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.428 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.428 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:03:24.428 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:03:24.429 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:03:24.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.563+0000 7fb458037700 1 Processor -- start 2020-11-20T12:03:24.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.563+0000 7fb458037700 1 -- start start 2020-11-20T12:03:24.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb458037700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb4500aa400 0x7fb4500a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb458037700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4500a9770 con 0x7fb45013ac60 2020-11-20T12:03:24.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb458037700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4500a6da0 con 0x7fb4500aa020 2020-11-20T12:03:24.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb458037700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4500a56c0 con 0x7fb4500aa400 2020-11-20T12:03:24.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb4555d2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb4500aa400 0x7fb4500a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb4565d4700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb45013ac60 0x7fb4500a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40666/0 (socket says 172.21.15.44:40666) 2020-11-20T12:03:24.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb4555d2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb4500aa400 0x7fb4500a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36916/0 (socket says 172.21.15.44:36916) 2020-11-20T12:03:24.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb4565d4700 1 -- 172.21.15.44:0/3626282735 learned_addr learned my addr 172.21.15.44:0/3626282735 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:24.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb4555d2700 1 -- 172.21.15.44:0/3626282735 >> v1:172.21.15.79:6789/0 conn(0x7fb4500aa020 legacy=0x7fb4500a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb4555d2700 1 -- 172.21.15.44:0/3626282735 >> v1:172.21.15.44:6789/0 conn(0x7fb45013ac60 legacy=0x7fb4500a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb4555d2700 1 -- 172.21.15.44:0/3626282735 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4500a7040 con 0x7fb4500aa400 2020-11-20T12:03:24.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.564+0000 7fb454dd1700 1 -- 172.21.15.44:0/3626282735 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007084873 0 0) 0x7fb44c0010d0 con 0x7fb45013ac60 2020-11-20T12:03:24.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb4555d2700 1 --2- 172.21.15.44:0/3626282735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa400 0x7fb4500a9280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fb44000b9a0 tx=0x7fb440006df0).ready entity=mon.0 client_cookie=7a43b824fecfef5a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb454dd1700 1 -- 172.21.15.44:0/3626282735 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400097b0 con 0x7fb4500aa400 2020-11-20T12:03:24.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb454dd1700 1 -- 172.21.15.44:0/3626282735 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb440009910 con 0x7fb4500aa400 2020-11-20T12:03:24.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb454dd1700 1 -- 172.21.15.44:0/3626282735 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440009c90 con 0x7fb4500aa400 2020-11-20T12:03:24.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb458037700 1 -- 172.21.15.44:0/3626282735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa400 msgr2=0x7fb4500a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb458037700 1 --2- 172.21.15.44:0/3626282735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa400 0x7fb4500a9280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fb44000b9a0 tx=0x7fb440006df0).stop 2020-11-20T12:03:24.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb458037700 1 -- 172.21.15.44:0/3626282735 shutdown_connections 2020-11-20T12:03:24.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb458037700 1 --2- 172.21.15.44:0/3626282735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa400 0x7fb4500a9280 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb458037700 1 -- 172.21.15.44:0/3626282735 >> 172.21.15.44:0/3626282735 conn(0x7fb450062690 msgr2=0x7fb4500a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:24.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb458037700 1 -- 172.21.15.44:0/3626282735 shutdown_connections 2020-11-20T12:03:24.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.565+0000 7fb458037700 1 -- 172.21.15.44:0/3626282735 wait complete. 2020-11-20T12:03:24.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.566+0000 7fb458037700 1 Processor -- start 2020-11-20T12:03:24.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.566+0000 7fb458037700 1 -- start start 2020-11-20T12:03:24.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb458037700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa020 0x7fb4501166d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb458037700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4500aa400 0x7fb4500acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb458037700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45013ac60 0x7fb45014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb458037700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500adce0 con 0x7fb4500aa020 2020-11-20T12:03:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb458037700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb450116d70 con 0x7fb4500aa400 2020-11-20T12:03:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb458037700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb450114c20 con 0x7fb45013ac60 2020-11-20T12:03:24.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa020 0x7fb4501166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa020 0x7fb4501166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36920/0 (socket says 172.21.15.44:36920) 2020-11-20T12:03:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 -- 172.21.15.44:0/558938764 learned_addr learned my addr 172.21.15.44:0/558938764 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb4555d2700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4500aa400 0x7fb4500acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb4565d4700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45013ac60 0x7fb45014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45013ac60 msgr2=0x7fb45014dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45013ac60 0x7fb45014dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4500aa400 msgr2=0x7fb4500acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4500aa400 0x7fb4500acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb455dd3700 1 -- 172.21.15.44:0/558938764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45014e120 con 0x7fb4500aa020 2020-11-20T12:03:24.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.567+0000 7fb4565d4700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45013ac60 0x7fb45014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.568+0000 7fb455dd3700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa020 0x7fb4501166d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fb44800ca60 tx=0x7fb448001460).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.568+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448006870 con 0x7fb4500aa020 2020-11-20T12:03:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.568+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4480069d0 con 0x7fb4500aa020 2020-11-20T12:03:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.568+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb450060dd0 con 0x7fb4500aa020 2020-11-20T12:03:24.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.568+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448006d50 con 0x7fb4500aa020 2020-11-20T12:03:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.568+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb450115280 con 0x7fb4500aa020 2020-11-20T12:03:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb458037700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb450134300 0x7fb4501346e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb450114aa0 con 0x7fb450134300 2020-11-20T12:03:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb4555d2700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb450134300 0x7fb4501346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb448018020 con 0x7fb4500aa020 2020-11-20T12:03:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb4555d2700 1 -- 172.21.15.44:0/558938764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb450116bc0 con 0x7fb450134300 2020-11-20T12:03:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb43dffb700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb428056d20 0x7fb4280591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb44806fca0 con 0x7fb4500aa020 2020-11-20T12:03:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb4555d2700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb450134300 0x7fb4501346e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb44000b9a0 tx=0x7fb440015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.569+0000 7fb4565d4700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb428056d20 0x7fb4280591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.570+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44001f440 con 0x7fb450134300 2020-11-20T12:03:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.570+0000 7fb4565d4700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb428056d20 0x7fb4280591e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fb4500ad980 tx=0x7fb44c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.570+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb44001e420 con 0x7fb450134300 2020-11-20T12:03:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.570+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb450134300 msgr2=0x7fb4501346e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.570+0000 7fb43dffb700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb450134300 0x7fb4501346e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb44000b9a0 tx=0x7fb440015000).stop 2020-11-20T12:03:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.570+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb450134300 msgr2=0x7fb4501346e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.570+0000 7fb43dffb700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb450134300 0x7fb4501346e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.578+0000 7fb458037700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45014a680 0x7fb45014aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.578+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb450116f30 con 0x7fb45014a680 2020-11-20T12:03:24.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.578+0000 7fb4555d2700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45014a680 0x7fb45014aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.578+0000 7fb4555d2700 1 -- 172.21.15.44:0/558938764 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb4500a8ae0 con 0x7fb45014a680 2020-11-20T12:03:24.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.578+0000 7fb4555d2700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45014a680 0x7fb45014aa60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb44000bf10 tx=0x7fb440021070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440020e20 con 0x7fb45014a680 2020-11-20T12:03:24.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4605 (secure 0 0 0) 0x7fb440011d70 con 0x7fb45014a680 2020-11-20T12:03:24.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45014a680 msgr2=0x7fb45014aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb43dffb700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45014a680 0x7fb45014aa60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb44000bf10 tx=0x7fb440021070).stop 2020-11-20T12:03:24.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb43dffb700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45014a680 msgr2=0x7fb45014aa60 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:24.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb43dffb700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb45014a680 0x7fb45014aa60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb428056d20 msgr2=0x7fb4280591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb428056d20 0x7fb4280591e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fb4500ad980 tx=0x7fb44c009040).stop 2020-11-20T12:03:24.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa020 msgr2=0x7fb4501166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 --2- 172.21.15.44:0/558938764 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb4500aa020 0x7fb4501166d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fb44800ca60 tx=0x7fb448001460).stop 2020-11-20T12:03:24.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb4565d4700 1 -- 172.21.15.44:0/558938764 reap_dead start 2020-11-20T12:03:24.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb4565d4700 1 -- 172.21.15.44:0/558938764 reap_dead start 2020-11-20T12:03:24.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 shutdown_connections 2020-11-20T12:03:24.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 >> 172.21.15.44:0/558938764 conn(0x7fb450062690 msgr2=0x7fb4501395b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:24.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 shutdown_connections 2020-11-20T12:03:24.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.579+0000 7fb458037700 1 -- 172.21.15.44:0/558938764 wait complete. 2020-11-20T12:03:24.591 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:03:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:03:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:03:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:03:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 4, 2020-11-20T12:03:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:03:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:03:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:03:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:03:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 43, 2020-11-20T12:03:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:03:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:03:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:03:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:03:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:03:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:03:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:03:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:03:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:03:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:03:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:03:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:03:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:03:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:03:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:03:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:03:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:03:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:03:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:03:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:03:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:03:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:03:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:03:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:03:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:03:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:03:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:03:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:03:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:03:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:03:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:03:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:03:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:03:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:03:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:03:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:03:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:03:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:03:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:03:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:03:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:03:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:03:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:03:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:03:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:03:24.624 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:03:24.626 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2020-11-20T12:03:24.626 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:03:24.626 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2020-11-20T12:03:24.626 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2020-11-20T12:03:24.627 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2020-11-20T12:03:24.627 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:03:24.640 INFO:tasks.ceph.mon.c:Stopped 2020-11-20T12:03:24.641 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:03:24.641 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:03:24.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.773+0000 7f1abd42c700 1 Processor -- start 2020-11-20T12:03:24.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.773+0000 7f1abd42c700 1 -- start start 2020-11-20T12:03:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1abd42c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8066de0 0x7f1ab80aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1abd42c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1ab8137bb0 0x7f1ab8138060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1abd42c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab8067470 con 0x7f1ab80a1a40 2020-11-20T12:03:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1abd42c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab80a9b30 con 0x7f1ab8137bb0 2020-11-20T12:03:24.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1abd42c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80ab240 con 0x7f1ab8066de0 2020-11-20T12:03:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1ab67fc700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8066de0 msgr2=0x7f1ab80aad50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1ab77fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1ab8137bb0 0x7f1ab8138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8066de0 0x7f1ab80aad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:24.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1ab77fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1ab8137bb0 0x7f1ab8138060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36948/0 (socket says 172.21.15.44:36948) 2020-11-20T12:03:24.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1ab77fe700 1 -- 172.21.15.44:0/1417050252 learned_addr learned my addr 172.21.15.44:0/1417050252 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:24.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1ab77fe700 1 -- 172.21.15.44:0/1417050252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8066de0 msgr2=0x7f1ab80aad50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:03:24.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.774+0000 7f1ab77fe700 1 --2- 172.21.15.44:0/1417050252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8066de0 0x7f1ab80aad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:24.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1ab77fe700 1 -- 172.21.15.44:0/1417050252 >> v1:172.21.15.79:6789/0 conn(0x7f1ab80a1a40 legacy=0x7f1ab80666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1ab77fe700 1 -- 172.21.15.44:0/1417050252 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8067710 con 0x7f1ab8137bb0 2020-11-20T12:03:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1ab77fe700 1 --2- 172.21.15.44:0/1417050252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab8137bb0 0x7f1ab8138060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f1ab0009d50 tx=0x7f1ab0009290).ready entity=mon.0 client_cookie=6e30159274f3771 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1ab5ffb700 1 -- 172.21.15.44:0/1417050252 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000bc80 con 0x7f1ab8137bb0 2020-11-20T12:03:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1ab5ffb700 1 -- 172.21.15.44:0/1417050252 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab000bde0 con 0x7f1ab8137bb0 2020-11-20T12:03:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1ab5ffb700 1 -- 172.21.15.44:0/1417050252 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0011610 con 0x7f1ab8137bb0 2020-11-20T12:03:24.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 -- 172.21.15.44:0/1417050252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab8137bb0 msgr2=0x7f1ab8138060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1417050252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab8137bb0 0x7f1ab8138060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f1ab0009d50 tx=0x7f1ab0009290).stop 2020-11-20T12:03:24.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 -- 172.21.15.44:0/1417050252 shutdown_connections 2020-11-20T12:03:24.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1417050252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab8137bb0 0x7f1ab8138060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1417050252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8066de0 0x7f1ab80aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:24.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 -- 172.21.15.44:0/1417050252 >> 172.21.15.44:0/1417050252 conn(0x7f1ab80af030 msgr2=0x7f1ab80af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 -- 172.21.15.44:0/1417050252 shutdown_connections 2020-11-20T12:03:24.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.775+0000 7f1abd42c700 1 -- 172.21.15.44:0/1417050252 wait complete. 2020-11-20T12:03:24.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.776+0000 7f1abd42c700 1 Processor -- start 2020-11-20T12:03:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.776+0000 7f1abd42c700 1 -- start start 2020-11-20T12:03:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1abd42c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab8066de0 0x7f1ab813f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1abd42c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab80a1a40 0x7f1ab8062480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1abd42c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8137bb0 0x7f1ab8131180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1abd42c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8139090 con 0x7f1ab80a1a40 2020-11-20T12:03:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1abd42c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80aeb60 con 0x7f1ab8066de0 2020-11-20T12:03:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1abd42c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8139280 con 0x7f1ab8137bb0 2020-11-20T12:03:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab77fe700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8137bb0 msgr2=0x7f1ab8131180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab80a1a40 0x7f1ab8062480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab77fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8137bb0 0x7f1ab8131180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:24.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab80a1a40 0x7f1ab8062480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36952/0 (socket says 172.21.15.44:36952) 2020-11-20T12:03:24.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 -- 172.21.15.44:0/1493319543 learned_addr learned my addr 172.21.15.44:0/1493319543 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:24.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab6ffd700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab8066de0 0x7f1ab813f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 -- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8137bb0 msgr2=0x7f1ab8131180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:03:24.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8137bb0 0x7f1ab8131180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:24.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 -- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab8066de0 msgr2=0x7f1ab813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:24.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab8066de0 0x7f1ab813f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:24.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.777+0000 7f1ab67fc700 1 -- 172.21.15.44:0/1493319543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8062970 con 0x7f1ab80a1a40 2020-11-20T12:03:24.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.778+0000 7f1ab67fc700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab80a1a40 0x7f1ab8062480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f1aa8009710 tx=0x7f1aa8006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.778+0000 7f1a9effd700 1 -- 172.21.15.44:0/1493319543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8006170 con 0x7f1ab80a1a40 2020-11-20T12:03:24.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.778+0000 7f1a9effd700 1 -- 172.21.15.44:0/1493319543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa8006310 con 0x7f1ab80a1a40 2020-11-20T12:03:24.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.778+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab8061f90 con 0x7f1ab80a1a40 2020-11-20T12:03:24.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.778+0000 7f1a9effd700 1 -- 172.21.15.44:0/1493319543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80038e0 con 0x7f1ab80a1a40 2020-11-20T12:03:24.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.778+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab8139720 con 0x7f1ab80a1a40 2020-11-20T12:03:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.779+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab81380e0 con 0x7f1ab80a1a40 2020-11-20T12:03:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.779+0000 7f1a9effd700 1 -- 172.21.15.44:0/1493319543 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1aa801a060 con 0x7f1ab80a1a40 2020-11-20T12:03:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.779+0000 7f1a9effd700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a94056cd0 0x7f1a94059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.779+0000 7f1a9effd700 1 -- 172.21.15.44:0/1493319543 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1aa806f920 con 0x7f1ab80a1a40 2020-11-20T12:03:24.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.780+0000 7f1ab6ffd700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a94056cd0 0x7f1a94059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.780+0000 7f1ab6ffd700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a94056cd0 0x7f1a94059190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1aac0035f0 tx=0x7f1aac007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:24.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:24.783+0000 7f1a9effd700 1 -- 172.21.15.44:0/1493319543 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1aa8044c10 con 0x7f1ab80a1a40 2020-11-20T12:03:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.027+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1ab8139440 con 0x7f1ab80a1a40 2020-11-20T12:03:25.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.027+0000 7f1a9effd700 1 -- 172.21.15.44:0/1493319543 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f1aa8018020 con 0x7f1ab80a1a40 2020-11-20T12:03:25.032 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":43,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:03:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a94056cd0 msgr2=0x7f1a94059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a94056cd0 0x7f1a94059190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1aac0035f0 tx=0x7f1aac007b90).stop 2020-11-20T12:03:25.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab80a1a40 msgr2=0x7f1ab8062480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab80a1a40 0x7f1ab8062480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f1aa8009710 tx=0x7f1aa8006e50).stop 2020-11-20T12:03:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 shutdown_connections 2020-11-20T12:03:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a94056cd0 0x7f1a94059190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ab8137bb0 0x7f1ab8131180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ab8066de0 0x7f1ab813f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 --2- 172.21.15.44:0/1493319543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab80a1a40 0x7f1ab8062480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 >> 172.21.15.44:0/1493319543 conn(0x7f1ab80af030 msgr2=0x7f1ab80a1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 shutdown_connections 2020-11-20T12:03:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:25.029+0000 7f1abd42c700 1 -- 172.21.15.44:0/1493319543 wait complete. 2020-11-20T12:03:28.045 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:03:28.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.180+0000 7f3a1c557700 1 Processor -- start 2020-11-20T12:03:28.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.180+0000 7f3a1c557700 1 -- start start 2020-11-20T12:03:28.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1c557700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a140aa020 0x7f3a140a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:28.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1c557700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a1413ac60 0x7f3a140a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:28.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1c557700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a140a9770 con 0x7f3a140aa400 2020-11-20T12:03:28.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1c557700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a140a56c0 con 0x7f3a140aa020 2020-11-20T12:03:28.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1c557700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140a6da0 con 0x7f3a1413ac60 2020-11-20T12:03:28.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1aaf4700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a1413ac60 msgr2=0x7f3a140a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:28.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1aaf4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a1413ac60 0x7f3a140a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:28.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a19af2700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f3a140aa400 0x7f3a140a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40720/0 (socket says 172.21.15.44:40720) 2020-11-20T12:03:28.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a19af2700 1 -- 172.21.15.44:0/2917130452 learned_addr learned my addr 172.21.15.44:0/2917130452 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:28.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a1a2f3700 1 --2- 172.21.15.44:0/2917130452 >> v2:172.21.15.79:3300/0 conn(0x7f3a140aa020 0x7f3a140a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:28.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a192f1700 1 -- 172.21.15.44:0/2917130452 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580171527 0 0) 0x7f3a10001450 con 0x7f3a140aa400 2020-11-20T12:03:28.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.181+0000 7f3a192f1700 1 -- 172.21.15.44:0/2917130452 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39fc003240 con 0x7f3a140aa400 2020-11-20T12:03:28.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1a2f3700 1 -- 172.21.15.44:0/2917130452 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a1413ac60 msgr2=0x7f3a140a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:03:28.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1a2f3700 1 --2- 172.21.15.44:0/2917130452 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a1413ac60 0x7f3a140a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:28.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1a2f3700 1 -- 172.21.15.44:0/2917130452 >> v1:172.21.15.44:6789/0 conn(0x7f3a140aa400 legacy=0x7f3a140a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:28.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1a2f3700 1 -- 172.21.15.44:0/2917130452 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140a7040 con 0x7f3a140aa020 2020-11-20T12:03:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1a2f3700 1 --2- 172.21.15.44:0/2917130452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a140aa020 0x7f3a140a9280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3a0400b9a0 tx=0x7f3a04006df0).ready entity=mon.1 client_cookie=86321ea3f02d1c9a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a192f1700 1 -- 172.21.15.44:0/2917130452 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040097b0 con 0x7f3a140aa020 2020-11-20T12:03:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a192f1700 1 -- 172.21.15.44:0/2917130452 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a04009910 con 0x7f3a140aa020 2020-11-20T12:03:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a192f1700 1 -- 172.21.15.44:0/2917130452 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04009c90 con 0x7f3a140aa020 2020-11-20T12:03:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 -- 172.21.15.44:0/2917130452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a140aa020 msgr2=0x7f3a140a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 --2- 172.21.15.44:0/2917130452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a140aa020 0x7f3a140a9280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3a0400b9a0 tx=0x7f3a04006df0).stop 2020-11-20T12:03:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 -- 172.21.15.44:0/2917130452 shutdown_connections 2020-11-20T12:03:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 --2- 172.21.15.44:0/2917130452 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a1413ac60 0x7f3a140a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 --2- 172.21.15.44:0/2917130452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a140aa020 0x7f3a140a9280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 -- 172.21.15.44:0/2917130452 >> 172.21.15.44:0/2917130452 conn(0x7f3a14062690 msgr2=0x7f3a140a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 -- 172.21.15.44:0/2917130452 shutdown_connections 2020-11-20T12:03:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.182+0000 7f3a1c557700 1 -- 172.21.15.44:0/2917130452 wait complete. 2020-11-20T12:03:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.183+0000 7f3a1c557700 1 Processor -- start 2020-11-20T12:03:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.183+0000 7f3a1c557700 1 -- start start 2020-11-20T12:03:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1c557700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a140aa020 0x7f3a140a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1c557700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a140aa400 0x7f3a14148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1c557700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a1413ac60 0x7f3a14148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1c557700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140a1c40 con 0x7f3a140aa020 2020-11-20T12:03:28.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1c557700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1409d5f0 con 0x7f3a1413ac60 2020-11-20T12:03:28.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1c557700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1409d890 con 0x7f3a140aa400 2020-11-20T12:03:28.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a19af2700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a140aa400 msgr2=0x7f3a14148220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:28.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a19af2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a140aa400 0x7f3a14148220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:28.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1a2f3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a140aa020 0x7f3a140a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1a2f3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a140aa020 0x7f3a140a2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36974/0 (socket says 172.21.15.44:36974) 2020-11-20T12:03:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1a2f3700 1 -- 172.21.15.44:0/3295895586 learned_addr learned my addr 172.21.15.44:0/3295895586 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.184+0000 7f3a1aaf4700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a1413ac60 0x7f3a14148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1a2f3700 1 -- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a140aa400 msgr2=0x7f3a14148220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:03:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1a2f3700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a140aa400 0x7f3a14148220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1a2f3700 1 -- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a1413ac60 msgr2=0x7f3a14148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1a2f3700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a1413ac60 0x7f3a14148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1a2f3700 1 -- 172.21.15.44:0/3295895586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1409dd30 con 0x7f3a140aa020 2020-11-20T12:03:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1aaf4700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a1413ac60 0x7f3a14148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1a2f3700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a140aa020 0x7f3a140a2920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f3a0400c010 tx=0x7f3a04015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a027fc700 1 -- 172.21.15.44:0/3295895586 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04011dd0 con 0x7f3a140aa020 2020-11-20T12:03:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a027fc700 1 -- 172.21.15.44:0/3295895586 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0401e460 con 0x7f3a140aa020 2020-11-20T12:03:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1409df20 con 0x7f3a140aa020 2020-11-20T12:03:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a027fc700 1 -- 172.21.15.44:0/3295895586 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0401e800 con 0x7f3a140aa020 2020-11-20T12:03:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.185+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1409e3b0 con 0x7f3a140aa020 2020-11-20T12:03:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.186+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a14148c60 con 0x7f3a140aa020 2020-11-20T12:03:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.186+0000 7f3a027fc700 1 -- 172.21.15.44:0/3295895586 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3a04025030 con 0x7f3a140aa020 2020-11-20T12:03:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.186+0000 7f3a027fc700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39ec056cd0 0x7f39ec059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.186+0000 7f3a027fc700 1 -- 172.21.15.44:0/3295895586 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3a04071890 con 0x7f3a140aa020 2020-11-20T12:03:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.187+0000 7f3a19af2700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39ec056cd0 0x7f39ec059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.187+0000 7f3a19af2700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39ec056cd0 0x7f39ec059190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3a100048b0 tx=0x7f3a10007990).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.190+0000 7f3a027fc700 1 -- 172.21.15.44:0/3295895586 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3a04046b80 con 0x7f3a140aa020 2020-11-20T12:03:28.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.430+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3a14061650 con 0x7f3a140aa020 2020-11-20T12:03:28.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.430+0000 7f3a027fc700 1 -- 172.21.15.44:0/3295895586 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f3a04023020 con 0x7f3a140aa020 2020-11-20T12:03:28.434 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":47,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:03:28.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39ec056cd0 msgr2=0x7f39ec059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:28.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39ec056cd0 0x7f39ec059190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3a100048b0 tx=0x7f3a10007990).stop 2020-11-20T12:03:28.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a140aa020 msgr2=0x7f3a140a2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:28.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a140aa020 0x7f3a140a2920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f3a0400c010 tx=0x7f3a04015000).stop 2020-11-20T12:03:28.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 shutdown_connections 2020-11-20T12:03:28.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39ec056cd0 0x7f39ec059190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:28.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a1413ac60 0x7f3a14148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:28.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a140aa020 0x7f3a140a2920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:28.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 --2- 172.21.15.44:0/3295895586 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a140aa400 0x7f3a14148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:28.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.431+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 >> 172.21.15.44:0/3295895586 conn(0x7f3a14062690 msgr2=0x7f3a14060a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:28.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.432+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 shutdown_connections 2020-11-20T12:03:28.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:28.432+0000 7f3a1c557700 1 -- 172.21.15.44:0/3295895586 wait complete. 2020-11-20T12:03:31.447 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:03:31.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f3092db4700 1 Processor -- start 2020-11-20T12:03:31.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f3092db4700 1 -- start start 2020-11-20T12:03:31.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f3092db4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f308c0aa020 0x7f308c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:31.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f3092db4700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c0a9770 con 0x7f308c0aa400 2020-11-20T12:03:31.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f3092db4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c0a6da0 con 0x7f308c13ac60 2020-11-20T12:03:31.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f3092db4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c0a56c0 con 0x7f308c0aa020 2020-11-20T12:03:31.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f308bfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f308c0aa400 0x7f308c0a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40734/0 (socket says 172.21.15.44:40734) 2020-11-20T12:03:31.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.584+0000 7f308bfff700 1 -- 172.21.15.44:0/3958399074 learned_addr learned my addr 172.21.15.44:0/3958399074 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:03:31.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f3090b50700 1 --2- 172.21.15.44:0/3958399074 >> v2:172.21.15.79:3300/0 conn(0x7f308c0aa020 0x7f308c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:31.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f3090b50700 1 -- 172.21.15.44:0/3958399074 >> v1:172.21.15.79:6789/0 conn(0x7f308c13ac60 legacy=0x7f308c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:31.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f3090b50700 1 -- 172.21.15.44:0/3958399074 >> v1:172.21.15.44:6789/0 conn(0x7f308c0aa400 legacy=0x7f308c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:31.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f3090b50700 1 -- 172.21.15.44:0/3958399074 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c0a7040 con 0x7f308c0aa020 2020-11-20T12:03:31.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f3090b50700 1 --2- 172.21.15.44:0/3958399074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0aa020 0x7f308c0a9280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f307c00b9a0 tx=0x7f307c006df0).ready entity=mon.1 client_cookie=19d1cbf9d2064067 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:31.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f308b7fe700 1 -- 172.21.15.44:0/3958399074 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c0097b0 con 0x7f308c0aa020 2020-11-20T12:03:31.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f308b7fe700 1 -- 172.21.15.44:0/3958399074 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c009910 con 0x7f308c0aa020 2020-11-20T12:03:31.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.585+0000 7f308b7fe700 1 -- 172.21.15.44:0/3958399074 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c009c90 con 0x7f308c0aa020 2020-11-20T12:03:31.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.586+0000 7f3092db4700 1 -- 172.21.15.44:0/3958399074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0aa020 msgr2=0x7f308c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:31.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.586+0000 7f3092db4700 1 --2- 172.21.15.44:0/3958399074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0aa020 0x7f308c0a9280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f307c00b9a0 tx=0x7f307c006df0).stop 2020-11-20T12:03:31.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.586+0000 7f3092db4700 1 -- 172.21.15.44:0/3958399074 shutdown_connections 2020-11-20T12:03:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.586+0000 7f3092db4700 1 --2- 172.21.15.44:0/3958399074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0aa020 0x7f308c0a9280 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.586+0000 7f3092db4700 1 -- 172.21.15.44:0/3958399074 >> 172.21.15.44:0/3958399074 conn(0x7f308c062690 msgr2=0x7f308c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.586+0000 7f3092db4700 1 -- 172.21.15.44:0/3958399074 shutdown_connections 2020-11-20T12:03:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.586+0000 7f3092db4700 1 -- 172.21.15.44:0/3958399074 wait complete. 2020-11-20T12:03:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.587+0000 7f3092db4700 1 Processor -- start 2020-11-20T12:03:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.587+0000 7f3092db4700 1 -- start start 2020-11-20T12:03:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0aa020 0x7f308c139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f308c0aa400 0x7f308c0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c13ac60 0x7f308c134820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c138f90 con 0x7f308c0aa020 2020-11-20T12:03:31.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0ab390 con 0x7f308c13ac60 2020-11-20T12:03:31.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0ab630 con 0x7f308c0aa400 2020-11-20T12:03:31.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0aa020 0x7f308c139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:31.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0aa020 0x7f308c139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36990/0 (socket says 172.21.15.44:36990) 2020-11-20T12:03:31.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 -- 172.21.15.44:0/2218920506 learned_addr learned my addr 172.21.15.44:0/2218920506 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:31.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f308bfff700 1 -- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f308c0aa400 msgr2=0x7f308c0ad9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:31.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f308bfff700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f308c0aa400 0x7f308c0ad9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:31.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3091351700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c13ac60 0x7f308c134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:31.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 -- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f308c0aa400 msgr2=0x7f308c0ad9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:03:31.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f308c0aa400 0x7f308c0ad9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:31.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 -- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c13ac60 msgr2=0x7f308c134820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:31.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c13ac60 0x7f308c134820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:31.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 -- 172.21.15.44:0/2218920506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c0ad540 con 0x7f308c0aa020 2020-11-20T12:03:31.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3090b50700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0aa020 0x7f308c139a00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f307c00bf10 tx=0x7f307c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:31.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3088ff9700 1 -- 172.21.15.44:0/2218920506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c01e460 con 0x7f308c0aa020 2020-11-20T12:03:31.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3088ff9700 1 -- 172.21.15.44:0/2218920506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c01e5c0 con 0x7f308c0aa020 2020-11-20T12:03:31.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3088ff9700 1 -- 172.21.15.44:0/2218920506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c01e960 con 0x7f308c0aa020 2020-11-20T12:03:31.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c0acf80 con 0x7f308c0aa020 2020-11-20T12:03:31.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.588+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c0ab050 con 0x7f308c0aa020 2020-11-20T12:03:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.590+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c061650 con 0x7f308c0aa020 2020-11-20T12:03:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.590+0000 7f3088ff9700 1 -- 172.21.15.44:0/2218920506 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f307c025030 con 0x7f308c0aa020 2020-11-20T12:03:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.590+0000 7f3088ff9700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3064056cd0 0x7f3064059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.590+0000 7f3088ff9700 1 -- 172.21.15.44:0/2218920506 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f307c071a10 con 0x7f308c0aa020 2020-11-20T12:03:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.590+0000 7f308bfff700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3064056cd0 0x7f3064059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.591+0000 7f308bfff700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3064056cd0 0x7f3064059190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f308c137650 tx=0x7f30800078a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.594+0000 7f3088ff9700 1 -- 172.21.15.44:0/2218920506 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f307c046db0 con 0x7f308c0aa020 2020-11-20T12:03:31.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.832+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f308c0ad6d0 con 0x7f308c0aa020 2020-11-20T12:03:31.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.833+0000 7f3088ff9700 1 -- 172.21.15.44:0/2218920506 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f307c023030 con 0x7f308c0aa020 2020-11-20T12:03:31.838 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":50,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:03:31.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3064056cd0 msgr2=0x7f3064059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:31.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3064056cd0 0x7f3064059190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f308c137650 tx=0x7f30800078a0).stop 2020-11-20T12:03:31.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0aa020 msgr2=0x7f308c139a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:31.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0aa020 0x7f308c139a00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f307c00bf10 tx=0x7f307c015000).stop 2020-11-20T12:03:31.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 shutdown_connections 2020-11-20T12:03:31.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3064056cd0 0x7f3064059190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:31.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c13ac60 0x7f308c134820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308c0aa020 0x7f308c139a00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 --2- 172.21.15.44:0/2218920506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f308c0aa400 0x7f308c0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 >> 172.21.15.44:0/2218920506 conn(0x7f308c062690 msgr2=0x7f308c1361d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.834+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 shutdown_connections 2020-11-20T12:03:31.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:31.835+0000 7f3092db4700 1 -- 172.21.15.44:0/2218920506 wait complete. 2020-11-20T12:03:34.851 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:03:34.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.986+0000 7fb0746fd700 1 Processor -- start 2020-11-20T12:03:34.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.986+0000 7fb0746fd700 1 -- start start 2020-11-20T12:03:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.986+0000 7fb0746fd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb0746fd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 0x7fb06c13b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb0746fd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:34.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb0746fd700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06c13ac80 con 0x7fb06c0a9460 2020-11-20T12:03:34.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb0746fd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06c0ab310 con 0x7fb06c13b560 2020-11-20T12:03:34.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb0746fd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0ab5b0 con 0x7fb06c0a4af0 2020-11-20T12:03:34.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb071c98700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 msgr2=0x7fb06c13b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:34.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb071c98700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 0x7fb06c13b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:34.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072499700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:34.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072499700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36998/0 (socket says 172.21.15.44:36998) 2020-11-20T12:03:34.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 learned_addr learned my addr 172.21.15.44:0/2413041083 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:34.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:34.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:03:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:03:34.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:34.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:34.987+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:35.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb071c98700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 msgr2=0x7fb06c13b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:35.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:35.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb071c98700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 0x7fb06c13b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:03:35.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:03:35.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:35.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:35.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:35.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:03:35.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:03:35.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:35.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:35.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.188+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:03:35.467 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:03:35.489 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:03:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb071c98700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 msgr2=0x7fb06c13b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb071c98700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 0x7fb06c13b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:03:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:03:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:03:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:03:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:35.589+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:03:36.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb071c98700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 msgr2=0x7fb06c13b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb071c98700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 0x7fb06c13b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:03:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:36.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:03:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:36.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:03:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:03:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:36.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:36.390+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:03:37.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 msgr2=0x7fb06c13b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:03:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb06c0a4af0 0x7fb06c13b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 msgr2=0x7fb06c137c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:03:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.79:3300/0 conn(0x7fb06c13b560 0x7fb06c137c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:37.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 -- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 msgr2=0x7fb06c0aad20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:03:37.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 --2- 172.21.15.44:0/2413041083 >> v2:172.21.15.44:3300/0 conn(0x7fb06c0a9460 0x7fb06c0aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:37.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 -- 172.21.15.44:0/2413041083 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06c0a4f50 con 0x7fb04c007030 2020-11-20T12:03:37.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:37.987+0000 7fb06bfff700 1 -- 172.21.15.44:0/2413041083 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c00bd80 con 0x7fb04c0033a0 2020-11-20T12:03:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.837+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008087198 0 0) 0x7fb04c00bd80 con 0x7fb04c0033a0 2020-11-20T12:03:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.837+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb058003240 con 0x7fb04c0033a0 2020-11-20T12:03:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.839+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044421850 0 0) 0x7fb06c0a4f50 con 0x7fb04c007030 2020-11-20T12:03:38.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.839+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c00bd80 con 0x7fb04c007030 2020-11-20T12:03:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.840+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3606419396 0 0) 0x7fb04c00bd80 con 0x7fb04c007030 2020-11-20T12:03:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.840+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 >> v1:172.21.15.79:6789/0 conn(0x7fb04c0033a0 legacy=0x7fb04c005800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:38.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.840+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c00cf80 con 0x7fb04c007030 2020-11-20T12:03:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.840+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fb060003e90 con 0x7fb04c007030 2020-11-20T12:03:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.840+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 >> v1:172.21.15.44:6789/0 conn(0x7fb04c007030 legacy=0x7fb04c0094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.840+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 reap_dead start 2020-11-20T12:03:38.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.840+0000 7fb071497700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb058003cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb071497700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb058004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb071497700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb05800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05800bc20 con 0x7fb0580044e0 2020-11-20T12:03:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05800bd60 con 0x7fb0580038f0 2020-11-20T12:03:38.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05800bea0 con 0x7fb058004e50 2020-11-20T12:03:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072c9a700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 msgr2=0x7fb05800b730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072c9a700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb05800b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb058003cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:38.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb071c98700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb058004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:38.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 msgr2=0x7fb05800b730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:03:38.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb05800b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:38.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 msgr2=0x7fb058004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:38.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb058004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:38.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072499700 1 -- 172.21.15.44:0/2413041083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c00bd80 con 0x7fb0580038f0 2020-11-20T12:03:38.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb071c98700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb058004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:03:38.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.841+0000 7fb072499700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb058003cd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fb064010e50 tx=0x7fb064008040).ready entity=mon.1 client_cookie=da9301c4686980fe server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:38.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.842+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064013e00 con 0x7fb0580038f0 2020-11-20T12:03:38.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.842+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06400c040 con 0x7fb0580038f0 2020-11-20T12:03:38.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.842+0000 7fb071497700 1 -- 172.21.15.44:0/2413041083 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064016670 con 0x7fb0580038f0 2020-11-20T12:03:38.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 -- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 msgr2=0x7fb058003cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:38.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb058003cd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fb064010e50 tx=0x7fb064008040).stop 2020-11-20T12:03:38.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 -- 172.21.15.44:0/2413041083 shutdown_connections 2020-11-20T12:03:38.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb05800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:38.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb058004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:38.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 --2- 172.21.15.44:0/2413041083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb058003cd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:38.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 -- 172.21.15.44:0/2413041083 >> 172.21.15.44:0/2413041083 conn(0x7fb06c0621c0 msgr2=0x7fb06c060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 -- 172.21.15.44:0/2413041083 shutdown_connections 2020-11-20T12:03:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.843+0000 7fb0746fd700 1 -- 172.21.15.44:0/2413041083 wait complete. 2020-11-20T12:03:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.844+0000 7fb0746fd700 1 Processor -- start 2020-11-20T12:03:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.844+0000 7fb0746fd700 1 -- start start 2020-11-20T12:03:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb0746fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb06c1371a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb0746fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb06c141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb0746fd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb06c140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb0746fd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c141d00 con 0x7fb0580044e0 2020-11-20T12:03:38.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb0746fd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c141540 con 0x7fb0580038f0 2020-11-20T12:03:38.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb0746fd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c130250 con 0x7fb058004e50 2020-11-20T12:03:38.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb072c9a700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 msgr2=0x7fb06c140100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:38.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb071c98700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb06c141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:38.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb072c9a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb06c140100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:38.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb071c98700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb06c141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37034/0 (socket says 172.21.15.44:37034) 2020-11-20T12:03:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb071c98700 1 -- 172.21.15.44:0/1044631526 learned_addr learned my addr 172.21.15.44:0/1044631526 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb072499700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb06c1371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb071c98700 1 -- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 msgr2=0x7fb06c140100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:03:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.845+0000 7fb071c98700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb06c140100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:38.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb071c98700 1 -- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 msgr2=0x7fb06c1371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:38.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb071c98700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb06c1371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:38.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb071c98700 1 -- 172.21.15.44:0/1044631526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c1304f0 con 0x7fb0580044e0 2020-11-20T12:03:38.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb071c98700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb06c141810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fb0600035a0 tx=0x7fb060007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:38.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb06a7fc700 1 -- 172.21.15.44:0/1044631526 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060005540 con 0x7fb0580044e0 2020-11-20T12:03:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb06a7fc700 1 -- 172.21.15.44:0/1044631526 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb060012040 con 0x7fb0580044e0 2020-11-20T12:03:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c130790 con 0x7fb0580044e0 2020-11-20T12:03:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb06a7fc700 1 -- 172.21.15.44:0/1044631526 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0600058b0 con 0x7fb0580044e0 2020-11-20T12:03:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.846+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c1405f0 con 0x7fb0580044e0 2020-11-20T12:03:38.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.847+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06c130920 con 0x7fb0580044e0 2020-11-20T12:03:38.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.847+0000 7fb06a7fc700 1 -- 172.21.15.44:0/1044631526 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb060005c20 con 0x7fb0580044e0 2020-11-20T12:03:38.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.848+0000 7fb06a7fc700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb044056cd0 0x7fb044059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:38.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.848+0000 7fb06a7fc700 1 -- 172.21.15.44:0/1044631526 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb0600715a0 con 0x7fb0580044e0 2020-11-20T12:03:38.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.848+0000 7fb072499700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb044056cd0 0x7fb044059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.849+0000 7fb072499700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb044056cd0 0x7fb044059190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb064008010 tx=0x7fb064008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:38.852+0000 7fb06a7fc700 1 -- 172.21.15.44:0/1044631526 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb060046910 con 0x7fb0580044e0 2020-11-20T12:03:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.095+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb06c060880 con 0x7fb0580044e0 2020-11-20T12:03:39.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.096+0000 7fb06a7fc700 1 -- 172.21.15.44:0/1044631526 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1462 (secure 0 0 0) 0x7fb060049f30 con 0x7fb0580044e0 2020-11-20T12:03:39.100 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":8,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:03:39.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb044056cd0 msgr2=0x7fb044059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb044056cd0 0x7fb044059190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb064008010 tx=0x7fb064008040).stop 2020-11-20T12:03:39.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 msgr2=0x7fb06c141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb06c141810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fb0600035a0 tx=0x7fb060007910).stop 2020-11-20T12:03:39.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 shutdown_connections 2020-11-20T12:03:39.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb044056cd0 0x7fb044059190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb058004e50 0x7fb06c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:39.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0580038f0 0x7fb06c1371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 --2- 172.21.15.44:0/1044631526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0580044e0 0x7fb06c141810 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 >> 172.21.15.44:0/1044631526 conn(0x7fb06c0621c0 msgr2=0x7fb06c133c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:39.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 shutdown_connections 2020-11-20T12:03:39.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.097+0000 7fb0746fd700 1 -- 172.21.15.44:0/1044631526 wait complete. 2020-11-20T12:03:39.112 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:03:39.113 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:03:39.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.276+0000 7fcafa7bf700 1 Processor -- start 2020-11-20T12:03:39.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.276+0000 7fcafa7bf700 1 -- start start 2020-11-20T12:03:39.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcafa7bf700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcaf40aa020 0x7fcaf40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcafa7bf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcaf413ac60 0x7fcaf40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcafa7bf700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf40a9770 con 0x7fcaf40aa400 2020-11-20T12:03:39.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcafa7bf700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf40a56c0 con 0x7fcaf40aa020 2020-11-20T12:03:39.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcafa7bf700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf40a6da0 con 0x7fcaf413ac60 2020-11-20T12:03:39.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcaf3fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcaf40aa020 0x7fcaf40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcaf3fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcaf40aa020 0x7fcaf40a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37040/0 (socket says 172.21.15.44:37040) 2020-11-20T12:03:39.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcaf3fff700 1 -- 172.21.15.44:0/2945727057 learned_addr learned my addr 172.21.15.44:0/2945727057 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:39.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.277+0000 7fcaf8d5c700 1 --2- 172.21.15.44:0/2945727057 >> v2:172.21.15.79:3300/0 conn(0x7fcaf413ac60 0x7fcaf40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf3fff700 1 -- 172.21.15.44:0/2945727057 >> v2:172.21.15.79:3300/0 conn(0x7fcaf413ac60 msgr2=0x7fcaf40a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:39.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf3fff700 1 --2- 172.21.15.44:0/2945727057 >> v2:172.21.15.79:3300/0 conn(0x7fcaf413ac60 0x7fcaf40a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf3fff700 1 -- 172.21.15.44:0/2945727057 >> v1:172.21.15.79:6789/0 conn(0x7fcaf40aa400 legacy=0x7fcaf40a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf3fff700 1 -- 172.21.15.44:0/2945727057 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf40a7040 con 0x7fcaf40aa020 2020-11-20T12:03:39.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf8d5c700 1 --2- 172.21.15.44:0/2945727057 >> v2:172.21.15.79:3300/0 conn(0x7fcaf413ac60 0x7fcaf40a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:03:39.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf3fff700 1 --2- 172.21.15.44:0/2945727057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa020 0x7fcaf40a9280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fcaf40a8f70 tx=0x7fcae4006eb0).ready entity=mon.0 client_cookie=a0e902ec3fdc4e76 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf2ffd700 1 -- 172.21.15.44:0/2945727057 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40098d0 con 0x7fcaf40aa020 2020-11-20T12:03:39.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf2ffd700 1 -- 172.21.15.44:0/2945727057 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4009a30 con 0x7fcaf40aa020 2020-11-20T12:03:39.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcaf2ffd700 1 -- 172.21.15.44:0/2945727057 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4009db0 con 0x7fcaf40aa020 2020-11-20T12:03:39.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 -- 172.21.15.44:0/2945727057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa020 msgr2=0x7fcaf40a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/2945727057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa020 0x7fcaf40a9280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fcaf40a8f70 tx=0x7fcae4006eb0).stop 2020-11-20T12:03:39.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 -- 172.21.15.44:0/2945727057 shutdown_connections 2020-11-20T12:03:39.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/2945727057 >> v2:172.21.15.79:3300/0 conn(0x7fcaf413ac60 0x7fcaf40a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/2945727057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa020 0x7fcaf40a9280 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 -- 172.21.15.44:0/2945727057 >> 172.21.15.44:0/2945727057 conn(0x7fcaf4062690 msgr2=0x7fcaf40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:39.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 -- 172.21.15.44:0/2945727057 shutdown_connections 2020-11-20T12:03:39.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.278+0000 7fcafa7bf700 1 -- 172.21.15.44:0/2945727057 wait complete. 2020-11-20T12:03:39.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.279+0000 7fcafa7bf700 1 Processor -- start 2020-11-20T12:03:39.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcafa7bf700 1 -- start start 2020-11-20T12:03:39.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcafa7bf700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcaf40aa020 0x7fcaf414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcafa7bf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa400 0x7fcaf4139860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcafa7bf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcaf413ac60 0x7fcaf4127f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcafa7bf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf414b2f0 con 0x7fcaf40aa400 2020-11-20T12:03:39.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcafa7bf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4126160 con 0x7fcaf413ac60 2020-11-20T12:03:39.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcafa7bf700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41270e0 con 0x7fcaf40aa020 2020-11-20T12:03:39.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf3fff700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcaf40aa020 msgr2=0x7fcaf414bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:39.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf3fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcaf40aa020 0x7fcaf414bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:39.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa400 0x7fcaf4139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa400 0x7fcaf4139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37048/0 (socket says 172.21.15.44:37048) 2020-11-20T12:03:39.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf37fe700 1 -- 172.21.15.44:0/1056350150 learned_addr learned my addr 172.21.15.44:0/1056350150 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:39.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf8d5c700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcaf413ac60 0x7fcaf4127f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf37fe700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcaf40aa020 msgr2=0x7fcaf414bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:03:39.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf37fe700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcaf40aa020 0x7fcaf414bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:39.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.280+0000 7fcaf37fe700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcaf413ac60 msgr2=0x7fcaf4127f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcaf37fe700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcaf413ac60 0x7fcaf4127f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcaf37fe700 1 -- 172.21.15.44:0/1056350150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4127380 con 0x7fcaf40aa400 2020-11-20T12:03:39.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcaf37fe700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa400 0x7fcaf4139860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fcae8001e70 tx=0x7fcae8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8007020 con 0x7fcaf40aa400 2020-11-20T12:03:39.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae8007180 con 0x7fcaf40aa400 2020-11-20T12:03:39.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf4127620 con 0x7fcaf40aa400 2020-11-20T12:03:39.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800e560 con 0x7fcaf40aa400 2020-11-20T12:03:39.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.281+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf414c0d0 con 0x7fcaf40aa400 2020-11-20T12:03:39.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf412a0d0 0x7fcaf40abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4061650 con 0x7fcaf412a0d0 2020-11-20T12:03:39.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcaf3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf412a0d0 0x7fcaf40abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcae8010070 con 0x7fcaf40aa400 2020-11-20T12:03:39.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcaf3fff700 1 -- 172.21.15.44:0/1056350150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcaf40446d0 con 0x7fcaf412a0d0 2020-11-20T12:03:39.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcaf3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf412a0d0 0x7fcaf40abc30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fcaf40a8f70 tx=0x7fcae4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcad3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcacc056d00 0x7fcacc0591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcae806f250 con 0x7fcaf40aa400 2020-11-20T12:03:39.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4011c80 con 0x7fcaf412a0d0 2020-11-20T12:03:39.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.282+0000 7fcaf8d5c700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcacc056d00 0x7fcacc0591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.283+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fcae4011de0 con 0x7fcaf412a0d0 2020-11-20T12:03:39.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.283+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf412a0d0 msgr2=0x7fcaf40abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.283+0000 7fcad3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf412a0d0 0x7fcaf40abc30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fcaf40a8f70 tx=0x7fcae4015000).stop 2020-11-20T12:03:39.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.283+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf412a0d0 msgr2=0x7fcaf40abc30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:39.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.283+0000 7fcad3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf412a0d0 0x7fcaf40abc30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.283+0000 7fcaf8d5c700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcacc056d00 0x7fcacc0591c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fcaec006f50 tx=0x7fcaec008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.290+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4128480 0x7fcaf40ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.290+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40a9770 con 0x7fcaf4128480 2020-11-20T12:03:39.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcaf3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4128480 0x7fcaf40ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcaf3fff700 1 -- 172.21.15.44:0/1056350150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcaf4069560 con 0x7fcaf4128480 2020-11-20T12:03:39.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcaf3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4128480 0x7fcaf40ac120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fcae4006ef0 tx=0x7fcae401e000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae401da00 con 0x7fcaf4128480 2020-11-20T12:03:39.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4765 (secure 0 0 0) 0x7fcae4011a70 con 0x7fcaf4128480 2020-11-20T12:03:39.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4128480 msgr2=0x7fcaf40ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcad3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4128480 0x7fcaf40ac120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fcae4006ef0 tx=0x7fcae401e000).stop 2020-11-20T12:03:39.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcad3fff700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4128480 msgr2=0x7fcaf40ac120 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:39.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.291+0000 7fcad3fff700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf4128480 0x7fcaf40ac120 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcacc056d00 msgr2=0x7fcacc0591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcacc056d00 0x7fcacc0591c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fcaec006f50 tx=0x7fcaec008040).stop 2020-11-20T12:03:39.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa400 msgr2=0x7fcaf4139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa400 0x7fcaf4139860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fcae8001e70 tx=0x7fcae8007c50).stop 2020-11-20T12:03:39.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcaf8d5c700 1 -- 172.21.15.44:0/1056350150 reap_dead start 2020-11-20T12:03:39.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 shutdown_connections 2020-11-20T12:03:39.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 --2- 172.21.15.44:0/1056350150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcaf40aa400 0x7fcaf4139860 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 >> 172.21.15.44:0/1056350150 conn(0x7fcaf4062690 msgr2=0x7fcaf4136ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:39.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 shutdown_connections 2020-11-20T12:03:39.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.292+0000 7fcafa7bf700 1 -- 172.21.15.44:0/1056350150 wait complete. 2020-11-20T12:03:39.307 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:03:39.308 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:03:39.308 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:03:39.309 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:03:39.309 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 8, 2020-11-20T12:03:39.309 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:03:39.310 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:03:39.310 INFO:teuthology.orchestra.run.smithi044.stdout: 1 2020-11-20T12:03:39.311 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.311 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:03:39.311 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:39.312 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:03:39.312 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:03:39.313 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:39.313 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:39.313 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:39.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:39.314 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:39.315 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:39.315 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:39.315 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:39.316 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.316 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:03:39.317 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:03:39.317 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:39.317 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:39.318 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:39.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:39.318 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:39.319 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:39.319 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:39.320 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:39.320 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.321 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:03:39.321 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:03:39.322 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:03:39.322 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:03:39.322 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:03:39.323 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:03:39.323 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:39.324 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:39.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:03:39.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:03:39.325 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:03:39.325 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:03:39.325 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:03:39.326 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:39.326 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:03:39.326 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:39.326 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:39.327 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:39.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:39.327 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:39.327 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:39.327 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:39.328 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:39.328 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.328 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:03:39.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.328 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:03:39.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.329 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:03:39.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:03:39.329 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:39.330 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:39.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:39.330 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:03:39.330 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:39.331 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:03:39.331 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.332 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:03:39.332 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:03:39.333 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:39.333 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:39.333 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:39.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.334 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:03:39.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:03:39.334 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:39.334 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:39.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:39.335 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:03:39.335 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:39.336 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:03:39.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.337 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.337 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:03:39.337 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:03:39.338 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:39.338 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:39.338 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:39.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.339 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:03:39.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:03:39.339 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:39.339 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:39.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:39.340 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:03:39.340 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:39.341 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:03:39.341 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.341 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.342 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:03:39.342 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:03:39.342 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:39.343 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:39.343 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:39.343 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.343 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.344 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:03:39.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:03:39.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.344 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.344 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.345 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:39.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.345 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.345 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:03:39.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.346 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.346 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.346 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:39.346 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.346 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.347 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:03:39.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.347 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.347 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.347 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:03:39.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.348 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.348 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:03:39.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.349 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.349 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.349 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:03:39.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.349 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.350 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:03:39.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.350 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.350 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.351 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:03:39.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.351 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.351 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:03:39.352 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:03:39.353 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:03:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.476+0000 7fcd923f3700 1 Processor -- start 2020-11-20T12:03:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.476+0000 7fcd923f3700 1 -- start start 2020-11-20T12:03:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd923f3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcd8c0aa020 0x7fcd8c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd923f3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcd8c13ac60 0x7fcd8c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd923f3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0a9770 con 0x7fcd8c0aa400 2020-11-20T12:03:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd923f3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c0a56c0 con 0x7fcd8c13ac60 2020-11-20T12:03:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd923f3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c0a6da0 con 0x7fcd8c0aa020 2020-11-20T12:03:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcd8c13ac60 0x7fcd8c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcd8c13ac60 0x7fcd8c0a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37058/0 (socket says 172.21.15.44:37058) 2020-11-20T12:03:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 -- 172.21.15.44:0/3518778289 learned_addr learned my addr 172.21.15.44:0/3518778289 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd8bfff700 1 --2- 172.21.15.44:0/3518778289 >> v2:172.21.15.79:3300/0 conn(0x7fcd8c0aa020 0x7fcd8c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 -- 172.21.15.44:0/3518778289 >> v2:172.21.15.79:3300/0 conn(0x7fcd8c0aa020 msgr2=0x7fcd8c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:03:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 --2- 172.21.15.44:0/3518778289 >> v2:172.21.15.79:3300/0 conn(0x7fcd8c0aa020 0x7fcd8c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 -- 172.21.15.44:0/3518778289 >> v1:172.21.15.79:6789/0 conn(0x7fcd8c0aa400 legacy=0x7fcd8c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 -- 172.21.15.44:0/3518778289 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0a7040 con 0x7fcd8c13ac60 2020-11-20T12:03:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd90990700 1 --2- 172.21.15.44:0/3518778289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c13ac60 0x7fcd8c0a3700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fcd8c0adec0 tx=0x7fcd84009290).ready entity=mon.0 client_cookie=cd8faad1eff43a0c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.477+0000 7fcd8affd700 1 -- 172.21.15.44:0/3518778289 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400bbc0 con 0x7fcd8c13ac60 2020-11-20T12:03:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd8affd700 1 -- 172.21.15.44:0/3518778289 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8400bd20 con 0x7fcd8c13ac60 2020-11-20T12:03:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd8affd700 1 -- 172.21.15.44:0/3518778289 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84011610 con 0x7fcd8c13ac60 2020-11-20T12:03:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 -- 172.21.15.44:0/3518778289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c13ac60 msgr2=0x7fcd8c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 --2- 172.21.15.44:0/3518778289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c13ac60 0x7fcd8c0a3700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fcd8c0adec0 tx=0x7fcd84009290).stop 2020-11-20T12:03:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 -- 172.21.15.44:0/3518778289 shutdown_connections 2020-11-20T12:03:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 --2- 172.21.15.44:0/3518778289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c13ac60 0x7fcd8c0a3700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 --2- 172.21.15.44:0/3518778289 >> v2:172.21.15.79:3300/0 conn(0x7fcd8c0aa020 0x7fcd8c0a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 -- 172.21.15.44:0/3518778289 >> 172.21.15.44:0/3518778289 conn(0x7fcd8c062690 msgr2=0x7fcd8c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 -- 172.21.15.44:0/3518778289 shutdown_connections 2020-11-20T12:03:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.478+0000 7fcd923f3700 1 -- 172.21.15.44:0/3518778289 wait complete. 2020-11-20T12:03:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.479+0000 7fcd923f3700 1 Processor -- start 2020-11-20T12:03:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.479+0000 7fcd923f3700 1 -- start start 2020-11-20T12:03:39.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd923f3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd8c0aa020 0x7fcd8c130f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd923f3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0aa400 0x7fcd8c0ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd923f3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c13ac60 0x7fcd8c0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd923f3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0ad4c0 con 0x7fcd8c0aa400 2020-11-20T12:03:39.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd923f3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0ab1e0 con 0x7fcd8c13ac60 2020-11-20T12:03:39.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd923f3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c130850 con 0x7fcd8c0aa020 2020-11-20T12:03:39.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8bfff700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd8c0aa020 msgr2=0x7fcd8c130f90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:03:39.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd8c0aa020 0x7fcd8c130f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:39.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0aa400 0x7fcd8c0ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0aa400 0x7fcd8c0ac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37064/0 (socket says 172.21.15.44:37064) 2020-11-20T12:03:39.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 -- 172.21.15.44:0/1509783205 learned_addr learned my addr 172.21.15.44:0/1509783205 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:39.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd90990700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c13ac60 0x7fcd8c0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd8c0aa020 msgr2=0x7fcd8c130f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:03:39.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd8c0aa020 0x7fcd8c130f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:03:39.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c13ac60 msgr2=0x7fcd8c0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c13ac60 0x7fcd8c0acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:39.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 -- 172.21.15.44:0/1509783205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c130af0 con 0x7fcd8c0aa400 2020-11-20T12:03:39.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd8b7fe700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0aa400 0x7fcd8c0ac7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fcd8000c6e0 tx=0x7fcd80007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80007020 con 0x7fcd8c0aa400 2020-11-20T12:03:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd80007180 con 0x7fcd8c0aa400 2020-11-20T12:03:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c0a31f0 con 0x7fcd8c0aa400 2020-11-20T12:03:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.480+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8000e560 con 0x7fcd8c0aa400 2020-11-20T12:03:39.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.481+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c131ce0 con 0x7fcd8c0aa400 2020-11-20T12:03:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.481+0000 7fcd923f3700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c0aac60 0x7fcd8c12f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.481+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0ab040 con 0x7fcd8c0aac60 2020-11-20T12:03:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.481+0000 7fcd8bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c0aac60 0x7fcd8c12f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcd80010070 con 0x7fcd8c0aa400 2020-11-20T12:03:39.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd8bfff700 1 -- 172.21.15.44:0/1509783205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd8c0a8fa0 con 0x7fcd8c0aac60 2020-11-20T12:03:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd8bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c0aac60 0x7fcd8c12f0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fcd7c006fd0 tx=0x7fcd7c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcd64056cd0 0x7fcd64059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fcd8006f2a0 con 0x7fcd8c0aa400 2020-11-20T12:03:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00c8b0 con 0x7fcd8c0aac60 2020-11-20T12:03:39.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd90990700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcd64056cd0 0x7fcd64059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fcd7c00ca10 con 0x7fcd8c0aac60 2020-11-20T12:03:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c0aac60 msgr2=0x7fcd8c12f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c0aac60 0x7fcd8c12f0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fcd7c006fd0 tx=0x7fcd7c006df0).stop 2020-11-20T12:03:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c0aac60 msgr2=0x7fcd8c12f0a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:39.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.482+0000 7fcd6bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c0aac60 0x7fcd8c12f0a0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0x7fcd7c006fd0 tx=0x7fcd7c006df0).stop 2020-11-20T12:03:39.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.483+0000 7fcd90990700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcd64056cd0 0x7fcd64059190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fcd840092d0 tx=0x7fcd84016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.490+0000 7fcd923f3700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c12f590 0x7fcd8c14d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.490+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0a9770 con 0x7fcd8c12f590 2020-11-20T12:03:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.490+0000 7fcd8bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c12f590 0x7fcd8c14d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd8bfff700 1 -- 172.21.15.44:0/1509783205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcd8c0a9330 con 0x7fcd8c12f590 2020-11-20T12:03:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd8bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c12f590 0x7fcd8c14d820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fcd7c000bc0 tx=0x7fcd7c019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c021640 con 0x7fcd8c12f590 2020-11-20T12:03:39.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4593 (secure 0 0 0) 0x7fcd7c00c6a0 con 0x7fcd8c12f590 2020-11-20T12:03:39.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c12f590 msgr2=0x7fcd8c14d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd6bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c12f590 0x7fcd8c14d820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fcd7c000bc0 tx=0x7fcd7c019000).stop 2020-11-20T12:03:39.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd6bfff700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c12f590 msgr2=0x7fcd8c14d820 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:03:39.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.491+0000 7fcd6bfff700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd8c12f590 0x7fcd8c14d820 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0x7fcd7c000bc0 tx=0x7fcd7c019000).stop 2020-11-20T12:03:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcd64056cd0 msgr2=0x7fcd64059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcd64056cd0 0x7fcd64059190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fcd840092d0 tx=0x7fcd84016000).stop 2020-11-20T12:03:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0aa400 msgr2=0x7fcd8c0ac7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 --2- 172.21.15.44:0/1509783205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd8c0aa400 0x7fcd8c0ac7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fcd8000c6e0 tx=0x7fcd80007c50).stop 2020-11-20T12:03:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd90990700 1 -- 172.21.15.44:0/1509783205 reap_dead start 2020-11-20T12:03:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd90990700 1 -- 172.21.15.44:0/1509783205 reap_dead start 2020-11-20T12:03:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 shutdown_connections 2020-11-20T12:03:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 >> 172.21.15.44:0/1509783205 conn(0x7fcd8c062690 msgr2=0x7fcd8c13bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 shutdown_connections 2020-11-20T12:03:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:39.492+0000 7fcd923f3700 1 -- 172.21.15.44:0/1509783205 wait complete. 2020-11-20T12:03:39.505 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:03:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:03:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:03:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:03:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 8, 2020-11-20T12:03:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:03:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:03:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: 1 2020-11-20T12:03:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:03:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:03:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:03:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:03:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:03:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:03:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:03:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:03:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:03:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:03:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:03:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:03:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:03:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:03:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:03:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:03:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:03:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:03:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:03:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:03:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:03:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:03:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:03:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:03:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:03:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:03:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:03:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:03:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:03:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:03:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:03:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:03:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:03:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:03:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:03:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:03:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:03:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:03:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:03:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:03:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:03:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:03:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:03:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:03:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:03:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:03:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:03:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:03:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:03:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:03:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:03:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:03:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:03:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:03:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:03:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:03:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:03:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:03:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:03:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:03:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:03:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:03:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:03:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:03:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:03:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:03:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:03:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:03:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:03:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:03:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:03:39.538 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:03:39.539 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:03:59.540 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2020-11-20T12:03:59.541 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2020-11-20T12:03:59.541 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-20T12:03:59.542 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-20T12:03:59.550 INFO:tasks.ceph.mon.c:Started 2020-11-20T12:03:59.551 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:03:59.551 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:03:59.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.693+0000 7ff8b1d2d700 1 Processor -- start 2020-11-20T12:03:59.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.693+0000 7ff8b1d2d700 1 -- start start 2020-11-20T12:03:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.693+0000 7ff8b1d2d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 0x7ff8ac065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:59.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.693+0000 7ff8b1d2d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff8ac13c880 0x7ff8ac066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.693+0000 7ff8b1d2d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac0a8bf0 con 0x7ff8ac0a7810 2020-11-20T12:03:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.693+0000 7ff8b1d2d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac066520 con 0x7ff8ac13c880 2020-11-20T12:03:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.693+0000 7ff8b1d2d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0a86b0 con 0x7ff8ac0aa1e0 2020-11-20T12:03:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8abfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff8ac13c880 0x7ff8ac066030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8ab7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 0x7ff8ac065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8ab7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 0x7ff8ac065b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48514/0 (socket says 172.21.15.44:48514) 2020-11-20T12:03:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8abfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff8ac13c880 0x7ff8ac066030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57362/0 (socket says 172.21.15.44:57362) 2020-11-20T12:03:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8ab7fe700 1 -- 172.21.15.44:0/2522145038 learned_addr learned my addr 172.21.15.44:0/2522145038 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:59.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8a3fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff8ac0a7810 0x7ff8ac13c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:43700/0 (socket says 172.21.15.44:43700) 2020-11-20T12:03:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8ab7fe700 1 -- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 msgr2=0x7ff8ac065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:03:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8ab7fe700 1 -- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 msgr2=0x7ff8ac065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:03:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8ab7fe700 1 --2- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 0x7ff8ac065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8ab7fe700 1 --2- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 0x7ff8ac065b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:59.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8abfff700 1 -- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 msgr2=0x7ff8ac065b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:03:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8abfff700 1 --2- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 0x7ff8ac065b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8abfff700 1 -- 172.21.15.44:0/2522145038 >> v1:172.21.15.79:6789/0 conn(0x7ff8ac0a7810 legacy=0x7ff8ac13c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8aaffd700 1 -- 172.21.15.44:0/2522145038 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218719848 0 0) 0x7ff8ac0a8bf0 con 0x7ff8ac0a7810 2020-11-20T12:03:59.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8abfff700 1 -- 172.21.15.44:0/2522145038 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac0a8950 con 0x7ff8ac13c880 2020-11-20T12:03:59.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8abfff700 1 --2- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac13c880 0x7ff8ac066030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff8a4009c90 tx=0x7ff8a40091d0).ready entity=mon.1 client_cookie=39d3cb82985d59d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:59.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8aaffd700 1 -- 172.21.15.44:0/2522145038 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400bbc0 con 0x7ff8ac13c880 2020-11-20T12:03:59.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.694+0000 7ff8aaffd700 1 -- 172.21.15.44:0/2522145038 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a400bd20 con 0x7ff8ac13c880 2020-11-20T12:03:59.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8aaffd700 1 -- 172.21.15.44:0/2522145038 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4011610 con 0x7ff8ac13c880 2020-11-20T12:03:59.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac13c880 msgr2=0x7ff8ac066030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac13c880 0x7ff8ac066030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff8a4009c90 tx=0x7ff8a40091d0).stop 2020-11-20T12:03:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/2522145038 shutdown_connections 2020-11-20T12:03:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac13c880 0x7ff8ac066030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/2522145038 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0aa1e0 0x7ff8ac065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/2522145038 >> 172.21.15.44:0/2522145038 conn(0x7ff8ac062690 msgr2=0x7ff8ac0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/2522145038 shutdown_connections 2020-11-20T12:03:59.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.695+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/2522145038 wait complete. 2020-11-20T12:03:59.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.697+0000 7ff8b1d2d700 1 Processor -- start 2020-11-20T12:03:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.697+0000 7ff8b1d2d700 1 -- start start 2020-11-20T12:03:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.697+0000 7ff8b1d2d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 0x7ff8ac13f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8b1d2d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac0aa1e0 0x7ff8ac0a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8b1d2d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ac13c880 0x7ff8ac137950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:59.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8b1d2d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0a01d0 con 0x7ff8ac13c880 2020-11-20T12:03:59.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8b1d2d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0a0470 con 0x7ff8ac0aa1e0 2020-11-20T12:03:59.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8b1d2d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0a0710 con 0x7ff8ac0a7810 2020-11-20T12:03:59.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ac13c880 0x7ff8ac137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:59.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ac13c880 0x7ff8ac137950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37086/0 (socket says 172.21.15.44:37086) 2020-11-20T12:03:59.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 -- 172.21.15.44:0/3707661639 learned_addr learned my addr 172.21.15.44:0/3707661639 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:03:59.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8ab7fe700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 0x7ff8ac13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:59.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8a3fff700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac0aa1e0 0x7ff8ac0a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:59.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8ab7fe700 1 -- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 msgr2=0x7ff8ac13f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:03:59.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8ab7fe700 1 -- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 msgr2=0x7ff8ac13f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:03:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8ab7fe700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 0x7ff8ac13f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:03:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8ab7fe700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 0x7ff8ac13f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:03:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 -- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 msgr2=0x7ff8ac13f160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:03:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 0x7ff8ac13f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 -- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac0aa1e0 msgr2=0x7ff8ac0a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac0aa1e0 0x7ff8ac0a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 -- 172.21.15.44:0/3707661639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac0a21b0 con 0x7ff8ac13c880 2020-11-20T12:03:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8abfff700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ac13c880 0x7ff8ac137950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff8a4009f80 tx=0x7ff8a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8a37fe700 1 -- 172.21.15.44:0/3707661639 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4011e20 con 0x7ff8ac13c880 2020-11-20T12:03:59.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8a37fe700 1 -- 172.21.15.44:0/3707661639 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a4010b90 con 0x7ff8ac13c880 2020-11-20T12:03:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8a37fe700 1 -- 172.21.15.44:0/3707661639 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a401d480 con 0x7ff8ac13c880 2020-11-20T12:03:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.698+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac0a23a0 con 0x7ff8ac13c880 2020-11-20T12:03:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.699+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac138000 con 0x7ff8ac13c880 2020-11-20T12:03:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.700+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac0a08d0 con 0x7ff8ac13c880 2020-11-20T12:03:59.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.700+0000 7ff8a37fe700 1 -- 172.21.15.44:0/3707661639 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff8a4017070 con 0x7ff8ac13c880 2020-11-20T12:03:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.701+0000 7ff8a37fe700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff888056d20 0x7ff8880591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:03:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.701+0000 7ff8a37fe700 1 -- 172.21.15.44:0/3707661639 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff8a40727f0 con 0x7ff8ac13c880 2020-11-20T12:03:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.701+0000 7ff8ab7fe700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff888056d20 0x7ff8880591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:03:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.701+0000 7ff8ab7fe700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff888056d20 0x7ff8880591e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7ff89c006fd0 tx=0x7ff89c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:03:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.704+0000 7ff8a37fe700 1 -- 172.21.15.44:0/3707661639 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff8a4047ae0 con 0x7ff8ac13c880 2020-11-20T12:03:59.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.948+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff8ac061650 con 0x7ff8ac13c880 2020-11-20T12:03:59.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.948+0000 7ff8a37fe700 1 -- 172.21.15.44:0/3707661639 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7ff8a4015070 con 0x7ff8ac13c880 2020-11-20T12:03:59.953 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:03:59.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff888056d20 msgr2=0x7ff8880591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:59.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff888056d20 0x7ff8880591e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7ff89c006fd0 tx=0x7ff89c008040).stop 2020-11-20T12:03:59.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ac13c880 msgr2=0x7ff8ac137950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:03:59.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ac13c880 0x7ff8ac137950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7ff8a4009f80 tx=0x7ff8a4016000).stop 2020-11-20T12:03:59.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 shutdown_connections 2020-11-20T12:03:59.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff888056d20 0x7ff8880591e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8ac13c880 0x7ff8ac137950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff8ac0a7810 0x7ff8ac13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 --2- 172.21.15.44:0/3707661639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8ac0aa1e0 0x7ff8ac0a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:03:59.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 >> 172.21.15.44:0/3707661639 conn(0x7ff8ac062690 msgr2=0x7ff8ac0a98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:03:59.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 shutdown_connections 2020-11-20T12:03:59.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:03:59.950+0000 7ff8b1d2d700 1 -- 172.21.15.44:0/3707661639 wait complete. 2020-11-20T12:03:59.965 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:03:59.966 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:04:00.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f122e206700 1 Processor -- start 2020-11-20T12:04:00.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f122e206700 1 -- start start 2020-11-20T12:04:00.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f122e206700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12280a7810 0x7f122813c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f122e206700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1228066250 0x7f122813c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f122e206700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12280a8bf0 con 0x7f12280aa1e0 2020-11-20T12:04:00.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f122e206700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12280a86b0 con 0x7f1228066250 2020-11-20T12:04:00.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f122e206700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12280a8950 con 0x7f12280a7810 2020-11-20T12:04:00.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f1227fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1228066250 0x7f122813c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f12277fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f12280aa1e0 0x7f1228065b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40838/0 (socket says 172.21.15.44:40838) 2020-11-20T12:04:00.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.103+0000 7f12277fe700 1 -- 172.21.15.44:0/3192237009 learned_addr learned my addr 172.21.15.44:0/3192237009 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:00.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f121ffff700 1 --2- 172.21.15.44:0/3192237009 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12280a7810 0x7f122813c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1227fff700 1 -- 172.21.15.44:0/3192237009 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12280a7810 msgr2=0x7f122813c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:00.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1227fff700 1 --2- 172.21.15.44:0/3192237009 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12280a7810 0x7f122813c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1227fff700 1 -- 172.21.15.44:0/3192237009 >> v1:172.21.15.44:6789/0 conn(0x7f12280aa1e0 legacy=0x7f1228065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1227fff700 1 -- 172.21.15.44:0/3192237009 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280aa5c0 con 0x7f1228066250 2020-11-20T12:04:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1226ffd700 1 -- 172.21.15.44:0/3192237009 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319166536 0 0) 0x7f12280a8bf0 con 0x7f12280aa1e0 2020-11-20T12:04:00.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1227fff700 1 --2- 172.21.15.44:0/3192237009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1228066250 0x7f122813c660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f1220009c90 tx=0x7f12200091d0).ready entity=mon.0 client_cookie=93acd47e20b44971 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1226ffd700 1 -- 172.21.15.44:0/3192237009 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122000bbc0 con 0x7f1228066250 2020-11-20T12:04:00.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1226ffd700 1 -- 172.21.15.44:0/3192237009 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122000bd20 con 0x7f1228066250 2020-11-20T12:04:00.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.104+0000 7f1226ffd700 1 -- 172.21.15.44:0/3192237009 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220011610 con 0x7f1228066250 2020-11-20T12:04:00.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 -- 172.21.15.44:0/3192237009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1228066250 msgr2=0x7f122813c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 --2- 172.21.15.44:0/3192237009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1228066250 0x7f122813c660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f1220009c90 tx=0x7f12200091d0).stop 2020-11-20T12:04:00.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 -- 172.21.15.44:0/3192237009 shutdown_connections 2020-11-20T12:04:00.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 --2- 172.21.15.44:0/3192237009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1228066250 0x7f122813c660 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 --2- 172.21.15.44:0/3192237009 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12280a7810 0x7f122813c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 -- 172.21.15.44:0/3192237009 >> 172.21.15.44:0/3192237009 conn(0x7f1228062690 msgr2=0x7f12280a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:00.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 -- 172.21.15.44:0/3192237009 shutdown_connections 2020-11-20T12:04:00.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.105+0000 7f122e206700 1 -- 172.21.15.44:0/3192237009 wait complete. 2020-11-20T12:04:00.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.107+0000 7f122e206700 1 Processor -- start 2020-11-20T12:04:00.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.107+0000 7f122e206700 1 -- start start 2020-11-20T12:04:00.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.107+0000 7f122e206700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 0x7f12280a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.107+0000 7f122e206700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12280a7810 0x7f1228139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.107+0000 7f122e206700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12280aa1e0 0x7f122813a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.107+0000 7f122e206700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122813a570 con 0x7f12280aa1e0 2020-11-20T12:04:00.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f122e206700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122809eda0 con 0x7f12280a7810 2020-11-20T12:04:00.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f122e206700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122809f040 con 0x7f1228066250 2020-11-20T12:04:00.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12280aa1e0 0x7f122813a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12280aa1e0 0x7f122813a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37100/0 (socket says 172.21.15.44:37100) 2020-11-20T12:04:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 -- 172.21.15.44:0/327063826 learned_addr learned my addr 172.21.15.44:0/327063826 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f12277fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 0x7f12280a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f121ffff700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12280a7810 0x7f1228139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f12277fe700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 msgr2=0x7f12280a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:04:00.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f12277fe700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 msgr2=0x7f12280a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f12277fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 0x7f12280a3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 msgr2=0x7f12280a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f12277fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 0x7f12280a3570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1228066250 0x7f12280a3570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12280a7810 msgr2=0x7f1228139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12280a7810 0x7f1228139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 -- 172.21.15.44:0/327063826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122813b4b0 con 0x7f12280aa1e0 2020-11-20T12:04:00.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f121ffff700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12280a7810 0x7f1228139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:00.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f1227fff700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12280aa1e0 0x7f122813a080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f1220009210 tx=0x7f1220016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220011e20 con 0x7f12280aa1e0 2020-11-20T12:04:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1220010c40 con 0x7f12280aa1e0 2020-11-20T12:04:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122001d480 con 0x7f12280aa1e0 2020-11-20T12:04:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122809e890 con 0x7f12280aa1e0 2020-11-20T12:04:00.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.108+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12280a0bf0 con 0x7f12280aa1e0 2020-11-20T12:04:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.109+0000 7f122e206700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809de60 0x7f122814d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.109+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228135230 con 0x7f122809de60 2020-11-20T12:04:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.110+0000 7f12277fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809de60 0x7f122814d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.110+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1220017070 con 0x7f12280aa1e0 2020-11-20T12:04:00.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.110+0000 7f12277fe700 1 -- 172.21.15.44:0/327063826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f122809f200 con 0x7f122809de60 2020-11-20T12:04:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.110+0000 7f12277fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809de60 0x7f122814d520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f121800c5e0 tx=0x7f1218009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.110+0000 7f121f7fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1204056d20 0x7f12040591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.110+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f12200727d0 con 0x7f12280aa1e0 2020-11-20T12:04:00.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121ffff700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1204056d20 0x7f12040591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218007320 con 0x7f122809de60 2020-11-20T12:04:00.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f1218007480 con 0x7f122809de60 2020-11-20T12:04:00.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809de60 msgr2=0x7f122814d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121f7fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809de60 0x7f122814d520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f121800c5e0 tx=0x7f1218009040).stop 2020-11-20T12:04:00.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809de60 msgr2=0x7f122814d520 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121f7fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809de60 0x7f122814d520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.111+0000 7f121ffff700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1204056d20 0x7f12040591e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1210006fd0 tx=0x7f1210008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.119+0000 7f122e206700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809e240 0x7f122814da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.119+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228061650 con 0x7f122809e240 2020-11-20T12:04:00.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.119+0000 7f12277fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809e240 0x7f122814da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.119+0000 7f12277fe700 1 -- 172.21.15.44:0/327063826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f12280446d0 con 0x7f122809e240 2020-11-20T12:04:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.119+0000 7f12277fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809e240 0x7f122814da10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f1218014000 tx=0x7f1218009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.119+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218014b60 con 0x7f122809e240 2020-11-20T12:04:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f1218007110 con 0x7f122809e240 2020-11-20T12:04:00.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809e240 msgr2=0x7f122814da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f121f7fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809e240 0x7f122814da10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f1218014000 tx=0x7f1218009040).stop 2020-11-20T12:04:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f121f7fe700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809e240 msgr2=0x7f122814da10 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f121f7fe700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f122809e240 0x7f122814da10 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0x7f1218014000 tx=0x7f1218009040).stop 2020-11-20T12:04:00.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1204056d20 msgr2=0x7f12040591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1204056d20 0x7f12040591e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1210006fd0 tx=0x7f1210008040).stop 2020-11-20T12:04:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12280aa1e0 msgr2=0x7f122813a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 --2- 172.21.15.44:0/327063826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12280aa1e0 0x7f122813a080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f1220009210 tx=0x7f1220016000).stop 2020-11-20T12:04:00.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f1227fff700 1 -- 172.21.15.44:0/327063826 reap_dead start 2020-11-20T12:04:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f1227fff700 1 -- 172.21.15.44:0/327063826 reap_dead start 2020-11-20T12:04:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 shutdown_connections 2020-11-20T12:04:00.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 >> 172.21.15.44:0/327063826 conn(0x7f1228062690 msgr2=0x7f12280aba60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 shutdown_connections 2020-11-20T12:04:00.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.120+0000 7f122e206700 1 -- 172.21.15.44:0/327063826 wait complete. 2020-11-20T12:04:00.141 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:00.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:00.142 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:00.142 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:04:00.143 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 10, 2020-11-20T12:04:00.143 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:00.143 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:00.144 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:00.144 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:00.144 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.145 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:04:00.145 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:00.145 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:00.146 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:00.146 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:00.146 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:00.147 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:00.147 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:00.147 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:00.147 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:00.148 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:00.148 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:00.148 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.148 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:00.149 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:00.149 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:00.149 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:00.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:00.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:00.150 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:00.150 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:00.150 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:00.151 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:00.151 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.151 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:00.151 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:00.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:00.152 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:00.152 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:00.152 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:00.152 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:00.153 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:00.153 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:00.153 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:00.153 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:00.154 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:00.154 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:00.154 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:00.154 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:00.154 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:00.155 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:00.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:00.155 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:00.155 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:00.155 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:00.156 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:00.156 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:00.156 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.156 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:00.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:00.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.157 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:00.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:00.158 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:00.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:00.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:00.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:00.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:00.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:00.160 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.160 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:00.161 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:00.161 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:00.161 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:00.161 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:00.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.162 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:00.162 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:00.162 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:00.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:00.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:00.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:00.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:00.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:00.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:00.165 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:00.166 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:00.166 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:00.166 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:00.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.167 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:00.167 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:00.167 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:00.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:00.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:00.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:00.168 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:00.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:00.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:00.170 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:00.170 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:00.171 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:00.171 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:00.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.171 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.172 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:00.172 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:00.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.172 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.172 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.173 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:00.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.173 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.173 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:04:00.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.174 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.174 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.174 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:00.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.174 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.175 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:04:00.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.175 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.175 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.175 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:04:00.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.176 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:00.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.176 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.177 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.177 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:04:00.177 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.177 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.178 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:04:00.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.178 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.178 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.178 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:04:00.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.179 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.179 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:00.179 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:00.181 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:04:00.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.316+0000 7f7668e18700 1 Processor -- start 2020-11-20T12:04:00.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.316+0000 7f7668e18700 1 -- start start 2020-11-20T12:04:00.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7668e18700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7668e18700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f76640a4c30 0x7f766413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7668e18700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76640a6260 con 0x7f766413ae80 2020-11-20T12:04:00.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7668e18700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76640a52e0 con 0x7f76640a4c30 2020-11-20T12:04:00.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7668e18700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76640a6490 con 0x7f7664060330 2020-11-20T12:04:00.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7662d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f766413ae80 0x7f7664136f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40858/0 (socket says 172.21.15.44:40858) 2020-11-20T12:04:00.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7662d9d700 1 -- 172.21.15.44:0/517842928 learned_addr learned my addr 172.21.15.44:0/517842928 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:00.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f765bfff700 1 --2- 172.21.15.44:0/517842928 >> v2:172.21.15.79:3300/0 conn(0x7f76640a4c30 0x7f766413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f766259c700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f766259c700 1 -- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 msgr2=0x7f76640a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:00.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f766259c700 1 -- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 msgr2=0x7f76640a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:00.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7661d9b700 1 -- 172.21.15.44:0/517842928 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418959329 0 0) 0x7f76640a6260 con 0x7f766413ae80 2020-11-20T12:04:00.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f766259c700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:00.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f766259c700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:00.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.317+0000 7f7661d9b700 1 -- 172.21.15.44:0/517842928 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7644003240 con 0x7f766413ae80 2020-11-20T12:04:00.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f765bfff700 1 -- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 msgr2=0x7f76640a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:00.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f765bfff700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f765bfff700 1 -- 172.21.15.44:0/517842928 >> v1:172.21.15.44:6789/0 conn(0x7f766413ae80 legacy=0x7f7664136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f765bfff700 1 -- 172.21.15.44:0/517842928 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640a6730 con 0x7f76640a4c30 2020-11-20T12:04:00.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f765bfff700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76640a4c30 0x7f766413a990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f764c00b9a0 tx=0x7f764c006df0).ready entity=mon.1 client_cookie=2908bcfb553bbef1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f7661d9b700 1 -- 172.21.15.44:0/517842928 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0097b0 con 0x7f76640a4c30 2020-11-20T12:04:00.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f7661d9b700 1 -- 172.21.15.44:0/517842928 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c009910 con 0x7f76640a4c30 2020-11-20T12:04:00.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.318+0000 7f7661d9b700 1 -- 172.21.15.44:0/517842928 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c009c90 con 0x7f76640a4c30 2020-11-20T12:04:00.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 -- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76640a4c30 msgr2=0x7f766413a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76640a4c30 0x7f766413a990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f764c00b9a0 tx=0x7f764c006df0).stop 2020-11-20T12:04:00.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 -- 172.21.15.44:0/517842928 shutdown_connections 2020-11-20T12:04:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 --2- 172.21.15.44:0/517842928 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76640a4c30 0x7f766413a990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 -- 172.21.15.44:0/517842928 >> 172.21.15.44:0/517842928 conn(0x7f7664062690 msgr2=0x7f76640a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 -- 172.21.15.44:0/517842928 shutdown_connections 2020-11-20T12:04:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.319+0000 7f7668e18700 1 -- 172.21.15.44:0/517842928 wait complete. 2020-11-20T12:04:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 Processor -- start 2020-11-20T12:04:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 -- start start 2020-11-20T12:04:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76640a4c30 0x7f7664066a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766413ae80 0x7f7664135e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640ac970 con 0x7f76640a4c30 2020-11-20T12:04:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766409d470 con 0x7f766413ae80 2020-11-20T12:04:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f7668e18700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76640a3650 con 0x7f7664060330 2020-11-20T12:04:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f765bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76640a4c30 0x7f7664066a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f765bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76640a4c30 0x7f7664066a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37114/0 (socket says 172.21.15.44:37114) 2020-11-20T12:04:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.320+0000 7f765bfff700 1 -- 172.21.15.44:0/2557746284 learned_addr learned my addr 172.21.15.44:0/2557746284 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f766259c700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f7662d9d700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766413ae80 0x7f7664135e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f766259c700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 msgr2=0x7f76640aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:04:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f766259c700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 msgr2=0x7f76640aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:00.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f766259c700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f766259c700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640aba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765bfff700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 msgr2=0x7f76640aba90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765bfff700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7664060330 0x7f76640aba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765bfff700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766413ae80 msgr2=0x7f7664135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765bfff700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766413ae80 0x7f7664135e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765bfff700 1 -- 172.21.15.44:0/2557746284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766409db00 con 0x7f76640a4c30 2020-11-20T12:04:00.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765bfff700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76640a4c30 0x7f7664066a90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f764c006e30 tx=0x7f764c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c011dd0 con 0x7f76640a4c30 2020-11-20T12:04:00.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c01e460 con 0x7f76640a4c30 2020-11-20T12:04:00.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76640ac6d0 con 0x7f76640a4c30 2020-11-20T12:04:00.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c01e7e0 con 0x7f76640a4c30 2020-11-20T12:04:00.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.321+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76640abf80 con 0x7f76640a4c30 2020-11-20T12:04:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.322+0000 7f7668e18700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136370 0x7f7664146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.322+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7664061650 con 0x7f7664136370 2020-11-20T12:04:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.322+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f764c025030 con 0x7f76640a4c30 2020-11-20T12:04:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.323+0000 7f766259c700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136370 0x7f7664146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.323+0000 7f766259c700 1 -- 172.21.15.44:0/2557746284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76640446d0 con 0x7f7664136370 2020-11-20T12:04:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.323+0000 7f765a7fc700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7640056d20 0x7f76400591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.323+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f764c071850 con 0x7f76640a4c30 2020-11-20T12:04:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.323+0000 7f766259c700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136370 0x7f7664146590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f7654006c00 tx=0x7f7654008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.323+0000 7f7662d9d700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7640056d20 0x7f76400591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.323+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7654011660 con 0x7f7664136370 2020-11-20T12:04:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.324+0000 7f7662d9d700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7640056d20 0x7f76400591e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f765c0013b0 tx=0x7f765c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.324+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f76540117c0 con 0x7f7664136370 2020-11-20T12:04:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.324+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136370 msgr2=0x7f7664146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.324+0000 7f765a7fc700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136370 0x7f7664146590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f7654006c00 tx=0x7f7654008040).stop 2020-11-20T12:04:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.324+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136370 msgr2=0x7f7664146590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.324+0000 7f765a7fc700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136370 0x7f7664146590 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.339+0000 7f7668e18700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136750 0x7f7664065340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.339+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640680f0 con 0x7f7664136750 2020-11-20T12:04:00.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.339+0000 7f766259c700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136750 0x7f7664065340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f766259c700 1 -- 172.21.15.44:0/2557746284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76640613b0 con 0x7f7664136750 2020-11-20T12:04:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f766259c700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136750 0x7f7664065340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f76540067d0 tx=0x7f765401b000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7654021660 con 0x7f7664136750 2020-11-20T12:04:00.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4605 (secure 0 0 0) 0x7f7654011450 con 0x7f7664136750 2020-11-20T12:04:00.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136750 msgr2=0x7f7664065340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f765a7fc700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136750 0x7f7664065340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f76540067d0 tx=0x7f765401b000).stop 2020-11-20T12:04:00.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f765a7fc700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136750 msgr2=0x7f7664065340 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f765a7fc700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664136750 0x7f7664065340 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0x7f76540067d0 tx=0x7f765401b000).stop 2020-11-20T12:04:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7640056d20 msgr2=0x7f76400591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f7668e18700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7640056d20 0x7f76400591e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f765c0013b0 tx=0x7f765c009040).stop 2020-11-20T12:04:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76640a4c30 msgr2=0x7f7664066a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.340+0000 7f7668e18700 1 --2- 172.21.15.44:0/2557746284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76640a4c30 0x7f7664066a90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f764c006e30 tx=0x7f764c015000).stop 2020-11-20T12:04:00.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.341+0000 7f7662d9d700 1 -- 172.21.15.44:0/2557746284 reap_dead start 2020-11-20T12:04:00.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.341+0000 7f7662d9d700 1 -- 172.21.15.44:0/2557746284 reap_dead start 2020-11-20T12:04:00.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.341+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 shutdown_connections 2020-11-20T12:04:00.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.341+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 >> 172.21.15.44:0/2557746284 conn(0x7f7664062690 msgr2=0x7f766405ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:00.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.341+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 shutdown_connections 2020-11-20T12:04:00.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.341+0000 7f7668e18700 1 -- 172.21.15.44:0/2557746284 wait complete. 2020-11-20T12:04:00.362 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:00.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:00.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:00.363 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:04:00.363 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 10, 2020-11-20T12:04:00.364 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:00.364 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:00.365 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:00.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:00.365 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.366 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:04:00.366 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:00.367 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:00.367 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:00.367 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:00.368 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:00.368 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:00.369 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:00.369 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:00.369 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:00.370 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:00.370 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:00.371 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.375 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:00.375 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:00.376 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:00.376 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:00.376 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:00.377 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:00.377 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:00.377 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:00.377 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:00.379 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:00.379 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:00.379 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:00.379 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:00.379 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:00.380 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:00.380 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:00.380 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:00.381 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:00.381 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:00.381 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:00.381 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:00.382 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:00.382 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:00.382 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:00.383 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:00.383 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:00.383 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:00.384 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:00.384 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:00.384 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:00.385 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:00.385 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.386 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:00.387 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:00.388 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:00.388 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:00.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.389 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:00.389 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:00.390 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.390 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:00.391 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:00.391 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.391 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.391 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.392 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:00.392 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:00.392 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:00.392 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:00.392 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:00.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:00.394 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:00.394 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:00.394 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:00.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:00.395 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:00.396 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:00.396 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:00.397 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.397 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.398 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:00.398 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:00.398 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:00.398 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:00.399 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:00.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.399 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:00.400 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:00.400 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:00.400 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:00.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.401 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:00.401 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:00.401 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:00.402 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:00.402 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:00.402 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.402 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.403 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:00.403 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:00.403 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:00.404 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:00.404 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:00.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.404 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.405 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:00.405 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:00.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.406 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.406 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.406 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:00.407 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.407 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.407 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:04:00.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.407 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.408 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.408 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:00.409 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.409 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.409 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:04:00.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.410 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.411 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.411 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:04:00.411 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.412 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:00.412 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:00.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:00.413 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:00.413 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:00.414 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:04:00.414 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:00.414 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:00.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:00.415 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:00.415 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:00.416 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:04:00.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.551+0000 7f0579125700 1 Processor -- start 2020-11-20T12:04:00.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.551+0000 7f0579125700 1 -- start start 2020-11-20T12:04:00.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.551+0000 7f0579125700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.551+0000 7f0579125700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f05740a4c30 0x7f057413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.551+0000 7f0579125700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05740a6260 con 0x7f057413ae80 2020-11-20T12:04:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.551+0000 7f0579125700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05740a52e0 con 0x7f05740a4c30 2020-11-20T12:04:00.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.551+0000 7f0579125700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05740a6490 con 0x7f0574060330 2020-11-20T12:04:00.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f056bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f05740a4c30 0x7f057413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f057359e700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f057413ae80 0x7f0574136f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40876/0 (socket says 172.21.15.44:40876) 2020-11-20T12:04:00.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f057359e700 1 -- 172.21.15.44:0/2441621378 learned_addr learned my addr 172.21.15.44:0/2441621378 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:00.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f0572d9d700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f0572d9d700 1 -- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 msgr2=0x7f05740a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f0572d9d700 1 -- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 msgr2=0x7f05740a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f0572d9d700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f057259c700 1 -- 172.21.15.44:0/2441621378 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179450022 0 0) 0x7f05740a6260 con 0x7f057413ae80 2020-11-20T12:04:00.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f056bfff700 1 -- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 msgr2=0x7f05740a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:00.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f0572d9d700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f056bfff700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f056bfff700 1 -- 172.21.15.44:0/2441621378 >> v1:172.21.15.44:6789/0 conn(0x7f057413ae80 legacy=0x7f0574136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f056bfff700 1 -- 172.21.15.44:0/2441621378 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740a6730 con 0x7f05740a4c30 2020-11-20T12:04:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f056bfff700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f057413a990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f055c00dc30 tx=0x7f055c006e80).ready entity=mon.0 client_cookie=e7f79b20c1f838f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f057259c700 1 -- 172.21.15.44:0/2441621378 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00bb90 con 0x7f05740a4c30 2020-11-20T12:04:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f057259c700 1 -- 172.21.15.44:0/2441621378 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c00bcf0 con 0x7f05740a4c30 2020-11-20T12:04:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.552+0000 7f057259c700 1 -- 172.21.15.44:0/2441621378 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c012980 con 0x7f05740a4c30 2020-11-20T12:04:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 -- 172.21.15.44:0/2441621378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 msgr2=0x7f057413a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f057413a990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f055c00dc30 tx=0x7f055c006e80).stop 2020-11-20T12:04:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 -- 172.21.15.44:0/2441621378 shutdown_connections 2020-11-20T12:04:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 --2- 172.21.15.44:0/2441621378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f057413a990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 -- 172.21.15.44:0/2441621378 >> 172.21.15.44:0/2441621378 conn(0x7f0574062690 msgr2=0x7f05740a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 -- 172.21.15.44:0/2441621378 shutdown_connections 2020-11-20T12:04:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.553+0000 7f0579125700 1 -- 172.21.15.44:0/2441621378 wait complete. 2020-11-20T12:04:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.554+0000 7f0579125700 1 Processor -- start 2020-11-20T12:04:00.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.554+0000 7f0579125700 1 -- start start 2020-11-20T12:04:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0579125700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0579125700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f05740ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0579125700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f057413ae80 0x7f05741490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0579125700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05740ab880 con 0x7f05740a4c30 2020-11-20T12:04:00.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0579125700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05741340e0 con 0x7f057413ae80 2020-11-20T12:04:00.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0579125700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05740acc40 con 0x7f0574060330 2020-11-20T12:04:00.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f05740ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f05740ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37132/0 (socket says 172.21.15.44:37132) 2020-11-20T12:04:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 -- 172.21.15.44:0/1482434752 learned_addr learned my addr 172.21.15.44:0/1482434752 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 msgr2=0x7f05740ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f057359e700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f057413ae80 0x7f05741490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f057413ae80 msgr2=0x7f05741490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f057413ae80 0x7f05741490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740acee0 con 0x7f05740a4c30 2020-11-20T12:04:00.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574060330 0x7f05740ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:04:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f057359e700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f057413ae80 0x7f05741490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:04:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.555+0000 7f056bfff700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f05740ab390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f055c006ec0 tx=0x7f055c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.556+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c021070 con 0x7f05740a4c30 2020-11-20T12:04:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.556+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c026420 con 0x7f05740a4c30 2020-11-20T12:04:00.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.556+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05740ad380 con 0x7f05740a4c30 2020-11-20T12:04:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.556+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0267c0 con 0x7f05740a4c30 2020-11-20T12:04:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.556+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0574149820 con 0x7f05740a4c30 2020-11-20T12:04:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.556+0000 7f0579125700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.556+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05741342a0 con 0x7f0574133b60 2020-11-20T12:04:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.557+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.557+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2020-11-20T12:04:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.557+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.557+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:00.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.557+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.558+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f055c02c020 con 0x7f05740a4c30 2020-11-20T12:04:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.558+0000 7f056affd700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0550056d20 0x7f05500591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.558+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f055c076340 con 0x7f05740a4c30 2020-11-20T12:04:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.558+0000 7f057359e700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0550056d20 0x7f05500591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.559+0000 7f057359e700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0550056d20 0x7f05500591e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f056c000f40 tx=0x7f056c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:00.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.758+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:00.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.758+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:04:00.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.758+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.758+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:00.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:00.758+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:04:01.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.159+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:01.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.159+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:04:01.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.159+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:01.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.159+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:01.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.159+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:04:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.960+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.960+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:04:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.960+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.960+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:01.961+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:04:03.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:03.563+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:03.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:03.563+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:04:03.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:03.563+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:03.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:03.563+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:03.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:03.563+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 3.200000 2020-11-20T12:04:04.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0569ffb700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:04.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0569ffb700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0569ffb700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:04.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0569ffb700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 msgr2=0x7f057412f170 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:04.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0569ffb700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0574133b60 0x7f057412f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0569ffb700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0544005d20 con 0x7f0544003230 2020-11-20T12:04:04.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 msgr2=0x7f05440056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:04:04.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 msgr2=0x7f05440056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:04.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:04.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.555+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:04.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.756+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0544005e90 con 0x7f0544003230 2020-11-20T12:04:04.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0564006c00 tx=0x7f0564008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:04.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056401d070 con 0x7f0544003230 2020-11-20T12:04:04.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f056400c040 con 0x7f0544003230 2020-11-20T12:04:04.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 msgr2=0x7f05440056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f056affd700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0564006c00 tx=0x7f0564008040).stop 2020-11-20T12:04:04.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 msgr2=0x7f05440056c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:04.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.757+0000 7f056affd700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0544003230 0x7f05440056c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.773+0000 7f0579125700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f057412f660 0x7f0574145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:04.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.773+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0574061650 con 0x7f057412f660 2020-11-20T12:04:04.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.773+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f057412f660 0x7f0574145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.773+0000 7f0572d9d700 1 -- 172.21.15.44:0/1482434752 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f05740446d0 con 0x7f057412f660 2020-11-20T12:04:04.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f0572d9d700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f057412f660 0x7f0574145b80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0564021000 tx=0x7f0564008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:04.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564021830 con 0x7f057412f660 2020-11-20T12:04:04.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4265 (secure 0 0 0) 0x7f0564014a30 con 0x7f057412f660 2020-11-20T12:04:04.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f057412f660 msgr2=0x7f0574145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f056affd700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f057412f660 0x7f0574145b80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0564021000 tx=0x7f0564008040).stop 2020-11-20T12:04:04.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f056affd700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f057412f660 msgr2=0x7f0574145b80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:04.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f056affd700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f057412f660 0x7f0574145b80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f057359e700 1 -- 172.21.15.44:0/1482434752 reap_dead start 2020-11-20T12:04:04.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0550056d20 msgr2=0x7f05500591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f0579125700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0550056d20 0x7f05500591e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f056c000f40 tx=0x7f056c009040).stop 2020-11-20T12:04:04.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.774+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 msgr2=0x7f05740ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.775+0000 7f0579125700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f05740ab390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f055c006ec0 tx=0x7f055c019000).stop 2020-11-20T12:04:04.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.775+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 shutdown_connections 2020-11-20T12:04:04.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.775+0000 7f0579125700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0550056d20 0x7f05500591e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.775+0000 7f0579125700 1 --2- 172.21.15.44:0/1482434752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740a4c30 0x7f05740ab390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.775+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 >> 172.21.15.44:0/1482434752 conn(0x7f0574062690 msgr2=0x7f057405ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:04.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.775+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 shutdown_connections 2020-11-20T12:04:04.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.775+0000 7f0579125700 1 -- 172.21.15.44:0/1482434752 wait complete. 2020-11-20T12:04:04.783 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:04.783 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:04.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:04.783 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:04:04.783 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 10, 2020-11-20T12:04:04.784 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:04.784 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:04.784 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:04.784 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:04.784 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:04.785 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 4, 2020-11-20T12:04:04.785 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:04.785 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:04.785 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:04.785 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:04.786 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:04.786 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:04.786 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:04.786 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:04.786 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:04.787 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:04.787 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:04.787 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:04.787 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:04.787 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:04.788 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:04.788 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:04.788 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:04.788 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:04.788 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:04.789 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:04.789 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:04.789 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:04.789 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:04.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.790 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:04.790 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:04.790 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:04.790 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:04.790 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:04.791 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:04.791 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:04.791 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:04.791 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:04.791 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:04.792 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:04.792 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:04.792 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:04.792 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:04.792 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:04.793 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:04.793 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:04.793 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:04.793 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:04.793 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:04.794 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:04.794 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:04.794 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:04.794 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:04.794 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:04.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.795 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:04.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:04.795 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:04.796 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:04.796 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:04.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:04.796 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:04.797 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:04.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.797 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:04.797 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:04.798 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:04.798 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:04.798 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:04.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.798 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:04.799 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:04.799 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:04.799 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:04.799 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:04.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:04.800 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:04.800 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:04.800 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:04.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:04.801 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:04.801 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:04.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:04.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:04.802 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:04.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:04.803 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:04.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.803 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:04.803 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:04.804 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:04.804 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:04.804 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:04.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:04.805 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:04.805 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:04.805 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:04.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:04.806 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:04.806 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:04.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.807 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:04.807 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:04.807 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:04.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:04.807 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:04.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.808 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:04.808 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:04.808 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:04.808 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:04.809 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:04.809 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:04.809 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:04.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.810 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:04.810 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:04.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.810 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:04.811 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:04.811 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:04.811 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:04.811 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:04.811 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:04.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:04.812 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:04.812 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:04.812 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:04.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:04.813 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:04.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:04.813 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:04.813 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:04.814 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:04:04.815 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:04:04.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.949+0000 7f50e51cb700 1 Processor -- start 2020-11-20T12:04:04.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.949+0000 7f50e51cb700 1 -- start start 2020-11-20T12:04:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.950+0000 7f50e51cb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f50e00a6520 0x7f50e01343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.950+0000 7f50e51cb700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e00a6980 con 0x7f50e00a74e0 2020-11-20T12:04:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.950+0000 7f50e51cb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e00a6bc0 con 0x7f50e00a4e30 2020-11-20T12:04:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.950+0000 7f50e51cb700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e0136d90 con 0x7f50e00a6520 2020-11-20T12:04:04.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.950+0000 7f50e49ca700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f50e00a6520 0x7f50e01343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.950+0000 7f50dffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f50e00a74e0 0x7f50e00a90f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40902/0 (socket says 172.21.15.44:40902) 2020-11-20T12:04:04.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50dffff700 1 -- 172.21.15.44:0/64295589 learned_addr learned my addr 172.21.15.44:0/64295589 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:04.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50e49ca700 1 -- 172.21.15.44:0/64295589 >> v1:172.21.15.79:6789/0 conn(0x7f50e00a4e30 legacy=0x7f50e0133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50e49ca700 1 -- 172.21.15.44:0/64295589 >> v1:172.21.15.44:6789/0 conn(0x7f50e00a74e0 legacy=0x7f50e00a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50e49ca700 1 -- 172.21.15.44:0/64295589 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e00a8c50 con 0x7f50e00a6520 2020-11-20T12:04:04.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50deffd700 1 -- 172.21.15.44:0/64295589 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131149922 0 0) 0x7f50d0001030 con 0x7f50e00a74e0 2020-11-20T12:04:04.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50e49ca700 1 --2- 172.21.15.44:0/64295589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e01343f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f50d800bf30 tx=0x7f50d800b470).ready entity=mon.0 client_cookie=9ff80a632b065ae5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:04.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50deffd700 1 -- 172.21.15.44:0/64295589 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d800de00 con 0x7f50e00a6520 2020-11-20T12:04:04.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50deffd700 1 -- 172.21.15.44:0/64295589 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8010680 con 0x7f50e00a6520 2020-11-20T12:04:04.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.951+0000 7f50deffd700 1 -- 172.21.15.44:0/64295589 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8010a00 con 0x7f50e00a6520 2020-11-20T12:04:04.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.952+0000 7f50e51cb700 1 -- 172.21.15.44:0/64295589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 msgr2=0x7f50e01343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.952+0000 7f50e51cb700 1 --2- 172.21.15.44:0/64295589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e01343f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f50d800bf30 tx=0x7f50d800b470).stop 2020-11-20T12:04:04.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.952+0000 7f50e51cb700 1 -- 172.21.15.44:0/64295589 shutdown_connections 2020-11-20T12:04:04.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.952+0000 7f50e51cb700 1 --2- 172.21.15.44:0/64295589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e01343f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.952+0000 7f50e51cb700 1 -- 172.21.15.44:0/64295589 >> 172.21.15.44:0/64295589 conn(0x7f50e00ad520 msgr2=0x7f50e00ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:04.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.952+0000 7f50e51cb700 1 -- 172.21.15.44:0/64295589 shutdown_connections 2020-11-20T12:04:04.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.952+0000 7f50e51cb700 1 -- 172.21.15.44:0/64295589 wait complete. 2020-11-20T12:04:04.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.953+0000 7f50e51cb700 1 Processor -- start 2020-11-20T12:04:04.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.953+0000 7f50e51cb700 1 -- start start 2020-11-20T12:04:04.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50e51cb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f50e00a4e30 0x7f50e01363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:04.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50e51cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e00a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:04.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50e51cb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50e00a74e0 0x7f50e00a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:04.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50e51cb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00a2390 con 0x7f50e00a6520 2020-11-20T12:04:04.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50e51cb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00a2800 con 0x7f50e00a74e0 2020-11-20T12:04:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50e51cb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00a2a90 con 0x7f50e00a4e30 2020-11-20T12:04:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e00a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e00a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37162/0 (socket says 172.21.15.44:37162) 2020-11-20T12:04:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 -- 172.21.15.44:0/1497245638 learned_addr learned my addr 172.21.15.44:0/1497245638 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50dffff700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f50e00a4e30 0x7f50e01363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50e49ca700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50e00a74e0 0x7f50e00a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 -- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f50e00a4e30 msgr2=0x7f50e01363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f50e00a4e30 0x7f50e01363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 -- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50e00a74e0 msgr2=0x7f50e00a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50e00a74e0 0x7f50e00a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:04.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 -- 172.21.15.44:0/1497245638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e00a2d30 con 0x7f50e00a6520 2020-11-20T12:04:04.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.954+0000 7f50df7fe700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e00a19b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f50d4001e70 tx=0x7f50d4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:04.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.955+0000 7f50bffff700 1 -- 172.21.15.44:0/1497245638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4007020 con 0x7f50e00a6520 2020-11-20T12:04:04.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.955+0000 7f50bffff700 1 -- 172.21.15.44:0/1497245638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d4007180 con 0x7f50e00a6520 2020-11-20T12:04:04.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.955+0000 7f50bffff700 1 -- 172.21.15.44:0/1497245638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400e560 con 0x7f50e00a6520 2020-11-20T12:04:04.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.955+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e00a2fd0 con 0x7f50e00a6520 2020-11-20T12:04:04.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.955+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e00ac700 con 0x7f50e00a6520 2020-11-20T12:04:04.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.956+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e00a3160 con 0x7f50e00a6520 2020-11-20T12:04:04.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.957+0000 7f50bffff700 1 -- 172.21.15.44:0/1497245638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f50d4010070 con 0x7f50e00a6520 2020-11-20T12:04:04.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.957+0000 7f50bffff700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f50b8056d20 0x7f50b80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:04.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.957+0000 7f50bffff700 1 -- 172.21.15.44:0/1497245638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f50d406f4a0 con 0x7f50e00a6520 2020-11-20T12:04:04.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.957+0000 7f50dffff700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f50b8056d20 0x7f50b80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:04.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.958+0000 7f50dffff700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f50b8056d20 0x7f50b80591e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f50d0000bc0 tx=0x7f50d0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:04.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:04.961+0000 7f50bffff700 1 -- 172.21.15.44:0/1497245638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f50d4044790 con 0x7f50e00a6520 2020-11-20T12:04:05.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.203+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f50e0057d80 con 0x7f50e00a6520 2020-11-20T12:04:05.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.204+0000 7f50bffff700 1 -- 172.21.15.44:0/1497245638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f50d4047db0 con 0x7f50e00a6520 2020-11-20T12:04:05.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f50b8056d20 msgr2=0x7f50b80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:05.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f50b8056d20 0x7f50b80591e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f50d0000bc0 tx=0x7f50d0009040).stop 2020-11-20T12:04:05.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 msgr2=0x7f50e00a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:05.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e00a19b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f50d4001e70 tx=0x7f50d4007c50).stop 2020-11-20T12:04:05.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 shutdown_connections 2020-11-20T12:04:05.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f50b8056d20 0x7f50b80591e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:05.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f50e00a74e0 0x7f50e00a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:05.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f50e00a4e30 0x7f50e01363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:05.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 --2- 172.21.15.44:0/1497245638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50e00a6520 0x7f50e00a19b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:05.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 >> 172.21.15.44:0/1497245638 conn(0x7f50e00ad520 msgr2=0x7f50e0133cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:05.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 shutdown_connections 2020-11-20T12:04:05.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:05.206+0000 7f50e51cb700 1 -- 172.21.15.44:0/1497245638 wait complete. 2020-11-20T12:04:05.225 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:04:05.564 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:04:05.567 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:04:14.900 INFO:tasks.workunit.client.0.smithi044.stderr:Note: checking out 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'. 2020-11-20T12:04:14.900 INFO:tasks.workunit.client.0.smithi044.stderr: 2020-11-20T12:04:14.900 INFO:tasks.workunit.client.0.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2020-11-20T12:04:14.901 INFO:tasks.workunit.client.0.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2020-11-20T12:04:14.901 INFO:tasks.workunit.client.0.smithi044.stderr:state without impacting any branches by performing another checkout. 2020-11-20T12:04:14.901 INFO:tasks.workunit.client.0.smithi044.stderr: 2020-11-20T12:04:14.901 INFO:tasks.workunit.client.0.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2020-11-20T12:04:14.901 INFO:tasks.workunit.client.0.smithi044.stderr:do so (now or later) by using -b with the checkout command again. Example: 2020-11-20T12:04:14.902 INFO:tasks.workunit.client.0.smithi044.stderr: 2020-11-20T12:04:14.902 INFO:tasks.workunit.client.0.smithi044.stderr: git checkout -b 2020-11-20T12:04:14.902 INFO:tasks.workunit.client.0.smithi044.stderr: 2020-11-20T12:04:14.902 INFO:tasks.workunit.client.0.smithi044.stderr:HEAD is now at b86e691bf1 Merge PR #37174 into wip-pdonnell-testing-20201120.053228 2020-11-20T12:04:14.904 INFO:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2020-11-20T12:04:14.928 INFO:tasks.workunit.client.0.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2020-11-20T12:04:14.930 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2020-11-20T12:04:14.930 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2020-11-20T12:04:15.226 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:04:15.226 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:04:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.350+0000 7f1d6ceb8700 1 Processor -- start 2020-11-20T12:04:15.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.350+0000 7f1d6ceb8700 1 -- start start 2020-11-20T12:04:15.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6ceb8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1d680aa020 0x7f1d680a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6ceb8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6ceb8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1d6813ac60 0x7f1d680a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6ceb8700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d680a9770 con 0x7f1d6813ac60 2020-11-20T12:04:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6ceb8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d680a6da0 con 0x7f1d680aa020 2020-11-20T12:04:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6ceb8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680a7040 con 0x7f1d680aa400 2020-11-20T12:04:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1d6813ac60 0x7f1d680a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1d6813ac60 0x7f1d680a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37172/0 (socket says 172.21.15.44:37172) 2020-11-20T12:04:15.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 -- 172.21.15.44:0/907372346 learned_addr learned my addr 172.21.15.44:0/907372346 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:15.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6659c700 1 --2- 172.21.15.44:0/907372346 >> v2:172.21.15.79:3300/0 conn(0x7f1d680aa020 0x7f1d680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d65d9b700 1 --2- 172.21.15.44:0/907372346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 -- 172.21.15.44:0/907372346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 msgr2=0x7f1d680a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 --2- 172.21.15.44:0/907372346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 -- 172.21.15.44:0/907372346 >> v2:172.21.15.79:3300/0 conn(0x7f1d680aa020 msgr2=0x7f1d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 --2- 172.21.15.44:0/907372346 >> v2:172.21.15.79:3300/0 conn(0x7f1d680aa020 0x7f1d680a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 -- 172.21.15.44:0/907372346 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680a6a90 con 0x7f1d6813ac60 2020-11-20T12:04:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d6659c700 1 --2- 172.21.15.44:0/907372346 >> v2:172.21.15.79:3300/0 conn(0x7f1d680aa020 0x7f1d680a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d65d9b700 1 --2- 172.21.15.44:0/907372346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:04:15.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.351+0000 7f1d66d9d700 1 --2- 172.21.15.44:0/907372346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d6813ac60 0x7f1d680a3700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f1d680adec0 tx=0x7f1d60006eb0).ready entity=mon.0 client_cookie=e1eada18fb819019 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6559a700 1 -- 172.21.15.44:0/907372346 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d600098d0 con 0x7f1d6813ac60 2020-11-20T12:04:15.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6559a700 1 -- 172.21.15.44:0/907372346 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60009a30 con 0x7f1d6813ac60 2020-11-20T12:04:15.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6559a700 1 -- 172.21.15.44:0/907372346 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60009db0 con 0x7f1d6813ac60 2020-11-20T12:04:15.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/907372346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d6813ac60 msgr2=0x7f1d680a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/907372346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d6813ac60 0x7f1d680a3700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f1d680adec0 tx=0x7f1d60006eb0).stop 2020-11-20T12:04:15.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/907372346 shutdown_connections 2020-11-20T12:04:15.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/907372346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d6813ac60 0x7f1d680a3700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/907372346 >> v2:172.21.15.79:3300/0 conn(0x7f1d680aa020 0x7f1d680a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/907372346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/907372346 >> 172.21.15.44:0/907372346 conn(0x7f1d68062690 msgr2=0x7f1d680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:15.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/907372346 shutdown_connections 2020-11-20T12:04:15.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.352+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/907372346 wait complete. 2020-11-20T12:04:15.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.353+0000 7f1d6ceb8700 1 Processor -- start 2020-11-20T12:04:15.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.353+0000 7f1d6ceb8700 1 -- start start 2020-11-20T12:04:15.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6ceb8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d680aa020 0x7f1d68137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6ceb8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6ceb8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d6813ac60 0x7f1d680ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6ceb8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680acba0 con 0x7f1d680aa020 2020-11-20T12:04:15.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6ceb8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680ad630 con 0x7f1d6813ac60 2020-11-20T12:04:15.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6ceb8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680ad8d0 con 0x7f1d680aa400 2020-11-20T12:04:15.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d680aa020 0x7f1d68137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d680aa020 0x7f1d68137a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37174/0 (socket says 172.21.15.44:37174) 2020-11-20T12:04:15.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 -- 172.21.15.44:0/203125863 learned_addr learned my addr 172.21.15.44:0/203125863 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:15.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d66d9d700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d6813ac60 0x7f1d680ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d65d9b700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 -- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 msgr2=0x7f1d680ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 -- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d6813ac60 msgr2=0x7f1d680ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d6813ac60 0x7f1d680ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 -- 172.21.15.44:0/203125863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680a2dc0 con 0x7f1d680aa020 2020-11-20T12:04:15.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6659c700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d680aa020 0x7f1d68137a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1d5c00db60 tx=0x7f1d5c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d567fc700 1 -- 172.21.15.44:0/203125863 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c009820 con 0x7f1d680aa020 2020-11-20T12:04:15.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d567fc700 1 -- 172.21.15.44:0/203125863 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c009980 con 0x7f1d680aa020 2020-11-20T12:04:15.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d567fc700 1 -- 172.21.15.44:0/203125863 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c009d20 con 0x7f1d680aa020 2020-11-20T12:04:15.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.354+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d680ad190 con 0x7f1d680aa020 2020-11-20T12:04:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.355+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d680669d0 con 0x7f1d680aa020 2020-11-20T12:04:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.355+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d68061650 con 0x7f1d680aa020 2020-11-20T12:04:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.356+0000 7f1d567fc700 1 -- 172.21.15.44:0/203125863 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1d5c013690 con 0x7f1d680aa020 2020-11-20T12:04:15.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.356+0000 7f1d567fc700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d3c056d20 0x7f1d3c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.356+0000 7f1d567fc700 1 -- 172.21.15.44:0/203125863 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1d5c070080 con 0x7f1d680aa020 2020-11-20T12:04:15.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.356+0000 7f1d65d9b700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d3c056d20 0x7f1d3c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.357+0000 7f1d65d9b700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d3c056d20 0x7f1d3c0591e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f1d68065af0 tx=0x7f1d58008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.360+0000 7f1d567fc700 1 -- 172.21.15.44:0/203125863 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1d5c045310 con 0x7f1d680aa020 2020-11-20T12:04:15.564 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2020-11-20T12:04:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.603+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1d680a6100 con 0x7f1d680aa020 2020-11-20T12:04:15.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.603+0000 7f1d567fc700 1 -- 172.21.15.44:0/203125863 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f1d5c048930 con 0x7f1d680aa020 2020-11-20T12:04:15.607 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:04:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d3c056d20 msgr2=0x7f1d3c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d3c056d20 0x7f1d3c0591e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f1d68065af0 tx=0x7f1d58008040).stop 2020-11-20T12:04:15.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d680aa020 msgr2=0x7f1d68137a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d680aa020 0x7f1d68137a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f1d5c00db60 tx=0x7f1d5c008040).stop 2020-11-20T12:04:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 shutdown_connections 2020-11-20T12:04:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d3c056d20 0x7f1d3c0591e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d6813ac60 0x7f1d680ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d680aa020 0x7f1d68137a10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 --2- 172.21.15.44:0/203125863 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d680aa400 0x7f1d680ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 >> 172.21.15.44:0/203125863 conn(0x7f1d68062690 msgr2=0x7f1d6812ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 shutdown_connections 2020-11-20T12:04:15.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.605+0000 7f1d6ceb8700 1 -- 172.21.15.44:0/203125863 wait complete. 2020-11-20T12:04:15.621 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:04:15.621 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:04:15.621 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:04:15.627 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2020-11-20T12:04:15.666 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2020-11-20T12:04:15.667 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2020-11-20T12:04:15.668 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2020-11-20T12:04:15.706 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2020-11-20T12:04:15.709 INFO:teuthology.orchestra.run.smithi044:> set -ex 2020-11-20T12:04:15.710 INFO:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2020-11-20T12:04:15.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.747+0000 7f5e1d077700 1 Processor -- start 2020-11-20T12:04:15.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.747+0000 7f5e1d077700 1 -- start start 2020-11-20T12:04:15.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e1d077700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5e180aa020 0x7f5e180a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e1d077700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5e180aa400 0x7f5e180a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e1d077700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e180a9770 con 0x7f5e1813ac60 2020-11-20T12:04:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e1d077700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180a6da0 con 0x7f5e180aa020 2020-11-20T12:04:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e1d077700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180a7040 con 0x7f5e180aa400 2020-11-20T12:04:15.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e16d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5e180aa020 0x7f5e180a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e16d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5e180aa020 0x7f5e180a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37182/0 (socket says 172.21.15.44:37182) 2020-11-20T12:04:15.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e16d9d700 1 -- 172.21.15.44:0/1119384726 learned_addr learned my addr 172.21.15.44:0/1119384726 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:15.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/1119384726 >> v2:172.21.15.79:3300/0 conn(0x7f5e180aa400 0x7f5e180a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e16d9d700 1 -- 172.21.15.44:0/1119384726 >> v2:172.21.15.79:3300/0 conn(0x7f5e180aa400 msgr2=0x7f5e180a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:15.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e16d9d700 1 --2- 172.21.15.44:0/1119384726 >> v2:172.21.15.79:3300/0 conn(0x7f5e180aa400 0x7f5e180a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e16d9d700 1 -- 172.21.15.44:0/1119384726 >> v1:172.21.15.79:6789/0 conn(0x7f5e1813ac60 legacy=0x7f5e180a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:15.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e16d9d700 1 -- 172.21.15.44:0/1119384726 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180a6a90 con 0x7f5e180aa020 2020-11-20T12:04:15.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.748+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/1119384726 >> v2:172.21.15.79:3300/0 conn(0x7f5e180aa400 0x7f5e180a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:04:15.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e16d9d700 1 --2- 172.21.15.44:0/1119384726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180a9280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f5e180a4990 tx=0x7f5e08006eb0).ready entity=mon.0 client_cookie=a689e902361c1898 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1659c700 1 -- 172.21.15.44:0/1119384726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08009890 con 0x7f5e180aa020 2020-11-20T12:04:15.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1659c700 1 -- 172.21.15.44:0/1119384726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e080099f0 con 0x7f5e180aa020 2020-11-20T12:04:15.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1659c700 1 -- 172.21.15.44:0/1119384726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08009d70 con 0x7f5e180aa020 2020-11-20T12:04:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1d077700 1 -- 172.21.15.44:0/1119384726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 msgr2=0x7f5e180a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1d077700 1 --2- 172.21.15.44:0/1119384726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180a9280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f5e180a4990 tx=0x7f5e08006eb0).stop 2020-11-20T12:04:15.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1d077700 1 -- 172.21.15.44:0/1119384726 shutdown_connections 2020-11-20T12:04:15.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1d077700 1 --2- 172.21.15.44:0/1119384726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180a9280 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1d077700 1 --2- 172.21.15.44:0/1119384726 >> v2:172.21.15.79:3300/0 conn(0x7f5e180aa400 0x7f5e180a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.749+0000 7f5e1d077700 1 -- 172.21.15.44:0/1119384726 >> 172.21.15.44:0/1119384726 conn(0x7f5e18062690 msgr2=0x7f5e180a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:15.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.750+0000 7f5e1d077700 1 -- 172.21.15.44:0/1119384726 shutdown_connections 2020-11-20T12:04:15.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.750+0000 7f5e1d077700 1 -- 172.21.15.44:0/1119384726 wait complete. 2020-11-20T12:04:15.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.750+0000 7f5e1d077700 1 Processor -- start 2020-11-20T12:04:15.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.750+0000 7f5e1d077700 1 -- start start 2020-11-20T12:04:15.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1d077700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1d077700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e180aa400 0x7f5e180a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1d077700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e1813ac60 0x7f5e181342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1d077700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1809dbf0 con 0x7f5e180aa020 2020-11-20T12:04:15.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1d077700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1809de90 con 0x7f5e180aa400 2020-11-20T12:04:15.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1d077700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1809e130 con 0x7f5e1813ac60 2020-11-20T12:04:15.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37188/0 (socket says 172.21.15.44:37188) 2020-11-20T12:04:15.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 -- 172.21.15.44:0/595911371 learned_addr learned my addr 172.21.15.44:0/595911371 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:15.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e180aa400 0x7f5e180a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1759e700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e1813ac60 0x7f5e181342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e1813ac60 msgr2=0x7f5e181342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e1813ac60 0x7f5e181342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e180aa400 msgr2=0x7f5e180a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e180aa400 0x7f5e180a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e16d9d700 1 -- 172.21.15.44:0/595911371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1809f610 con 0x7f5e180aa020 2020-11-20T12:04:15.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e180aa400 0x7f5e180a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:15.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.751+0000 7f5e1759e700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e1813ac60 0x7f5e181342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:15.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.752+0000 7f5e16d9d700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180abc30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f5e18061280 tx=0x7f5e08015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.752+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08011e50 con 0x7f5e180aa020 2020-11-20T12:04:15.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.752+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0801e460 con 0x7f5e180aa020 2020-11-20T12:04:15.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.752+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e180ac120 con 0x7f5e180aa020 2020-11-20T12:04:15.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.752+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0801e7e0 con 0x7f5e180aa020 2020-11-20T12:04:15.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.752+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e180ad9f0 con 0x7f5e180aa020 2020-11-20T12:04:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.753+0000 7f5e1d077700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1809ec60 0x7f5e1809e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.753+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1809e2f0 con 0x7f5e1809ec60 2020-11-20T12:04:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.753+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1809ec60 0x7f5e1809e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.753+0000 7f5e0ffff700 1 -- 172.21.15.44:0/595911371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e18061650 con 0x7f5e1809ec60 2020-11-20T12:04:15.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.753+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1809ec60 0x7f5e1809e4f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f5e18061050 tx=0x7f5e00008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.753+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000a890 con 0x7f5e1809ec60 2020-11-20T12:04:15.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.753+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f5e08025030 con 0x7f5e180aa020 2020-11-20T12:04:15.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e0effd700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5df4056df0 0x7f5df40592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5e08071760 con 0x7f5e180aa020 2020-11-20T12:04:15.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f5e0000a9f0 con 0x7f5e1809ec60 2020-11-20T12:04:15.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1809ec60 msgr2=0x7f5e1809e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e0effd700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1809ec60 0x7f5e1809e4f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f5e18061050 tx=0x7f5e00008040).stop 2020-11-20T12:04:15.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1809ec60 msgr2=0x7f5e1809e4f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:15.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e0effd700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e1809ec60 0x7f5e1809e4f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e1759e700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5df4056df0 0x7f5df40592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.754+0000 7f5e1759e700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5df4056df0 0x7f5df40592b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5e18142fa0 tx=0x7f5e10008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.787 INFO:teuthology.orchestra.run.smithi044.stderr:+ dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2020-11-20T12:04:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:9+1 records in 2020-11-20T12:04:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:9+1 records out 2020-11-20T12:04:15.788 INFO:teuthology.orchestra.run.smithi044.stderr:4744 bytes (4.7 kB, 4.6 KiB) copied, 4.8186e-05 s, 98.5 MB/s 2020-11-20T12:04:15.788 INFO:teuthology.orchestra.run.smithi044.stdout:rename/prepare.shrename/pri_rem.shrename/pri_nul.shrename/dir_pri_pri.shrename/pri_pri.shrename/rem_nul.shrename/dir_pri_nul.shrename/rem_pri.shrename/rem_rem.shrename/all.shcaps/mon_commands.shdirect_io/big.shdirect_io/misc.shdirect_io/test_short_dio_readdirect_io/direct_io_testdirect_io/test_sync_iokernel_untar_build.shlibcephfs/test.shcephtool/test_daemon.shcephtool/test.shcephtool/test_kvstore_tool.shmgr/test_localpool.shrgw/s3_bucket_quota.plrgw/test_rgw_orphan_list.shrgw/test_rgw_throttle.shrgw/test_rgw_gc_log.shrgw/run-s3tests.shrgw/s3_user_quota.plrgw/test_rgw_obj.shrgw/run-reshard.shrgw/test_rgw_reshard.pyrgw/s3_multipart_upload.plosdc/stress_objectcacher.shpost-file.shsuites/blogbench.shsuites/dbench.shsuites/fsx.shsuites/ffsb.shsuites/fsstress.shsuites/iogen.shsuites/cephfs_journal_tool_smoke.shsuites/fsync-tester.shsuites/dbench-short.shsuites/pjd.shsuites/bonnie.shsuites/iozone.shsuites/iozone-sync.shsuites/fio.shrados/stress_watch.shrados/test_cache_pool.shrados/clone.shrados/test_python.shrados/test_pool_access.shrados/test_crash.shrados/load-gen-mix.shrados/test_health_warnings.shrados/load-gen-mostlyread.shrados/test_pool_quota.shrados/load-gen-mix-small-long.shrados/load-gen-big.shrados/test_librados_build.shrados/test_hang.shrados/version_number_sanity.shrados/test_dedup_tool.shrados/test_envlibrados_for_rocksdb.shrados/test_large_omap_detection.pyrados/test_alloc_hint.shrados/test_rados_timeouts.shrados/load-gen-mix-small.shrados/test.shrados/test_rados_tool.shobjectstore/test_fuse.shcephadm/test_cephadm.shcephadm/create_iscsi_disks.shcephadm/test_adoption.shcephadm/test_dashboard_e2e.shcephadm/test_repos.shceph-helpers-root.sherasure-code/encode-decode-non-regression.sherasure-code/bench.shfs/test_o_truncfs/snap-hierarchy.shfs/norstats/kernel_untar_tar.shfs/test_python.shfs/misc/multiple_rsync.shfs/misc/layout_vxattrs.shfs/misc/chmod.shfs/misc/dirfrag.shfs/misc/xattrs.shfs/misc/filelock_interrupt.pyfs/misc/filelock_deadlock.pyfs/misc/rstats.shfs/misc/subvolume.shfs/misc/trivial_sync.shfs/misc/i_complete_vs_rename.shfs/misc/acl.shfs/misc/mkpool_layout_vxattrs.shfs/misc/direct_io.pyfs/quota/quota.shfs/upgrade/volume_clientfs/snaps/snaptest-double-null.shfs/snaps/snaptest-git-ceph.shfs/snaps/snaptest-2.shfs/snaps/snaptest-dir-rename.shfs/snaps/snaptest-0.shfs/snaps/snaptest-1.shfs/snaps/untar_snap_rm.shfs/snaps/snaptest-hardlink.shfs/snaps/snap-rm-diff.shfs/snaps/snaptest-authwb.shfs/snaps/snaptest-snap-rm-cmp.shfs/snaps/snaptest-intodir.shfs/snaps/snaptest-upchildrealms.shfs/snaps/snaptest-realm-split.shfs/snaps/snaptest-capwb.shfs/snaps/snaptest-parents.shfs/snaps/snaptest-xattrwb.shfs/snaps/snaptest-multiple-capsnaps.shfs/snaps/snaptest-estale.shfs/snaps/snaptest-snap-rename.shfs/test_o_trunc.shfs/multiclient_sync_read_eof.pycls/test_cls_rgw.shcls/test_cls_rbd.shcls/test_cls_hello.shcls/test_cls_sdk.shcls/test_cls_log.shcls/test_cls_lock.shcls/test_cls_cmpomap.shcls/test_cls_journal.shcls/test_cls_2pc_queue.shcls/test_cls_numops.shcls/test_cls_refcount.shrest/test_mgr_rest_api.pyrest/test-restful.shrbd/test_admin_socket.shrbd/test_rbdmap_RBDMAPFILE.shrbd/notify_master.shrbd/journal.shrbd/krbd_stable_pages_required.shrbd/rbd_mirror_snapshot.shrbd/krbd_namespaces.shrbd/krbd_udev_netlink_enobufs.shrbd/rbd_mirror_fsx_prepare.shrbd/krbd_huge_osdmap.shrbd/diff.shrbd/rbd_mirror_bootstrap.shrbd/simple_big.shrbd/krbd_fallocate.shrbd/test_librbd.shrbd/rbd_mirror_fsx_compare.shrbd/qemu-iotests.shrbd/krbd_udev_enumerate.shrbd/concurrent.shrbd/rbd-ggate.shrbd/krbd_latest_osdmap_on_map.shrbd/issue-20295.shrbd/import_export.shrbd/notify_slave.shrbd/cli_generic.shrbd/qemu_dynamic_features.shrbd/krbd_data_pool.shrbd/krbd_udev_netns.shrbd/rbd_mirror_ha.shrbd/rbd-nbd.shrbd/rbd_mirror_journal.shrbd/test_librbd_python.shrbd/read-flags.shrbd/krbd_exclusive_option.shrbd/huge-tickets.shrbd/map-snapshot-io.shrbd/image_read.shrbd/diff_continuous.shrbd/permissions.shrbd/map-unmap.shrbd/merge_diff.shrbd/test_rbd_tasks.shrbd/test_lock_fence.shrbd/rbd_mirror_helpers.shrbd/qos.shrbd/kernel.shrbd/krbd_wac.shrbd/rbd_mirror_stress.shrbd/qemu_rebuild_object_map.shrbd/krbd_udev_symlinks.shrbd/rbd_groups.shrbd/verify_pool.shrbd/test_rbd_mirror.shrestart/test-backtraces.pymon/pool_ops.shmon/test_mon_config_key.pymon/caps.shmon/rbd_snaps_ops.shmon/auth_caps.shmon/pg_autoscaler.shmon/test_config_key_caps.shmon/osd.shmon/config.shmon/ping.pymon/test_mon_osdmap_prune.shmon/crush_ops.shceph-tests/ceph-admin-commands.shtrue.shhadoop/terasort.shhadoop/repl.shhadoop/wordcount.sh 2020-11-20T12:04:15.789 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2020-11-20T12:04:15.790 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2020-11-20T12:04:15.791 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2020-11-20T12:04:15.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.762+0000 7f5e1d077700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e181347a0 0x7f5e1814b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.762+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1809dd30 con 0x7f5e181347a0 2020-11-20T12:04:15.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.762+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e181347a0 0x7f5e1814b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.762+0000 7f5e0ffff700 1 -- 172.21.15.44:0/595911371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5e180a9770 con 0x7f5e181347a0 2020-11-20T12:04:15.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.762+0000 7f5e0ffff700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e181347a0 0x7f5e1814b2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f5e00015000 tx=0x7f5e00008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00013730 con 0x7f5e181347a0 2020-11-20T12:04:15.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7f5e0000a680 con 0x7f5e181347a0 2020-11-20T12:04:15.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e181347a0 msgr2=0x7f5e1814b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e0effd700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e181347a0 0x7f5e1814b2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f5e00015000 tx=0x7f5e00008040).stop 2020-11-20T12:04:15.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e0effd700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e181347a0 msgr2=0x7f5e1814b2f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:15.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e0effd700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e181347a0 0x7f5e1814b2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5df4056df0 msgr2=0x7f5df40592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5df4056df0 0x7f5df40592b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f5e18142fa0 tx=0x7f5e10008040).stop 2020-11-20T12:04:15.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 msgr2=0x7f5e180abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 --2- 172.21.15.44:0/595911371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e180aa020 0x7f5e180abc30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f5e18061280 tx=0x7f5e08015000).stop 2020-11-20T12:04:15.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1759e700 1 -- 172.21.15.44:0/595911371 reap_dead start 2020-11-20T12:04:15.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1759e700 1 -- 172.21.15.44:0/595911371 reap_dead start 2020-11-20T12:04:15.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 shutdown_connections 2020-11-20T12:04:15.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 >> 172.21.15.44:0/595911371 conn(0x7f5e18062690 msgr2=0x7f5e18135fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:15.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 shutdown_connections 2020-11-20T12:04:15.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.763+0000 7f5e1d077700 1 -- 172.21.15.44:0/595911371 wait complete. 2020-11-20T12:04:15.796 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:15.796 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:15.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:15.797 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:04:15.797 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 10, 2020-11-20T12:04:15.797 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:15.797 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:15.798 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:15.798 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:15.798 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:15.799 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:04:15.799 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:15.799 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:15.799 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:15.800 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:15.800 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:15.800 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:15.800 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:15.801 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:15.801 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:15.801 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:15.801 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:15.802 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:15.802 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:15.802 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:15.803 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:15.803 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:15.803 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:15.803 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:15.804 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:15.804 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:15.804 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:15.804 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:15.804 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:15.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.805 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:15.805 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:15.805 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:15.806 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:15.806 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:15.806 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:15.806 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:15.806 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:15.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:15.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:15.807 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:15.807 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:15.807 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:15.808 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:15.808 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:15.808 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:15.808 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:15.808 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:15.809 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:15.809 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:15.809 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:15.809 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:15.809 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:15.810 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:15.810 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:15.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.810 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:15.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:15.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:15.811 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:15.811 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:15.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.812 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:15.812 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:15.812 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:15.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:15.813 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:15.813 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:15.814 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.814 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:15.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.815 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:15.815 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:15.816 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:15.816 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:15.816 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:15.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.818 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:15.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:15.819 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:15.819 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:15.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:15.820 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:15.820 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:15.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:15.822 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:15.822 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:15.823 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.823 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:15.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.824 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:15.824 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:15.825 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:15.825 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:15.825 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:15.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.827 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:15.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:15.827 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:15.828 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:15.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:15.829 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:15.829 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:15.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.830 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:15.831 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:15.831 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:15.831 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.832 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:15.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.832 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:15.833 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:15.833 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:15.834 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:15.834 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:15.834 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.835 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:15.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.835 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:15.836 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:15.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.836 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:15.836 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:15.837 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:15.837 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.837 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:15.837 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:04:15.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.838 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:15.838 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:15.838 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:15.838 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.839 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:15.839 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:04:15.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.839 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:15.839 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:15.840 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:04:15.840 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.840 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:15.840 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:15.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.841 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:15.841 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:15.841 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:04:15.841 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.842 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:15.842 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:04:15.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:15.842 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:15.842 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:15.843 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:04:15.843 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:15.843 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:15.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:15.843 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:15.844 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:15.844 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:04:15.874 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:04:15.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.971+0000 7f0bf016f700 1 Processor -- start 2020-11-20T12:04:15.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.971+0000 7f0bf016f700 1 -- start start 2020-11-20T12:04:15.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bf016f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0be80aad00 0x7f0be80a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bf016f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 0x7f0be813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bf016f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be81385f0 con 0x7f0be80a86d0 2020-11-20T12:04:15.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bf016f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be813e250 con 0x7f0be80aad00 2020-11-20T12:04:15.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bf016f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be813e4f0 con 0x7f0be80ab0e0 2020-11-20T12:04:15.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bedf0b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0be80aad00 0x7f0be80a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bee70c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0be80a86d0 0x7f0be80abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40950/0 (socket says 172.21.15.44:40950) 2020-11-20T12:04:15.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bee70c700 1 -- 172.21.15.44:0/1299724583 learned_addr learned my addr 172.21.15.44:0/1299724583 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:15.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bed70a700 1 --2- 172.21.15.44:0/1299724583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 0x7f0be813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bedf0b700 1 -- 172.21.15.44:0/1299724583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 msgr2=0x7f0be813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:15.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bedf0b700 1 --2- 172.21.15.44:0/1299724583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 0x7f0be813ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bedf0b700 1 -- 172.21.15.44:0/1299724583 >> v1:172.21.15.44:6789/0 conn(0x7f0be80a86d0 legacy=0x7f0be80abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bedf0b700 1 -- 172.21.15.44:0/1299724583 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80ac340 con 0x7f0be80aad00 2020-11-20T12:04:15.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0becf09700 1 -- 172.21.15.44:0/1299724583 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527107850 0 0) 0x7f0be81385f0 con 0x7f0be80a86d0 2020-11-20T12:04:15.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.972+0000 7f0bedf0b700 1 --2- 172.21.15.44:0/1299724583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a94d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f0be0009c90 tx=0x7f0be00091d0).ready entity=mon.0 client_cookie=6253ed9f5edcba10 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0becf09700 1 -- 172.21.15.44:0/1299724583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000bc50 con 0x7f0be80aad00 2020-11-20T12:04:15.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0becf09700 1 -- 172.21.15.44:0/1299724583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be000bdb0 con 0x7f0be80aad00 2020-11-20T12:04:15.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0becf09700 1 -- 172.21.15.44:0/1299724583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0011610 con 0x7f0be80aad00 2020-11-20T12:04:15.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 -- 172.21.15.44:0/1299724583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 msgr2=0x7f0be80a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 --2- 172.21.15.44:0/1299724583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a94d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f0be0009c90 tx=0x7f0be00091d0).stop 2020-11-20T12:04:15.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 -- 172.21.15.44:0/1299724583 shutdown_connections 2020-11-20T12:04:15.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 --2- 172.21.15.44:0/1299724583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a94d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 --2- 172.21.15.44:0/1299724583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 0x7f0be813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 -- 172.21.15.44:0/1299724583 >> 172.21.15.44:0/1299724583 conn(0x7f0be8060170 msgr2=0x7f0be8060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:15.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 -- 172.21.15.44:0/1299724583 shutdown_connections 2020-11-20T12:04:15.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.973+0000 7f0bf016f700 1 -- 172.21.15.44:0/1299724583 wait complete. 2020-11-20T12:04:15.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.974+0000 7f0bf016f700 1 Processor -- start 2020-11-20T12:04:15.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.974+0000 7f0bf016f700 1 -- start start 2020-11-20T12:04:15.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bf016f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be80a86d0 0x7f0be809e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bf016f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bf016f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 0x7f0be80a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bf016f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80a4840 con 0x7f0be80aad00 2020-11-20T12:04:15.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bf016f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80a7310 con 0x7f0be80a86d0 2020-11-20T12:04:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bf016f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80a75b0 con 0x7f0be80ab0e0 2020-11-20T12:04:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37212/0 (socket says 172.21.15.44:37212) 2020-11-20T12:04:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bedf0b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be80a86d0 0x7f0be809e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 -- 172.21.15.44:0/3846808002 learned_addr learned my addr 172.21.15.44:0/3846808002 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:15.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bee70c700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 0x7f0be80a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 msgr2=0x7f0be80a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0be80ab0e0 0x7f0be80a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be80a86d0 msgr2=0x7f0be809e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be80a86d0 0x7f0be809e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:15.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bed70a700 1 -- 172.21.15.44:0/3846808002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80a7850 con 0x7f0be80aad00 2020-11-20T12:04:15.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.975+0000 7f0bedf0b700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be80a86d0 0x7f0be809e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:15.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.976+0000 7f0bed70a700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a4350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0bdc00d8e0 tx=0x7f0bdc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.976+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc0125f0 con 0x7f0be80aad00 2020-11-20T12:04:15.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.976+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc012750 con 0x7f0be80aad00 2020-11-20T12:04:15.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.976+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be80a6ee0 con 0x7f0be80aad00 2020-11-20T12:04:15.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.976+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc012af0 con 0x7f0be80aad00 2020-11-20T12:04:15.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.976+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be80a65d0 con 0x7f0be80aad00 2020-11-20T12:04:15.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.977+0000 7f0bf016f700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be81334e0 0x7f0be81338c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.977+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be805c5f0 con 0x7f0be81334e0 2020-11-20T12:04:15.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.977+0000 7f0bedf0b700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be81334e0 0x7f0be81338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.977+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0bdc013480 con 0x7f0be80aad00 2020-11-20T12:04:15.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0bedf0b700 1 -- 172.21.15.44:0/3846808002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0be80a6850 con 0x7f0be81334e0 2020-11-20T12:04:15.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0bedf0b700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be81334e0 0x7f0be81338c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f0be0009210 tx=0x7f0be0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0bc0056d20 0x7f0bc00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:15.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0bdc070280 con 0x7f0be80aad00 2020-11-20T12:04:15.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0bee70c700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0bc0056d20 0x7f0bc00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:15.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be001dcc0 con 0x7f0be81334e0 2020-11-20T12:04:15.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f0be001de20 con 0x7f0be81334e0 2020-11-20T12:04:15.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be81334e0 msgr2=0x7f0be81338c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:15.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be81334e0 0x7f0be81338c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f0be0009210 tx=0x7f0be0016000).stop 2020-11-20T12:04:15.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be81334e0 msgr2=0x7f0be81338c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:15.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.978+0000 7f0be5ffb700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be81334e0 0x7f0be81338c0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0x7f0be0009210 tx=0x7f0be0016000).stop 2020-11-20T12:04:15.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.979+0000 7f0bee70c700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0bc0056d20 0x7f0bc00591e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0be809def0 tx=0x7f0bd8007980).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:15.999 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.973+0000 7fe819acc700 1 Processor -- start 2020-11-20T12:04:15.999 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.973+0000 7fe819acc700 1 -- start start 2020-11-20T12:04:16.000 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe819acc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe8140619f0 0x7fe8140aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.000 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe819acc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe81413b900 0x7fe814137e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.000 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe819acc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8140ab8f0 con 0x7fe814061dd0 2020-11-20T12:04:16.000 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe819acc700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81413af20 con 0x7fe81413b900 2020-11-20T12:04:16.000 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe819acc700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81413b1c0 con 0x7fe8140619f0 2020-11-20T12:04:16.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe813fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe81413b900 0x7fe814137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe813fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe81413b900 0x7fe814137e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37208/0 (socket says 172.21.15.44:37208) 2020-11-20T12:04:16.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe813fff700 1 -- 172.21.15.44:0/1870753742 learned_addr learned my addr 172.21.15.44:0/1870753742 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:16.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe8137fe700 1 --2- 172.21.15.44:0/1870753742 >> v2:172.21.15.79:3300/0 conn(0x7fe8140619f0 0x7fe8140aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe813fff700 1 -- 172.21.15.44:0/1870753742 >> v2:172.21.15.79:3300/0 conn(0x7fe8140619f0 msgr2=0x7fe8140aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:16.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe813fff700 1 --2- 172.21.15.44:0/1870753742 >> v2:172.21.15.79:3300/0 conn(0x7fe8140619f0 0x7fe8140aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe813fff700 1 -- 172.21.15.44:0/1870753742 >> v1:172.21.15.79:6789/0 conn(0x7fe814061dd0 legacy=0x7fe814065d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.974+0000 7fe813fff700 1 -- 172.21.15.44:0/1870753742 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81413b460 con 0x7fe81413b900 2020-11-20T12:04:16.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe813fff700 1 --2- 172.21.15.44:0/1870753742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe814137e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fe8140af090 tx=0x7fe80c009290).ready entity=mon.0 client_cookie=2bb069c3e71e8ec5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe8127fc700 1 -- 172.21.15.44:0/1870753742 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00bbc0 con 0x7fe81413b900 2020-11-20T12:04:16.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe8127fc700 1 -- 172.21.15.44:0/1870753742 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c00bd20 con 0x7fe81413b900 2020-11-20T12:04:16.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe8127fc700 1 -- 172.21.15.44:0/1870753742 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c011610 con 0x7fe81413b900 2020-11-20T12:04:16.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe819acc700 1 -- 172.21.15.44:0/1870753742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 msgr2=0x7fe814137e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe819acc700 1 --2- 172.21.15.44:0/1870753742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe814137e90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fe8140af090 tx=0x7fe80c009290).stop 2020-11-20T12:04:16.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe819acc700 1 -- 172.21.15.44:0/1870753742 shutdown_connections 2020-11-20T12:04:16.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe819acc700 1 --2- 172.21.15.44:0/1870753742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe814137e90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe819acc700 1 --2- 172.21.15.44:0/1870753742 >> v2:172.21.15.79:3300/0 conn(0x7fe8140619f0 0x7fe8140aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.975+0000 7fe819acc700 1 -- 172.21.15.44:0/1870753742 >> 172.21.15.44:0/1870753742 conn(0x7fe8140633d0 msgr2=0x7fe8140612e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:16.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.976+0000 7fe819acc700 1 -- 172.21.15.44:0/1870753742 shutdown_connections 2020-11-20T12:04:16.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.976+0000 7fe819acc700 1 -- 172.21.15.44:0/1870753742 wait complete. 2020-11-20T12:04:16.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.977+0000 7fe819acc700 1 Processor -- start 2020-11-20T12:04:16.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.977+0000 7fe819acc700 1 -- start start 2020-11-20T12:04:16.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.977+0000 7fe819acc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe8140a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.977+0000 7fe819acc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe8140a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.977+0000 7fe819acc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe81413e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.977+0000 7fe819acc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140a5510 con 0x7fe81413b900 2020-11-20T12:04:16.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.977+0000 7fe819acc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140a57b0 con 0x7fe814061dd0 2020-11-20T12:04:16.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe819acc700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140a7f50 con 0x7fe8140619f0 2020-11-20T12:04:16.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe81413e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe81413e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37222/0 (socket says 172.21.15.44:37222) 2020-11-20T12:04:16.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 -- 172.21.15.44:0/2455386321 learned_addr learned my addr 172.21.15.44:0/2455386321 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:16.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe8137fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe8140a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.008 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe812ffd700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe8140a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.008 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 msgr2=0x7fe8140a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.008 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe8140a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.008 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 msgr2=0x7fe8140a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.008 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe8140a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81413ecb0 con 0x7fe81413b900 2020-11-20T12:04:16.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe813fff700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe81413e7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fe814062230 tx=0x7fe80c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00bbc0 con 0x7fe81413b900 2020-11-20T12:04:16.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c00be90 con 0x7fe81413b900 2020-11-20T12:04:16.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81414d4e0 con 0x7fe81413b900 2020-11-20T12:04:16.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c010cd0 con 0x7fe81413b900 2020-11-20T12:04:16.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.978+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81414dbb0 con 0x7fe81413b900 2020-11-20T12:04:16.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.979+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8140aa830 con 0x7fe81413b900 2020-11-20T12:04:16.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.980+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe80c017070 con 0x7fe81413b900 2020-11-20T12:04:16.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.980+0000 7fe7f37fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe7ec056d20 0x7fe7ec0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.980+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe80c0721d0 con 0x7fe81413b900 2020-11-20T12:04:16.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.981+0000 7fe8137fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe7ec056d20 0x7fe7ec0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.981+0000 7fe8137fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe7ec056d20 0x7fe7ec0591e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fe814060360 tx=0x7fe804006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.014 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:15.984+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe80c0474c0 con 0x7fe81413b900 2020-11-20T12:04:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.987+0000 7f0bf016f700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be8149fd0 0x7f0be814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.987+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80a70d0 con 0x7f0be8149fd0 2020-11-20T12:04:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0bedf0b700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be8149fd0 0x7f0be814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0bedf0b700 1 -- 172.21.15.44:0/3846808002 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0be8069680 con 0x7f0be8149fd0 2020-11-20T12:04:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0bedf0b700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be8149fd0 0x7f0be814a3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0be805eef0 tx=0x7f0be001f040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be002c070 con 0x7f0be8149fd0 2020-11-20T12:04:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f0be001dab0 con 0x7f0be8149fd0 2020-11-20T12:04:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be8149fd0 msgr2=0x7f0be814a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0be5ffb700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be8149fd0 0x7f0be814a3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0be805eef0 tx=0x7f0be001f040).stop 2020-11-20T12:04:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0be5ffb700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be8149fd0 msgr2=0x7f0be814a3e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.988+0000 7f0be5ffb700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0be8149fd0 0x7f0be814a3e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0bc0056d20 msgr2=0x7f0bc00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0bc0056d20 0x7f0bc00591e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0be809def0 tx=0x7f0bd8007980).stop 2020-11-20T12:04:16.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 msgr2=0x7f0be80a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 --2- 172.21.15.44:0/3846808002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0be80aad00 0x7f0be80a4350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0bdc00d8e0 tx=0x7f0bdc008040).stop 2020-11-20T12:04:16.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bee70c700 1 -- 172.21.15.44:0/3846808002 reap_dead start 2020-11-20T12:04:16.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bee70c700 1 -- 172.21.15.44:0/3846808002 reap_dead start 2020-11-20T12:04:16.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 shutdown_connections 2020-11-20T12:04:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 >> 172.21.15.44:0/3846808002 conn(0x7f0be8060170 msgr2=0x7f0be8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 shutdown_connections 2020-11-20T12:04:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:15.989+0000 7f0bf016f700 1 -- 172.21.15.44:0/3846808002 wait complete. 2020-11-20T12:04:16.018 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:16.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:16.019 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:16.019 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:04:16.019 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 10, 2020-11-20T12:04:16.020 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:16.020 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:16.020 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:16.020 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:16.020 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.021 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 16, 2020-11-20T12:04:16.021 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:16.021 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:16.021 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:16.021 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:16.022 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:16.022 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:16.022 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:16.022 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:16.023 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:16.023 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:16.023 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:16.023 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.023 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:16.024 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:16.024 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:16.024 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:16.024 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:16.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:16.025 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:16.025 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:16.025 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:16.025 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:16.025 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.026 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:16.026 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:16.026 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:16.026 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:16.027 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:16.027 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:16.027 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:16.027 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:16.027 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:16.028 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:16.028 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:16.028 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:16.028 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:16.029 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:16.029 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:16.029 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:16.029 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:16.029 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:16.030 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:16.030 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:16.030 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:16.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:16.030 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:16.031 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.031 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:16.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.031 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:16.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.032 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:16.032 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:16.032 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:16.032 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:16.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:16.033 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:16.033 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.034 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:16.034 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:16.034 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.035 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.035 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:16.035 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:16.036 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:16.036 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:16.036 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:16.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.037 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:16.037 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:16.037 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:16.037 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:16.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:16.038 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:16.038 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.039 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:16.039 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:16.039 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.040 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.040 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:16.040 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:16.040 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:16.041 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:16.041 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:16.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.041 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:16.042 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:16.042 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:16.042 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:16.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:16.043 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:16.043 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:16.044 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:16.044 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.044 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.045 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:16.045 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:16.045 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:16.045 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:16.046 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:16.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.046 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.046 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:16.047 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:16.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.047 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:16.047 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:16.047 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:16.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.048 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:04:16.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.048 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:16.049 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:16.049 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:16.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.049 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.049 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:04:16.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.050 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:16.050 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:16.050 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:04:16.051 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.051 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.051 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:16.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.051 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:16.052 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:16.052 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:04:16.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.052 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.053 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:16.053 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:16.053 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:04:16.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f079a9b5700 1 Processor -- start 2020-11-20T12:04:16.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f079a9b5700 1 -- start start 2020-11-20T12:04:16.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f079a9b5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07940aa1e0 0x7f0794065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f079a9b5700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f07940a7810 0x7f0794066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f079a9b5700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940a8bf0 con 0x7f079413c170 2020-11-20T12:04:16.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f079a9b5700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0794066520 con 0x7f07940a7810 2020-11-20T12:04:16.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f079a9b5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07940a86b0 con 0x7f07940aa1e0 2020-11-20T12:04:16.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f07937fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f07940a7810 0x7f0794066030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f0798f52700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f079413c170 0x7f079413c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:40980/0 (socket says 172.21.15.44:40980) 2020-11-20T12:04:16.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.178+0000 7f0798f52700 1 -- 172.21.15.44:0/3511795199 learned_addr learned my addr 172.21.15.44:0/3511795199 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:16.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f0793fff700 1 --2- 172.21.15.44:0/3511795199 >> v2:172.21.15.79:3300/0 conn(0x7f07940aa1e0 0x7f0794065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f0792ffd700 1 -- 172.21.15.44:0/3511795199 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282224117 0 0) 0x7f07940a8bf0 con 0x7f079413c170 2020-11-20T12:04:16.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f0792ffd700 1 -- 172.21.15.44:0/3511795199 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0774003240 con 0x7f079413c170 2020-11-20T12:04:16.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f07937fe700 1 -- 172.21.15.44:0/3511795199 >> v2:172.21.15.79:3300/0 conn(0x7f07940aa1e0 msgr2=0x7f0794065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:16.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f07937fe700 1 --2- 172.21.15.44:0/3511795199 >> v2:172.21.15.79:3300/0 conn(0x7f07940aa1e0 0x7f0794065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f07937fe700 1 -- 172.21.15.44:0/3511795199 >> v1:172.21.15.44:6789/0 conn(0x7f079413c170 legacy=0x7f079413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f07937fe700 1 -- 172.21.15.44:0/3511795199 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940a8950 con 0x7f07940a7810 2020-11-20T12:04:16.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f07937fe700 1 --2- 172.21.15.44:0/3511795199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07940a7810 0x7f0794066030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f07940adec0 tx=0x7f0784006df0).ready entity=mon.0 client_cookie=612e28f9b651308c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f0792ffd700 1 -- 172.21.15.44:0/3511795199 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840096d0 con 0x7f07940a7810 2020-11-20T12:04:16.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f0792ffd700 1 -- 172.21.15.44:0/3511795199 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784009830 con 0x7f07940a7810 2020-11-20T12:04:16.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.179+0000 7f0792ffd700 1 -- 172.21.15.44:0/3511795199 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784009bb0 con 0x7f07940a7810 2020-11-20T12:04:16.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 -- 172.21.15.44:0/3511795199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07940a7810 msgr2=0x7f0794066030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 --2- 172.21.15.44:0/3511795199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07940a7810 0x7f0794066030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f07940adec0 tx=0x7f0784006df0).stop 2020-11-20T12:04:16.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 -- 172.21.15.44:0/3511795199 shutdown_connections 2020-11-20T12:04:16.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 --2- 172.21.15.44:0/3511795199 >> v2:172.21.15.79:3300/0 conn(0x7f07940aa1e0 0x7f0794065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 --2- 172.21.15.44:0/3511795199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07940a7810 0x7f0794066030 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 -- 172.21.15.44:0/3511795199 >> 172.21.15.44:0/3511795199 conn(0x7f0794062690 msgr2=0x7f07940a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:16.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 -- 172.21.15.44:0/3511795199 shutdown_connections 2020-11-20T12:04:16.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.180+0000 7f079a9b5700 1 -- 172.21.15.44:0/3511795199 wait complete. 2020-11-20T12:04:16.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 Processor -- start 2020-11-20T12:04:16.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 -- start start 2020-11-20T12:04:16.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07940a7810 0x7f079413b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940aa1e0 0x7f0794149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f079413c170 0x7f079414a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940a2a90 con 0x7f079413c170 2020-11-20T12:04:16.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079414a830 con 0x7f07940a7810 2020-11-20T12:04:16.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.181+0000 7f079a9b5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940a0490 con 0x7f07940aa1e0 2020-11-20T12:04:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f079413c170 0x7f079414a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f079413c170 0x7f079414a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37240/0 (socket says 172.21.15.44:37240) 2020-11-20T12:04:16.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 -- 172.21.15.44:0/1575567255 learned_addr learned my addr 172.21.15.44:0/1575567255 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:16.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f07937fe700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940aa1e0 0x7f0794149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0793fff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07940a7810 0x7f079413b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940aa1e0 msgr2=0x7f0794149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940aa1e0 0x7f0794149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07940a7810 msgr2=0x7f079413b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07940a7810 0x7f079413b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 -- 172.21.15.44:0/1575567255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940a15b0 con 0x7f079413c170 2020-11-20T12:04:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f0798f52700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f079413c170 0x7f079414a340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f078c001420 tx=0x7f078c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00ce10 con 0x7f079413c170 2020-11-20T12:04:16.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c005b80 con 0x7f079413c170 2020-11-20T12:04:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07940a1850 con 0x7f079413c170 2020-11-20T12:04:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c007110 con 0x7f079413c170 2020-11-20T12:04:16.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.182+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0794118eb0 con 0x7f079413c170 2020-11-20T12:04:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.183+0000 7f079a9b5700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f079413b8f0 0x7f07940a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.183+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0794061650 con 0x7f079413b8f0 2020-11-20T12:04:16.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.183+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f078c010030 con 0x7f079413c170 2020-11-20T12:04:16.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f0793fff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f079413b8f0 0x7f07940a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f078bfff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0770056d20 0x7f07700591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f078c06ff70 con 0x7f079413c170 2020-11-20T12:04:16.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f0793fff700 1 -- 172.21.15.44:0/1575567255 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07940a6780 con 0x7f079413b8f0 2020-11-20T12:04:16.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f07937fe700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0770056d20 0x7f07700591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f0793fff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f079413b8f0 0x7f07940a07f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f077c011c70 tx=0x7f077c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00fd30 con 0x7f079413b8f0 2020-11-20T12:04:16.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.184+0000 7f07937fe700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0770056d20 0x7f07700591e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0784006e30 tx=0x7f0784015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.185+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f077c00fe90 con 0x7f079413b8f0 2020-11-20T12:04:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.185+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f079413b8f0 msgr2=0x7f07940a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.185+0000 7f078bfff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f079413b8f0 0x7f07940a07f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f077c011c70 tx=0x7f077c006df0).stop 2020-11-20T12:04:16.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.185+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f079413b8f0 msgr2=0x7f07940a07f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.185+0000 7f078bfff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f079413b8f0 0x7f07940a07f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.192+0000 7f079a9b5700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940a0ce0 0x7f079414d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.192+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940446d0 con 0x7f07940a0ce0 2020-11-20T12:04:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.193+0000 7f0793fff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940a0ce0 0x7f079414d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.193+0000 7f0793fff700 1 -- 172.21.15.44:0/1575567255 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f07940613b0 con 0x7f07940a0ce0 2020-11-20T12:04:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.193+0000 7f0793fff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940a0ce0 0x7f079414d420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f077c000bc0 tx=0x7f077c01a000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.193+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00ba90 con 0x7f07940a0ce0 2020-11-20T12:04:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f077c027070 con 0x7f07940a0ce0 2020-11-20T12:04:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940a0ce0 msgr2=0x7f079414d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f078bfff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940a0ce0 0x7f079414d420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f077c000bc0 tx=0x7f077c01a000).stop 2020-11-20T12:04:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f078bfff700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940a0ce0 msgr2=0x7f079414d420 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f078bfff700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07940a0ce0 0x7f079414d420 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0770056d20 msgr2=0x7f07700591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0770056d20 0x7f07700591e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0784006e30 tx=0x7f0784015000).stop 2020-11-20T12:04:16.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f079413c170 msgr2=0x7f079414a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 --2- 172.21.15.44:0/1575567255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f079413c170 0x7f079414a340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f078c001420 tx=0x7f078c007e20).stop 2020-11-20T12:04:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f0798f52700 1 -- 172.21.15.44:0/1575567255 reap_dead start 2020-11-20T12:04:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f0798f52700 1 -- 172.21.15.44:0/1575567255 reap_dead start 2020-11-20T12:04:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 shutdown_connections 2020-11-20T12:04:16.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 >> 172.21.15.44:0/1575567255 conn(0x7f0794062690 msgr2=0x7f079413c600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 shutdown_connections 2020-11-20T12:04:16.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.194+0000 7f079a9b5700 1 -- 172.21.15.44:0/1575567255 wait complete. 2020-11-20T12:04:16.218 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:16.218 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:16.219 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:16.219 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:04:16.219 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 10, 2020-11-20T12:04:16.220 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:16.220 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:16.221 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:16.221 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:16.221 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.222 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 16, 2020-11-20T12:04:16.222 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:16.223 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:16.223 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:16.223 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:16.224 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:16.224 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:16.225 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:16.225 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:16.225 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:16.226 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:16.226 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:16.227 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.227 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:16.227 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:16.228 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:16.228 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:16.228 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:16.229 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:16.229 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:16.230 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:16.230 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:16.230 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:16.231 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.232 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:16.232 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:16.232 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:16.233 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:16.233 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:16.234 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:16.234 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:16.234 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:16.235 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:16.235 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:16.236 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:16.236 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:16.236 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:16.237 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:16.237 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:16.238 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:16.238 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:16.238 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:16.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:16.239 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:16.239 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:16.240 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:16.240 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:16.241 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.241 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:16.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:16.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.242 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:16.243 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:16.243 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:16.243 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:16.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:16.245 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:16.245 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:16.246 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:16.247 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.247 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.248 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.248 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:16.249 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:16.249 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:16.250 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:16.250 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:16.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.251 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:16.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:16.252 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:16.252 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:16.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:16.254 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:16.254 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:16.256 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:16.256 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.257 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.258 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:16.258 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:16.258 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:16.259 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:16.259 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:16.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.260 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:16.261 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:16.261 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:16.262 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:16.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:16.263 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:16.263 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.264 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:16.265 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:16.265 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:16.266 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.266 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.267 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:16.267 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:16.267 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:16.268 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:16.268 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:16.269 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.269 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.270 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:16.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:16.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.271 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:16.271 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:16.272 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:16.272 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.273 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:16.273 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:16.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:16.274 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:16.274 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:16.275 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:16.275 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:16.275 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:16.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:16.276 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:16.277 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:16.279 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:04:16.279 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:04:16.280 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:04:16.280 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:04:16.281 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:04:16.281 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:04:16.282 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.226+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe8140a5970 con 0x7fe81413b900 2020-11-20T12:04:16.284 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.280+0000 7fe813fff700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 msgr2=0x7fe81413e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:04:16.284 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.280+0000 7fe813fff700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 msgr2=0x7fe81413e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:16.285 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.280+0000 7fe813fff700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe81413e7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fe814062230 tx=0x7fe80c016000).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:16.285 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.280+0000 7fe813fff700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe81413e7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fe814062230 tx=0x7fe80c016000).stop 2020-11-20T12:04:16.285 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 msgr2=0x7fe81413e7c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:16.285 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81413b900 0x7fe81413e7c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe7ec060a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ec060f40 0x7fe7ec0633b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe7ec065a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec065f70 con 0x7fe7ec060f40 2020-11-20T12:04:16.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0660b0 con 0x7fe814061dd0 2020-11-20T12:04:16.287 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0661f0 con 0x7fe8140619f0 2020-11-20T12:04:16.287 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe813fff700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ec060f40 msgr2=0x7fe7ec0633b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:16.287 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe813fff700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ec060f40 0x7fe7ec0633b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:16.287 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.281+0000 7fe812ffd700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe7ec060a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.288 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe8137fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe7ec065a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.288 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe812ffd700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 msgr2=0x7fe7ec065a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.289 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe812ffd700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe7ec065a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.289 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe812ffd700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ec060f40 msgr2=0x7fe7ec0633b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:16.289 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe812ffd700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ec060f40 0x7fe7ec0633b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:16.289 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe812ffd700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7fe7ec066410 con 0x7fe8140619f0 2020-11-20T12:04:16.290 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe812ffd700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe8140ab8f0 con 0x7fe8140619f0 2020-11-20T12:04:16.290 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe8137fe700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe7ec065a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:04:16.290 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.282+0000 7fe812ffd700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe7ec060a50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe8140a5110 tx=0x7fe808007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.290 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.283+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808007240 con 0x7fe8140619f0 2020-11-20T12:04:16.290 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.283+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8080073a0 con 0x7fe8140619f0 2020-11-20T12:04:16.291 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:16.285+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe80800d5b0 con 0x7fe8140619f0 2020-11-20T12:04:16.293 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:04:16.293 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:04:16.293 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:04:16.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.454+0000 7f80e9c46700 1 Processor -- start 2020-11-20T12:04:16.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.454+0000 7f80e9c46700 1 -- start start 2020-11-20T12:04:16.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e9c46700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80e40aa020 0x7f80e40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e9c46700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f80e413ac60 0x7f80e40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e9c46700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e40a9770 con 0x7f80e40aa400 2020-11-20T12:04:16.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e9c46700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e40a56c0 con 0x7f80e413ac60 2020-11-20T12:04:16.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e9c46700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e40a6da0 con 0x7f80e40aa020 2020-11-20T12:04:16.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e3fff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f80e413ac60 msgr2=0x7f80e40a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:16.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e3fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f80e413ac60 0x7f80e40a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:16.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80e40aa020 0x7f80e40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e2ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f80e40aa400 0x7f80e40a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:43914/0 (socket says 172.21.15.44:43914) 2020-11-20T12:04:16.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80e40aa020 0x7f80e40a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57572/0 (socket says 172.21.15.44:57572) 2020-11-20T12:04:16.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e2ffd700 1 -- 172.21.15.44:0/2290710292 learned_addr learned my addr 172.21.15.44:0/2290710292 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:16.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e37fe700 1 -- 172.21.15.44:0/2290710292 >> v2:172.21.15.44:3300/0 conn(0x7f80e413ac60 msgr2=0x7f80e40a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:16.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e37fe700 1 --2- 172.21.15.44:0/2290710292 >> v2:172.21.15.44:3300/0 conn(0x7f80e413ac60 0x7f80e40a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:16.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e37fe700 1 -- 172.21.15.44:0/2290710292 >> v1:172.21.15.79:6789/0 conn(0x7f80e40aa400 legacy=0x7f80e40a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e37fe700 1 -- 172.21.15.44:0/2290710292 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40a7040 con 0x7f80e40aa020 2020-11-20T12:04:16.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.455+0000 7f80e27fc700 1 -- 172.21.15.44:0/2290710292 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191541730 0 0) 0x7f80d80010d0 con 0x7f80e40aa400 2020-11-20T12:04:16.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e37fe700 1 --2- 172.21.15.44:0/2290710292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa020 0x7f80e40a9280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f80e40adec0 tx=0x7f80d4006df0).ready entity=mon.1 client_cookie=d487403324f6123b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e27fc700 1 -- 172.21.15.44:0/2290710292 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d40097b0 con 0x7f80e40aa020 2020-11-20T12:04:16.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e27fc700 1 -- 172.21.15.44:0/2290710292 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d4009910 con 0x7f80e40aa020 2020-11-20T12:04:16.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e27fc700 1 -- 172.21.15.44:0/2290710292 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4009c90 con 0x7f80e40aa020 2020-11-20T12:04:16.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 -- 172.21.15.44:0/2290710292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa020 msgr2=0x7f80e40a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2290710292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa020 0x7f80e40a9280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f80e40adec0 tx=0x7f80d4006df0).stop 2020-11-20T12:04:16.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 -- 172.21.15.44:0/2290710292 shutdown_connections 2020-11-20T12:04:16.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2290710292 >> v2:172.21.15.44:3300/0 conn(0x7f80e413ac60 0x7f80e40a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:16.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2290710292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa020 0x7f80e40a9280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 -- 172.21.15.44:0/2290710292 >> 172.21.15.44:0/2290710292 conn(0x7f80e4062690 msgr2=0x7f80e40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:16.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 -- 172.21.15.44:0/2290710292 shutdown_connections 2020-11-20T12:04:16.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.456+0000 7f80e9c46700 1 -- 172.21.15.44:0/2290710292 wait complete. 2020-11-20T12:04:16.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.457+0000 7f80e9c46700 1 Processor -- start 2020-11-20T12:04:16.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.457+0000 7f80e9c46700 1 -- start start 2020-11-20T12:04:16.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e9c46700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e40aa020 0x7f80e4139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e9c46700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa400 0x7f80e409ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e9c46700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80e413ac60 0x7f80e409f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e9c46700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e414a0f0 con 0x7f80e40aa020 2020-11-20T12:04:16.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e9c46700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e409d5f0 con 0x7f80e40aa400 2020-11-20T12:04:16.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e9c46700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e409d890 con 0x7f80e413ac60 2020-11-20T12:04:16.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e37fe700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e40aa020 msgr2=0x7f80e4139cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:16.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e40aa020 0x7f80e4139cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:16.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa400 0x7f80e409ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e3fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80e413ac60 0x7f80e409f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa400 0x7f80e409ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57580/0 (socket says 172.21.15.44:57580) 2020-11-20T12:04:16.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.458+0000 7f80e2ffd700 1 -- 172.21.15.44:0/2094489512 learned_addr learned my addr 172.21.15.44:0/2094489512 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:16.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e2ffd700 1 -- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80e413ac60 msgr2=0x7f80e409f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e2ffd700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80e413ac60 0x7f80e409f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e2ffd700 1 -- 172.21.15.44:0/2094489512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e40aa020 msgr2=0x7f80e4139cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:16.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e2ffd700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e40aa020 0x7f80e4139cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:16.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e2ffd700 1 -- 172.21.15.44:0/2094489512 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e412b980 con 0x7f80e40aa400 2020-11-20T12:04:16.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e2ffd700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa400 0x7f80e409ec10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f80d800c760 tx=0x7f80d8007920).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80c37fe700 1 -- 172.21.15.44:0/2094489512 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80072e0 con 0x7f80e40aa400 2020-11-20T12:04:16.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80c37fe700 1 -- 172.21.15.44:0/2094489512 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8007440 con 0x7f80e40aa400 2020-11-20T12:04:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e40678c0 con 0x7f80e40aa400 2020-11-20T12:04:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80c37fe700 1 -- 172.21.15.44:0/2094489512 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8005800 con 0x7f80e40aa400 2020-11-20T12:04:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.459+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e4067230 con 0x7f80e40aa400 2020-11-20T12:04:16.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.460+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e4061650 con 0x7f80e40aa400 2020-11-20T12:04:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.460+0000 7f80c37fe700 1 -- 172.21.15.44:0/2094489512 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f80d801a060 con 0x7f80e40aa400 2020-11-20T12:04:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.460+0000 7f80c37fe700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80bc056cd0 0x7f80bc059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.460+0000 7f80c37fe700 1 -- 172.21.15.44:0/2094489512 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f80d806f710 con 0x7f80e40aa400 2020-11-20T12:04:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.461+0000 7f80e37fe700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80bc056cd0 0x7f80bc059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:16.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.461+0000 7f80e37fe700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80bc056cd0 0x7f80bc059190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f80e4067e00 tx=0x7f80d4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:16.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.465+0000 7f80c37fe700 1 -- 172.21.15.44:0/2094489512 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f80d8044a00 con 0x7f80e40aa400 2020-11-20T12:04:16.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.705+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f80e40a9770 con 0x7f80e40aa400 2020-11-20T12:04:16.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.706+0000 7f80c37fe700 1 -- 172.21.15.44:0/2094489512 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f80e40a9770 con 0x7f80e40aa400 2020-11-20T12:04:16.710 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:04:16.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80bc056cd0 msgr2=0x7f80bc059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80bc056cd0 0x7f80bc059190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f80e4067e00 tx=0x7f80d4015000).stop 2020-11-20T12:04:16.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa400 msgr2=0x7f80e409ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:16.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa400 0x7f80e409ec10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f80d800c760 tx=0x7f80d8007920).stop 2020-11-20T12:04:16.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 shutdown_connections 2020-11-20T12:04:16.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80bc056cd0 0x7f80bc059190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80e413ac60 0x7f80e409f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e40aa020 0x7f80e4139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:16.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 --2- 172.21.15.44:0/2094489512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80e40aa400 0x7f80e409ec10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:16.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 >> 172.21.15.44:0/2094489512 conn(0x7f80e4062690 msgr2=0x7f80e40a3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:16.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 shutdown_connections 2020-11-20T12:04:16.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:16.707+0000 7f80e9c46700 1 -- 172.21.15.44:0/2094489512 wait complete. 2020-11-20T12:04:16.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2020-11-20T12:04:19.723 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:04:19.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.858+0000 7f11c6d0d700 1 Processor -- start 2020-11-20T12:04:19.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.858+0000 7f11c6d0d700 1 -- start start 2020-11-20T12:04:19.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c6d0d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11c00aa400 0x7f11c00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:19.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c6d0d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c013ac60 0x7f11c00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:19.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c6d0d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c00a9770 con 0x7f11c00aa020 2020-11-20T12:04:19.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c6d0d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c00a56c0 con 0x7f11c00aa400 2020-11-20T12:04:19.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c6d0d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00a6da0 con 0x7f11c013ac60 2020-11-20T12:04:19.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c52aa700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c013ac60 0x7f11c00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:19.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c4aa9700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f11c00aa020 0x7f11c00a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:43928/0 (socket says 172.21.15.44:43928) 2020-11-20T12:04:19.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11c4aa9700 1 -- 172.21.15.44:0/4186346690 learned_addr learned my addr 172.21.15.44:0/4186346690 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:19.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.859+0000 7f11bffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11c00aa400 0x7f11c00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:19.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11bf7fe700 1 -- 172.21.15.44:0/4186346690 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289854050 0 0) 0x7f11b40010d0 con 0x7f11c00aa020 2020-11-20T12:04:19.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11bf7fe700 1 -- 172.21.15.44:0/4186346690 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a4003240 con 0x7f11c00aa020 2020-11-20T12:04:19.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11c52aa700 1 -- 172.21.15.44:0/4186346690 >> v2:172.21.15.79:3300/0 conn(0x7f11c00aa400 msgr2=0x7f11c00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:19.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11c52aa700 1 --2- 172.21.15.44:0/4186346690 >> v2:172.21.15.79:3300/0 conn(0x7f11c00aa400 0x7f11c00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:19.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11c52aa700 1 -- 172.21.15.44:0/4186346690 >> v1:172.21.15.79:6789/0 conn(0x7f11c00aa020 legacy=0x7f11c00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11c52aa700 1 -- 172.21.15.44:0/4186346690 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00a7040 con 0x7f11c013ac60 2020-11-20T12:04:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11bffff700 1 --2- 172.21.15.44:0/4186346690 >> v2:172.21.15.79:3300/0 conn(0x7f11c00aa400 0x7f11c00a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11c52aa700 1 --2- 172.21.15.44:0/4186346690 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c013ac60 0x7f11c00a3700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f11b8009c90 tx=0x7f11b80091d0).ready entity=mon.2 client_cookie=fe782cd10db0839 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11bf7fe700 1 -- 172.21.15.44:0/4186346690 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800b9c0 con 0x7f11c013ac60 2020-11-20T12:04:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11bf7fe700 1 -- 172.21.15.44:0/4186346690 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b800bb20 con 0x7f11c013ac60 2020-11-20T12:04:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.860+0000 7f11bf7fe700 1 -- 172.21.15.44:0/4186346690 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800bea0 con 0x7f11c013ac60 2020-11-20T12:04:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 -- 172.21.15.44:0/4186346690 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c013ac60 msgr2=0x7f11c00a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/4186346690 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c013ac60 0x7f11c00a3700 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f11b8009c90 tx=0x7f11b80091d0).stop 2020-11-20T12:04:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 -- 172.21.15.44:0/4186346690 shutdown_connections 2020-11-20T12:04:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/4186346690 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c013ac60 0x7f11c00a3700 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/4186346690 >> v2:172.21.15.79:3300/0 conn(0x7f11c00aa400 0x7f11c00a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 -- 172.21.15.44:0/4186346690 >> 172.21.15.44:0/4186346690 conn(0x7f11c0062690 msgr2=0x7f11c00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 -- 172.21.15.44:0/4186346690 shutdown_connections 2020-11-20T12:04:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.861+0000 7f11c6d0d700 1 -- 172.21.15.44:0/4186346690 wait complete. 2020-11-20T12:04:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 Processor -- start 2020-11-20T12:04:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 -- start start 2020-11-20T12:04:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11c00aa020 0x7f11c0137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c00aa400 0x7f11c00ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11c013ac60 0x7f11c00ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c012e870 con 0x7f11c00aa020 2020-11-20T12:04:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c012eb10 con 0x7f11c013ac60 2020-11-20T12:04:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.862+0000 7f11c6d0d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c012edb0 con 0x7f11c00aa400 2020-11-20T12:04:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11c4aa9700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11c00aa020 msgr2=0x7f11c0137f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11c4aa9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11c00aa020 0x7f11c0137f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c00aa400 0x7f11c00ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c00aa400 0x7f11c00ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48750/0 (socket says 172.21.15.44:48750) 2020-11-20T12:04:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 -- 172.21.15.44:0/967005489 learned_addr learned my addr 172.21.15.44:0/967005489 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11c52aa700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11c013ac60 0x7f11c00ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 -- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11c013ac60 msgr2=0x7f11c00ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11c013ac60 0x7f11c00ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 -- 172.21.15.44:0/967005489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11c00aa020 msgr2=0x7f11c0137f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11c00aa020 0x7f11c0137f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 -- 172.21.15.44:0/967005489 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c012e4f0 con 0x7f11c00aa400 2020-11-20T12:04:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bffff700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c00aa400 0x7f11c00ad580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f11ac00ef50 tx=0x7f11ac006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bcff9700 1 -- 172.21.15.44:0/967005489 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00cb30 con 0x7f11c00aa400 2020-11-20T12:04:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bcff9700 1 -- 172.21.15.44:0/967005489 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac00cc90 con 0x7f11c00aa400 2020-11-20T12:04:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c00ada70 con 0x7f11c00aa400 2020-11-20T12:04:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11bcff9700 1 -- 172.21.15.44:0/967005489 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac012600 con 0x7f11c00aa400 2020-11-20T12:04:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.863+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c00abe30 con 0x7f11c00aa400 2020-11-20T12:04:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.864+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c00a9530 con 0x7f11c00aa400 2020-11-20T12:04:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.865+0000 7f11bcff9700 1 -- 172.21.15.44:0/967005489 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f11ac012970 con 0x7f11c00aa400 2020-11-20T12:04:19.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.865+0000 7f11bcff9700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1198056cd0 0x7f1198059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:19.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.865+0000 7f11bcff9700 1 -- 172.21.15.44:0/967005489 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f11ac0706a0 con 0x7f11c00aa400 2020-11-20T12:04:19.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.865+0000 7f11c4aa9700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1198056cd0 0x7f1198059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:19.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.866+0000 7f11c4aa9700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1198056cd0 0x7f1198059190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f11b4004530 tx=0x7f11b4007610).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:19.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:19.870+0000 7f11bcff9700 1 -- 172.21.15.44:0/967005489 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f11ac045990 con 0x7f11c00aa400 2020-11-20T12:04:20.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.112+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f11c00a91a0 con 0x7f11c00aa400 2020-11-20T12:04:20.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.113+0000 7f11bcff9700 1 -- 172.21.15.44:0/967005489 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f11ac048fb0 con 0x7f11c00aa400 2020-11-20T12:04:20.119 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:04:20.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1198056cd0 msgr2=0x7f1198059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1198056cd0 0x7f1198059190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f11b4004530 tx=0x7f11b4007610).stop 2020-11-20T12:04:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c00aa400 msgr2=0x7f11c00ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c00aa400 0x7f11c00ad580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f11ac00ef50 tx=0x7f11ac006df0).stop 2020-11-20T12:04:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 shutdown_connections 2020-11-20T12:04:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1198056cd0 0x7f1198059190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:20.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11c013ac60 0x7f11c00ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:20.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11c00aa020 0x7f11c0137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:20.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 --2- 172.21.15.44:0/967005489 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f11c00aa400 0x7f11c00ad580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:20.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 >> 172.21.15.44:0/967005489 conn(0x7f11c0062690 msgr2=0x7f11c0138b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:20.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 shutdown_connections 2020-11-20T12:04:20.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:20.114+0000 7f11c6d0d700 1 -- 172.21.15.44:0/967005489 wait complete. 2020-11-20T12:04:23.130 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:04:23.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.273+0000 7f9cb2f3b700 1 Processor -- start 2020-11-20T12:04:23.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.273+0000 7f9cb2f3b700 1 -- start start 2020-11-20T12:04:23.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb2f3b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9cac0aa020 0x7f9cac0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb2f3b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9cac13ac60 0x7f9cac0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb2f3b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac0a9770 con 0x7f9cac0aa400 2020-11-20T12:04:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb2f3b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac0a56c0 con 0x7f9cac13ac60 2020-11-20T12:04:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb2f3b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac0a6da0 con 0x7f9cac0aa020 2020-11-20T12:04:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cabfff700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f9cac0aa400 legacy=0x7f9cac0a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:23.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb14d8700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f9cac13ac60 msgr2=0x7f9cac0a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:23.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb14d8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9cac13ac60 0x7f9cac0a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:23.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb0cd7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9cac0aa020 0x7f9cac0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:23.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb0cd7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9cac0aa020 0x7f9cac0a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57602/0 (socket says 172.21.15.44:57602) 2020-11-20T12:04:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb0cd7700 1 -- 172.21.15.44:0/89352380 learned_addr learned my addr 172.21.15.44:0/89352380 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:23.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb0cd7700 1 -- 172.21.15.44:0/89352380 >> v2:172.21.15.44:3300/0 conn(0x7f9cac13ac60 msgr2=0x7f9cac0a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:23.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb0cd7700 1 --2- 172.21.15.44:0/89352380 >> v2:172.21.15.44:3300/0 conn(0x7f9cac13ac60 0x7f9cac0a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:23.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb0cd7700 1 -- 172.21.15.44:0/89352380 >> v1:172.21.15.44:6789/0 conn(0x7f9cac0aa400 legacy=0x7f9cac0a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:23.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.274+0000 7f9cb0cd7700 1 -- 172.21.15.44:0/89352380 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac0a7040 con 0x7f9cac0aa020 2020-11-20T12:04:23.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb0cd7700 1 --2- 172.21.15.44:0/89352380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac0aa020 0x7f9cac0a9280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f9cac0adec0 tx=0x7f9c9c006eb0).ready entity=mon.1 client_cookie=6f86bd042fad69cc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:23.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cab7fe700 1 -- 172.21.15.44:0/89352380 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c009890 con 0x7f9cac0aa020 2020-11-20T12:04:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cab7fe700 1 -- 172.21.15.44:0/89352380 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c0099f0 con 0x7f9cac0aa020 2020-11-20T12:04:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cab7fe700 1 -- 172.21.15.44:0/89352380 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c009d70 con 0x7f9cac0aa020 2020-11-20T12:04:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/89352380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac0aa020 msgr2=0x7f9cac0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/89352380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac0aa020 0x7f9cac0a9280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f9cac0adec0 tx=0x7f9c9c006eb0).stop 2020-11-20T12:04:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/89352380 shutdown_connections 2020-11-20T12:04:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/89352380 >> v2:172.21.15.44:3300/0 conn(0x7f9cac13ac60 0x7f9cac0a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/89352380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac0aa020 0x7f9cac0a9280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/89352380 >> 172.21.15.44:0/89352380 conn(0x7f9cac062690 msgr2=0x7f9cac0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/89352380 shutdown_connections 2020-11-20T12:04:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.275+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/89352380 wait complete. 2020-11-20T12:04:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.276+0000 7f9cb2f3b700 1 Processor -- start 2020-11-20T12:04:23.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb2f3b700 1 -- start start 2020-11-20T12:04:23.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb2f3b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9cac0aa020 0x7f9cac146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb2f3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0aa400 0x7f9cac0ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb2f3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac13ac60 0x7f9cac144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb2f3b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0acb90 con 0x7f9cac0aa400 2020-11-20T12:04:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb2f3b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1473a0 con 0x7f9cac13ac60 2020-11-20T12:04:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb2f3b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1475c0 con 0x7f9cac0aa020 2020-11-20T12:04:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cabfff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0aa400 msgr2=0x7f9cac0ad590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cabfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0aa400 0x7f9cac0ad590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb14d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac13ac60 0x7f9cac144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb0cd7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9cac0aa020 0x7f9cac146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb14d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac13ac60 0x7f9cac144180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57610/0 (socket says 172.21.15.44:57610) 2020-11-20T12:04:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.277+0000 7f9cb14d8700 1 -- 172.21.15.44:0/2840562145 learned_addr learned my addr 172.21.15.44:0/2840562145 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb14d8700 1 -- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9cac0aa020 msgr2=0x7f9cac146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb14d8700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9cac0aa020 0x7f9cac146eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb14d8700 1 -- 172.21.15.44:0/2840562145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0aa400 msgr2=0x7f9cac0ad590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb14d8700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0aa400 0x7f9cac0ad590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb14d8700 1 -- 172.21.15.44:0/2840562145 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac147860 con 0x7f9cac13ac60 2020-11-20T12:04:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb0cd7700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9cac0aa020 0x7f9cac146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb14d8700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac13ac60 0x7f9cac144180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f9ca400bb10 tx=0x7f9ca4009120).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9ca8ff9700 1 -- 172.21.15.44:0/2840562145 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4003c60 con 0x7f9cac13ac60 2020-11-20T12:04:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9ca8ff9700 1 -- 172.21.15.44:0/2840562145 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4003dc0 con 0x7f9cac13ac60 2020-11-20T12:04:23.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac147b00 con 0x7f9cac13ac60 2020-11-20T12:04:23.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9ca8ff9700 1 -- 172.21.15.44:0/2840562145 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4006170 con 0x7f9cac13ac60 2020-11-20T12:04:23.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.278+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac1446f0 con 0x7f9cac13ac60 2020-11-20T12:04:23.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.279+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac147cc0 con 0x7f9cac13ac60 2020-11-20T12:04:23.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.280+0000 7f9ca8ff9700 1 -- 172.21.15.44:0/2840562145 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9ca4006520 con 0x7f9cac13ac60 2020-11-20T12:04:23.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.280+0000 7f9ca8ff9700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c84056cd0 0x7f9c84059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:23.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.280+0000 7f9ca8ff9700 1 -- 172.21.15.44:0/2840562145 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9ca4011020 con 0x7f9cac13ac60 2020-11-20T12:04:23.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.280+0000 7f9cb0cd7700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c84056cd0 0x7f9c84059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:23.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.281+0000 7f9cb0cd7700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c84056cd0 0x7f9c84059190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f9c9c006ef0 tx=0x7f9c9c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:23.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2020-11-20T12:04:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.284+0000 7f9ca8ff9700 1 -- 172.21.15.44:0/2840562145 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9ca4044da0 con 0x7f9cac13ac60 2020-11-20T12:04:23.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.523+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9cac0a7220 con 0x7f9cac13ac60 2020-11-20T12:04:23.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.524+0000 7f9ca8ff9700 1 -- 172.21.15.44:0/2840562145 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f9ca40483c0 con 0x7f9cac13ac60 2020-11-20T12:04:23.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c84056cd0 msgr2=0x7f9c84059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c84056cd0 0x7f9c84059190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f9c9c006ef0 tx=0x7f9c9c015000).stop 2020-11-20T12:04:23.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac13ac60 msgr2=0x7f9cac144180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:23.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac13ac60 0x7f9cac144180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f9ca400bb10 tx=0x7f9ca4009120).stop 2020-11-20T12:04:23.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 shutdown_connections 2020-11-20T12:04:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9c84056cd0 0x7f9c84059190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cac13ac60 0x7f9cac144180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9cac0aa020 0x7f9cac146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 --2- 172.21.15.44:0/2840562145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cac0aa400 0x7f9cac0ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.525+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 >> 172.21.15.44:0/2840562145 conn(0x7f9cac062690 msgr2=0x7f9cac0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.526+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 shutdown_connections 2020-11-20T12:04:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:23.526+0000 7f9cb2f3b700 1 -- 172.21.15.44:0/2840562145 wait complete. 2020-11-20T12:04:23.543 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:04:26.545 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:04:26.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.679+0000 7fbf7455e700 1 Processor -- start 2020-11-20T12:04:26.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.679+0000 7fbf7455e700 1 -- start start 2020-11-20T12:04:26.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf7455e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf6c13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf7455e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf7455e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c0a6260 con 0x7fbf6c060330 2020-11-20T12:04:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf7455e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0a52e0 con 0x7fbf6c13a990 2020-11-20T12:04:26.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf7455e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0a6490 con 0x7fbf6c13f800 2020-11-20T12:04:26.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf722fa700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fbf6c060330 legacy=0x7fbf6c0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:26.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf71af9700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf6c13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:26.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf71af9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf6c13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf72afb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf72afb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48774/0 (socket says 172.21.15.44:48774) 2020-11-20T12:04:26.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 learned_addr learned my addr 172.21.15.44:0/1417776720 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:04:26.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:26.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:26.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.680+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf6c13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.44:6789/0 conn(0x7fbf6c060330 legacy=0x7fbf6c0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf6c13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:04:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:26.881+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:04:27.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf6c13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.44:6789/0 conn(0x7fbf6c060330 legacy=0x7fbf6c0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf6c13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:04:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:27.282+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:04:28.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.083+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf6c13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.083+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.44:6789/0 conn(0x7fbf6c060330 legacy=0x7fbf6c0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:28.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.083+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf6c13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:04:28.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.083+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:28.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.084+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:28.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.084+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:28.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.084+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:28.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:28.084+0000 7fbf72afb700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:04:29.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 msgr2=0x7fbf6c141bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:29.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c13f800 0x7fbf6c141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:29.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf6c13ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:29.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf6c13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:29.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.44:6789/0 conn(0x7fbf6c060330 legacy=0x7fbf6c0a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:29.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:29.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c0a6730 con 0x7fbf4c005e10 2020-11-20T12:04:29.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0a52e0 con 0x7fbf4c003390 2020-11-20T12:04:29.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:29.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:29.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:29.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:29.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.680+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.882+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.882+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.882+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:29.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.882+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:29.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:29.882+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:04:30.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:30.283+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:30.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:30.283+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:30.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:30.283+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:30.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:30.283+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:30.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:30.283+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:04:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:31.084+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:31.084+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:31.084+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:31.084+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:31.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:31.084+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:04:32.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:32.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.79:6789/0 conn(0x7fbf4c005e10 legacy=0x7fbf4c0082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 reap_dead start 2020-11-20T12:04:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf4c009c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:32.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c0a6260 con 0x7fbf4c006270 2020-11-20T12:04:32.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0a52e0 con 0x7fbf6c13a990 2020-11-20T12:04:32.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0a6490 con 0x7fbf4c003390 2020-11-20T12:04:32.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf4c009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:32.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf4c009c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:32.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.680+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.681+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.681+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.681+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:32.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.681+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.881+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf4c009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.881+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf4c009c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:04:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.882+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.882+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.882+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.882+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:32.882+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:04:33.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:33.282+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf4c009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:33.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:33.282+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf4c009c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:04:33.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:33.283+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:33.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:33.283+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:33.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:33.283+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:33.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:33.283+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:33.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:33.283+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:04:34.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:34.083+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf4c009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:34.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:34.083+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf4c009c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:04:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:34.084+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:34.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:34.085+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:04:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:34.085+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:34.085+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:34.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:34.085+0000 7fbf722fa700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:04:35.650 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:04:35.656 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:04:35.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 msgr2=0x7fbf4c005810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:35.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.79:3300/0 conn(0x7fbf4c003390 0x7fbf4c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:35.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 msgr2=0x7fbf4c009c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:35.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> v2:172.21.15.44:3300/0 conn(0x7fbf6c13a990 0x7fbf4c009c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:35.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.680+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.79:6789/0 conn(0x7fbf4c006270 legacy=0x7fbf4c00a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:35.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.680+0000 7fbf5bfff700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 0x7fbf4c014f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:35.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c00a990 con 0x7fbf4c00efc0 2020-11-20T12:04:35.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf5bfff700 1 -- 172.21.15.44:0/1417776720 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0a52e0 con 0x7fbf4c012ad0 2020-11-20T12:04:35.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.44:6789/0 conn(0x7fbf4c00efc0 legacy=0x7fbf4c0113b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:35.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 0x7fbf4c014f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:35.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 msgr2=0x7fbf4c014f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:04:35.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 msgr2=0x7fbf4c014f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:35.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 0x7fbf4c014f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:35.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.681+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 0x7fbf4c014f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:35.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.882+0000 7fbf722fa700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.44:6789/0 conn(0x7fbf4c00efc0 legacy=0x7fbf4c0113b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:35.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.882+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 0x7fbf4c014f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:35.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.882+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 >> v1:172.21.15.44:6789/0 conn(0x7fbf4c00efc0 legacy=0x7fbf4c0113b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:35.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.882+0000 7fbf71af9700 1 -- 172.21.15.44:0/1417776720 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0166a0 con 0x7fbf4c012ad0 2020-11-20T12:04:35.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.882+0000 7fbf71af9700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 0x7fbf4c014f20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fbf680036a0 tx=0x7fbf68008040).ready entity=mon.2 client_cookie=91b47fe47ba1468d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:35.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.883+0000 7fbf712f8700 1 -- 172.21.15.44:0/1417776720 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68003ca0 con 0x7fbf4c012ad0 2020-11-20T12:04:35.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.883+0000 7fbf712f8700 1 -- 172.21.15.44:0/1417776720 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68003e00 con 0x7fbf4c012ad0 2020-11-20T12:04:35.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.883+0000 7fbf712f8700 1 -- 172.21.15.44:0/1417776720 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800dc00 con 0x7fbf4c012ad0 2020-11-20T12:04:35.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.883+0000 7fbf7455e700 1 -- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 msgr2=0x7fbf4c014f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:35.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.883+0000 7fbf7455e700 1 --2- 172.21.15.44:0/1417776720 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf4c012ad0 0x7fbf4c014f20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fbf680036a0 tx=0x7fbf68008040).stop 2020-11-20T12:04:35.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.883+0000 7fbf72afb700 1 -- 172.21.15.44:0/1417776720 reap_dead start 2020-11-20T12:04:35.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.884+0000 7fbf7455e700 1 -- 172.21.15.44:0/1417776720 shutdown_connections 2020-11-20T12:04:35.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.884+0000 7fbf7455e700 1 -- 172.21.15.44:0/1417776720 >> 172.21.15.44:0/1417776720 conn(0x7fbf6c062690 msgr2=0x7fbf6c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:35.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.884+0000 7fbf7455e700 1 -- 172.21.15.44:0/1417776720 shutdown_connections 2020-11-20T12:04:35.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.884+0000 7fbf7455e700 1 -- 172.21.15.44:0/1417776720 wait complete. 2020-11-20T12:04:35.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.885+0000 7fbf7455e700 1 Processor -- start 2020-11-20T12:04:35.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.885+0000 7fbf7455e700 1 -- start start 2020-11-20T12:04:35.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf7455e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c012ad0 0x7fbf6c0ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:35.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf7455e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c0ad5a0 0x7fbf6c0657a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:35.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf7455e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf6c137190 0x7fbf6c0ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:35.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf7455e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0abac0 con 0x7fbf4c012ad0 2020-11-20T12:04:35.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf7455e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0abd10 con 0x7fbf6c137190 2020-11-20T12:04:35.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf7455e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c136a50 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf722fa700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c012ad0 msgr2=0x7fbf6c0ac010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:35.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf722fa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c012ad0 0x7fbf6c0ac010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:35.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf71af9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c0ad5a0 0x7fbf6c0657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:35.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf72afb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf6c137190 0x7fbf6c0ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:35.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf72afb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf6c137190 0x7fbf6c0ab5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57678/0 (socket says 172.21.15.44:57678) 2020-11-20T12:04:35.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf71af9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c0ad5a0 0x7fbf6c0657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48832/0 (socket says 172.21.15.44:48832) 2020-11-20T12:04:35.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf72afb700 1 -- 172.21.15.44:0/3731774459 learned_addr learned my addr 172.21.15.44:0/3731774459 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:35.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf71af9700 1 -- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf6c137190 msgr2=0x7fbf6c0ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:35.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf71af9700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf6c137190 0x7fbf6c0ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:35.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf71af9700 1 -- 172.21.15.44:0/3731774459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c012ad0 msgr2=0x7fbf6c0ac010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:35.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf71af9700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c012ad0 0x7fbf6c0ac010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:35.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf71af9700 1 -- 172.21.15.44:0/3731774459 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c136cf0 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.886+0000 7fbf72afb700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf6c137190 0x7fbf6c0ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:35.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.887+0000 7fbf71af9700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c0ad5a0 0x7fbf6c0657a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fbf680036a0 tx=0x7fbf68008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:35.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.887+0000 7fbf5a7fc700 1 -- 172.21.15.44:0/3731774459 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68006820 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.887+0000 7fbf5a7fc700 1 -- 172.21.15.44:0/3731774459 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68006980 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.887+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c137610 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.887+0000 7fbf5a7fc700 1 -- 172.21.15.44:0/3731774459 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68021660 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.887+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c0acd60 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.888+0000 7fbf5a7fc700 1 -- 172.21.15.44:0/3731774459 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbf6801e070 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.888+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c061650 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.888+0000 7fbf5a7fc700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbf44056cd0 0x7fbf44059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:35.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.889+0000 7fbf5a7fc700 1 -- 172.21.15.44:0/3731774459 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbf68071590 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.889+0000 7fbf722fa700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbf44056cd0 0x7fbf44059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:35.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.889+0000 7fbf722fa700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbf44056cd0 0x7fbf44059190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fbf5c006fd0 tx=0x7fbf5c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:35.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:35.893+0000 7fbf5a7fc700 1 -- 172.21.15.44:0/3731774459 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbf68046880 con 0x7fbf6c0ad5a0 2020-11-20T12:04:35.904 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:35.900+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v8) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe808048020 con 0x7fe8140619f0 2020-11-20T12:04:36.087 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.083+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe814066420 con 0x7fe8140619f0 2020-11-20T12:04:36.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.142+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbf6c061650 con 0x7fbf6c0ad5a0 2020-11-20T12:04:36.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.142+0000 7fbf5a7fc700 1 -- 172.21.15.44:0/3731774459 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fbf6801b020 con 0x7fbf6c0ad5a0 2020-11-20T12:04:36.146 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":14,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:04:36.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbf44056cd0 msgr2=0x7fbf44059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbf44056cd0 0x7fbf44059190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fbf5c006fd0 tx=0x7fbf5c008040).stop 2020-11-20T12:04:36.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c0ad5a0 msgr2=0x7fbf6c0657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c0ad5a0 0x7fbf6c0657a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fbf680036a0 tx=0x7fbf68008040).stop 2020-11-20T12:04:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 shutdown_connections 2020-11-20T12:04:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbf44056cd0 0x7fbf44059190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf6c137190 0x7fbf6c0ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c012ad0 0x7fbf6c0ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:36.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 --2- 172.21.15.44:0/3731774459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbf6c0ad5a0 0x7fbf6c0657a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 >> 172.21.15.44:0/3731774459 conn(0x7fbf6c062690 msgr2=0x7fbf6c139260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:36.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 shutdown_connections 2020-11-20T12:04:36.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.144+0000 7fbf7455e700 1 -- 172.21.15.44:0/3731774459 wait complete. 2020-11-20T12:04:36.159 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:04:36.160 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:04:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.285+0000 7fb0ed2a9700 1 Processor -- start 2020-11-20T12:04:36.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.285+0000 7fb0ed2a9700 1 -- start start 2020-11-20T12:04:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.285+0000 7fb0ed2a9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e80aa020 0x7fb0e80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.285+0000 7fb0ed2a9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb0e813ac60 0x7fb0e80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.285+0000 7fb0ed2a9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e80a9770 con 0x7fb0e80aa400 2020-11-20T12:04:36.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.285+0000 7fb0ed2a9700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e80a56c0 con 0x7fb0e813ac60 2020-11-20T12:04:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.285+0000 7fb0ed2a9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80a6da0 con 0x7fb0e80aa020 2020-11-20T12:04:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e77fe700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fb0e813ac60 msgr2=0x7fb0e80a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e77fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb0e813ac60 0x7fb0e80a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e80aa020 0x7fb0e80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e80aa020 0x7fb0e80a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48842/0 (socket says 172.21.15.44:48842) 2020-11-20T12:04:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 -- 172.21.15.44:0/1831437724 learned_addr learned my addr 172.21.15.44:0/1831437724 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e67fc700 1 --1- 172.21.15.44:0/1831437724 >> v1:172.21.15.79:6789/0 conn(0x7fb0e80aa400 0x7fb0e80a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44024/0 (socket says 172.21.15.44:44024) 2020-11-20T12:04:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e5ffb700 1 -- 172.21.15.44:0/1831437724 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724664227 0 0) 0x7fb0d8001450 con 0x7fb0e80aa400 2020-11-20T12:04:36.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 -- 172.21.15.44:0/1831437724 >> v2:172.21.15.44:3300/0 conn(0x7fb0e813ac60 msgr2=0x7fb0e80a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 --2- 172.21.15.44:0/1831437724 >> v2:172.21.15.44:3300/0 conn(0x7fb0e813ac60 0x7fb0e80a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 -- 172.21.15.44:0/1831437724 >> v1:172.21.15.79:6789/0 conn(0x7fb0e80aa400 legacy=0x7fb0e80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 -- 172.21.15.44:0/1831437724 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80a7040 con 0x7fb0e80aa020 2020-11-20T12:04:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.286+0000 7fb0e6ffd700 1 --2- 172.21.15.44:0/1831437724 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e80aa020 0x7fb0e80a9280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb0dc006fd0 tx=0x7fb0dc006eb0).ready entity=mon.2 client_cookie=6bd2d27c8ca84455 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0e5ffb700 1 -- 172.21.15.44:0/1831437724 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0098d0 con 0x7fb0e80aa020 2020-11-20T12:04:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0e5ffb700 1 -- 172.21.15.44:0/1831437724 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc009a30 con 0x7fb0e80aa020 2020-11-20T12:04:36.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0e5ffb700 1 -- 172.21.15.44:0/1831437724 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc009db0 con 0x7fb0e80aa020 2020-11-20T12:04:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/1831437724 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e80aa020 msgr2=0x7fb0e80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 --2- 172.21.15.44:0/1831437724 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e80aa020 0x7fb0e80a9280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb0dc006fd0 tx=0x7fb0dc006eb0).stop 2020-11-20T12:04:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/1831437724 shutdown_connections 2020-11-20T12:04:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 --2- 172.21.15.44:0/1831437724 >> v2:172.21.15.44:3300/0 conn(0x7fb0e813ac60 0x7fb0e80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:36.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 --2- 172.21.15.44:0/1831437724 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e80aa020 0x7fb0e80a9280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/1831437724 >> 172.21.15.44:0/1831437724 conn(0x7fb0e8062690 msgr2=0x7fb0e80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:36.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/1831437724 shutdown_connections 2020-11-20T12:04:36.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.287+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/1831437724 wait complete. 2020-11-20T12:04:36.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.288+0000 7fb0ed2a9700 1 Processor -- start 2020-11-20T12:04:36.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.288+0000 7fb0ed2a9700 1 -- start start 2020-11-20T12:04:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0ed2a9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0e80aa020 0x7fb0e80ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0ed2a9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e80aa400 0x7fb0e8139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0ed2a9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e813ac60 0x7fb0e8065e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0ed2a9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8067680 con 0x7fb0e80aa020 2020-11-20T12:04:36.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0ed2a9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80aba60 con 0x7fb0e80aa400 2020-11-20T12:04:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0ed2a9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e81490c0 con 0x7fb0e813ac60 2020-11-20T12:04:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e6ffd700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0e80aa020 msgr2=0x7fb0e80ab1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0e80aa020 0x7fb0e80ab1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e80aa400 0x7fb0e8139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e77fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e813ac60 0x7fb0e8065e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e77fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e813ac60 0x7fb0e8065e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48848/0 (socket says 172.21.15.44:48848) 2020-11-20T12:04:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e77fe700 1 -- 172.21.15.44:0/2173141137 learned_addr learned my addr 172.21.15.44:0/2173141137 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e80aa400 0x7fb0e8139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57690/0 (socket says 172.21.15.44:57690) 2020-11-20T12:04:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e813ac60 msgr2=0x7fb0e8065e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e813ac60 0x7fb0e8065e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0e80aa020 msgr2=0x7fb0e80ab1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:36.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0e80aa020 0x7fb0e80ab1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:36.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 -- 172.21.15.44:0/2173141137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80ab7c0 con 0x7fb0e80aa400 2020-11-20T12:04:36.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e77fe700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb0e813ac60 0x7fb0e8065e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:36.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.289+0000 7fb0e67fc700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e80aa400 0x7fb0e8139ea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb0e8061320 tx=0x7fb0d8007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.290+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8005ac0 con 0x7fb0e80aa400 2020-11-20T12:04:36.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.290+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8005c20 con 0x7fb0e80aa400 2020-11-20T12:04:36.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.290+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e80659f0 con 0x7fb0e80aa400 2020-11-20T12:04:36.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.290+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8007110 con 0x7fb0e80aa400 2020-11-20T12:04:36.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.290+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e8066a00 con 0x7fb0e80aa400 2020-11-20T12:04:36.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.290+0000 7fb0ed2a9700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8066380 0x7fb0e8149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.290+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8066cb0 con 0x7fb0e8066380 2020-11-20T12:04:36.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0e6ffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8066380 0x7fb0e8149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb0d8010030 con 0x7fb0e80aa400 2020-11-20T12:04:36.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0e6ffd700 1 -- 172.21.15.44:0/2173141137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0e8061650 con 0x7fb0e8066380 2020-11-20T12:04:36.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0e6ffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8066380 0x7fb0e8149710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fb0dc006fd0 tx=0x7fb0dc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0ceffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0c4056cd0 0x7fb0c4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb0d806ff80 con 0x7fb0e80aa400 2020-11-20T12:04:36.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011e50 con 0x7fb0e8066380 2020-11-20T12:04:36.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.291+0000 7fb0e77fe700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0c4056cd0 0x7fb0c4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.292+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb0dc01e460 con 0x7fb0e8066380 2020-11-20T12:04:36.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.292+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8066380 msgr2=0x7fb0e8149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.292+0000 7fb0ceffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8066380 0x7fb0e8149710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fb0dc006fd0 tx=0x7fb0dc015000).stop 2020-11-20T12:04:36.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.292+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8066380 msgr2=0x7fb0e8149710 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:36.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.292+0000 7fb0ceffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8066380 0x7fb0e8149710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.292+0000 7fb0e77fe700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0c4056cd0 0x7fb0c4059190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fb0e0003800 tx=0x7fb0e0006cf0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.299+0000 7fb0ed2a9700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8149c00 0x7fb0e813bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.299+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80a9770 con 0x7fb0e8149c00 2020-11-20T12:04:36.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.300+0000 7fb0e6ffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8149c00 0x7fb0e813bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.300+0000 7fb0e6ffd700 1 -- 172.21.15.44:0/2173141137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb0e8149fe0 con 0x7fb0e8149c00 2020-11-20T12:04:36.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.300+0000 7fb0e6ffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8149c00 0x7fb0e813bd90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb0e8066790 tx=0x7fb0dc01e000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.300+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc01db60 con 0x7fb0e8149c00 2020-11-20T12:04:36.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4766 (secure 0 0 0) 0x7fb0dc011c40 con 0x7fb0e8149c00 2020-11-20T12:04:36.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8149c00 msgr2=0x7fb0e813bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ceffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8149c00 0x7fb0e813bd90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb0e8066790 tx=0x7fb0dc01e000).stop 2020-11-20T12:04:36.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ceffd700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8149c00 msgr2=0x7fb0e813bd90 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:36.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ceffd700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e8149c00 0x7fb0e813bd90 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0x7fb0e8066790 tx=0x7fb0dc01e000).stop 2020-11-20T12:04:36.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0c4056cd0 msgr2=0x7fb0c4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0c4056cd0 0x7fb0c4059190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fb0e0003800 tx=0x7fb0e0006cf0).stop 2020-11-20T12:04:36.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e80aa400 msgr2=0x7fb0e8139ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 --2- 172.21.15.44:0/2173141137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0e80aa400 0x7fb0e8139ea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb0e8061320 tx=0x7fb0d8007c20).stop 2020-11-20T12:04:36.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0e77fe700 1 -- 172.21.15.44:0/2173141137 reap_dead start 2020-11-20T12:04:36.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0e77fe700 1 -- 172.21.15.44:0/2173141137 reap_dead start 2020-11-20T12:04:36.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 shutdown_connections 2020-11-20T12:04:36.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 >> 172.21.15.44:0/2173141137 conn(0x7fb0e8062690 msgr2=0x7fb0e80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:36.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 shutdown_connections 2020-11-20T12:04:36.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.301+0000 7fb0ed2a9700 1 -- 172.21.15.44:0/2173141137 wait complete. 2020-11-20T12:04:36.320 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:36.321 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:36.321 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:36.321 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:04:36.321 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 14, 2020-11-20T12:04:36.321 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:36.322 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:36.322 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:36.322 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.323 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:04:36.323 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:36.323 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:36.323 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:36.324 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:36.324 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:36.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:36.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:36.324 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:36.325 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:36.325 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:36.325 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:36.325 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.325 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:36.326 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:36.326 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:36.326 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:36.326 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:36.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:36.327 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:36.327 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:36.327 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:36.327 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:36.327 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.328 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:36.328 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:36.328 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:36.328 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:36.329 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:36.329 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:36.329 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:36.329 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:36.329 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:36.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:36.330 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:36.330 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:36.330 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:36.330 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:36.331 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:36.331 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:36.331 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:36.331 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:36.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:36.332 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:36.332 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:36.332 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:36.332 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:36.332 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.333 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:36.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.333 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:36.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.333 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:36.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:36.334 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:36.334 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:36.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:36.335 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:36.335 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:36.336 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:36.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.336 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.337 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:36.337 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:36.337 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:36.337 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:36.338 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:36.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.338 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:36.338 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:36.339 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:36.339 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:36.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:36.339 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:36.340 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:36.340 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:36.341 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.341 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.341 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:36.342 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:36.342 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:36.342 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:36.342 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:36.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.343 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:36.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:36.343 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:36.343 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:36.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:36.344 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:36.344 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:36.345 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:36.345 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.346 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.346 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:36.346 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:36.346 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:36.347 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:36.347 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:36.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.347 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.348 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:36.348 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:36.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.348 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.348 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.349 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:36.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.349 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.349 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:04:36.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.350 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.350 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.350 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:04:36.350 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.350 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.351 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:04:36.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.351 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.351 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.351 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 8 2020-11-20T12:04:36.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.352 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:36.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.352 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.353 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.353 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:04:36.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.353 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.353 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:04:36.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.354 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.354 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.354 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:36.354 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.355 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.355 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:36.355 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:36.356 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:04:36.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.480+0000 7f2200024700 1 Processor -- start 2020-11-20T12:04:36.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.480+0000 7f2200024700 1 -- start start 2020-11-20T12:04:36.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f2200024700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f21f80aa020 0x7f21f80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f2200024700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80a9770 con 0x7f21f80aa400 2020-11-20T12:04:36.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f2200024700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80a6da0 con 0x7f21f813ac60 2020-11-20T12:04:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f2200024700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80a56c0 con 0x7f21f80aa020 2020-11-20T12:04:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f21fd5bf700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f21f80aa400 legacy=0x7f21f80a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:04:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f21fddc0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f21f80aa020 0x7f21f80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f21fddc0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f21f80aa020 0x7f21f80a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57700/0 (socket says 172.21.15.44:57700) 2020-11-20T12:04:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f21fe5c1700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f21f813ac60 0x7f21f80a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44046/0 (socket says 172.21.15.44:44046) 2020-11-20T12:04:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.481+0000 7f21fddc0700 1 -- 172.21.15.44:0/1991387736 learned_addr learned my addr 172.21.15.44:0/1991387736 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f21fddc0700 1 -- 172.21.15.44:0/1991387736 >> v1:172.21.15.79:6789/0 conn(0x7f21f813ac60 legacy=0x7f21f80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f21fddc0700 1 -- 172.21.15.44:0/1991387736 >> v1:172.21.15.44:6789/0 conn(0x7f21f80aa400 legacy=0x7f21f80a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:36.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f21fddc0700 1 -- 172.21.15.44:0/1991387736 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80a7040 con 0x7f21f80aa020 2020-11-20T12:04:36.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f21fddc0700 1 --2- 172.21.15.44:0/1991387736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f80aa020 0x7f21f80a9280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f21e8006fd0 tx=0x7f21e8006eb0).ready entity=mon.1 client_cookie=756f37dc5bbdbbf1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f21fcdbe700 1 -- 172.21.15.44:0/1991387736 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8009890 con 0x7f21f80aa020 2020-11-20T12:04:36.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f21fcdbe700 1 -- 172.21.15.44:0/1991387736 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e80099f0 con 0x7f21f80aa020 2020-11-20T12:04:36.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f21fcdbe700 1 -- 172.21.15.44:0/1991387736 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8009d70 con 0x7f21f80aa020 2020-11-20T12:04:36.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f2200024700 1 -- 172.21.15.44:0/1991387736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f80aa020 msgr2=0x7f21f80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f2200024700 1 --2- 172.21.15.44:0/1991387736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f80aa020 0x7f21f80a9280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f21e8006fd0 tx=0x7f21e8006eb0).stop 2020-11-20T12:04:36.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f2200024700 1 -- 172.21.15.44:0/1991387736 shutdown_connections 2020-11-20T12:04:36.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f2200024700 1 --2- 172.21.15.44:0/1991387736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f80aa020 0x7f21f80a9280 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.482+0000 7f2200024700 1 -- 172.21.15.44:0/1991387736 >> 172.21.15.44:0/1991387736 conn(0x7f21f8062690 msgr2=0x7f21f80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:36.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.483+0000 7f2200024700 1 -- 172.21.15.44:0/1991387736 shutdown_connections 2020-11-20T12:04:36.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.483+0000 7f2200024700 1 -- 172.21.15.44:0/1991387736 wait complete. 2020-11-20T12:04:36.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.483+0000 7f2200024700 1 Processor -- start 2020-11-20T12:04:36.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f2200024700 1 -- start start 2020-11-20T12:04:36.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f2200024700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21f80aa020 0x7f21f80ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f2200024700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f80aa400 0x7f21f80a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f2200024700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f813ac60 0x7f21f80678c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f21fddc0700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21f80aa020 msgr2=0x7f21f80ada30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:04:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f2200024700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80ab380 con 0x7f21f80aa020 2020-11-20T12:04:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f21fddc0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21f80aa020 0x7f21f80ada30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f2200024700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80ab620 con 0x7f21f813ac60 2020-11-20T12:04:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f2200024700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80ad590 con 0x7f21f80aa400 2020-11-20T12:04:36.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f21fe5c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f813ac60 0x7f21f80678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f21fd5bf700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f80aa400 0x7f21f80a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f21fe5c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f813ac60 0x7f21f80678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:57710/0 (socket says 172.21.15.44:57710) 2020-11-20T12:04:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f21fd5bf700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f80aa400 0x7f21f80a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:48864/0 (socket says 172.21.15.44:48864) 2020-11-20T12:04:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.484+0000 7f21fe5c1700 1 -- 172.21.15.44:0/257510023 learned_addr learned my addr 172.21.15.44:0/257510023 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:36.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21fd5bf700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f813ac60 msgr2=0x7f21f80678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21fd5bf700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f813ac60 0x7f21f80678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21fd5bf700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21f80aa020 msgr2=0x7f21f80ada30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:36.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21fd5bf700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f21f80aa020 0x7f21f80ada30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:36.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21fd5bf700 1 -- 172.21.15.44:0/257510023 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f8135300 con 0x7f21f80aa400 2020-11-20T12:04:36.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21fe5c1700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21f813ac60 0x7f21f80678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21fd5bf700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f80aa400 0x7f21f80a2280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f21f40096b0 tx=0x7f21f4006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4006170 con 0x7f21f80aa400 2020-11-20T12:04:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f40062f0 con 0x7f21f80aa400 2020-11-20T12:04:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f80aae70 con 0x7f21f80aa400 2020-11-20T12:04:36.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f40038e0 con 0x7f21f80aa400 2020-11-20T12:04:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.485+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f80ad190 con 0x7f21f80aa400 2020-11-20T12:04:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.486+0000 7f2200024700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8131650 0x7f21f8131a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.486+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80ab4c0 con 0x7f21f8131650 2020-11-20T12:04:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.486+0000 7f21fddc0700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8131650 0x7f21f8131a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.486+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f21f401a060 con 0x7f21f80aa400 2020-11-20T12:04:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21d0056cd0 0x7f21d0059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21fddc0700 1 -- 172.21.15.44:0/257510023 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21f8131f20 con 0x7f21f8131650 2020-11-20T12:04:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f21f406f9a0 con 0x7f21f80aa400 2020-11-20T12:04:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21fddc0700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8131650 0x7f21f8131a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f21e8006fd0 tx=0x7f21e8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21fe5c1700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21d0056cd0 0x7f21d0059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8011dd0 con 0x7f21f8131650 2020-11-20T12:04:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21fe5c1700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21d0056cd0 0x7f21d0059190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f21f0001b10 tx=0x7f21f0001360).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f21e801e460 con 0x7f21f8131650 2020-11-20T12:04:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8131650 msgr2=0x7f21f8131a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8131650 0x7f21f8131a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f21e8006fd0 tx=0x7f21e8015000).stop 2020-11-20T12:04:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8131650 msgr2=0x7f21f8131a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:36.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.487+0000 7f21e5ffb700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8131650 0x7f21f8131a30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.495+0000 7f2200024700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8060170 0x7f21f8060580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:36.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.495+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80ab1a0 con 0x7f21f8060170 2020-11-20T12:04:36.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.495+0000 7f21fddc0700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8060170 0x7f21f8060580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:36.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21fddc0700 1 -- 172.21.15.44:0/257510023 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f21f80a7220 con 0x7f21f8060170 2020-11-20T12:04:36.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21fddc0700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8060170 0x7f21f8060580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f21f80ab760 tx=0x7f21e801e000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:36.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e801ddb0 con 0x7f21f8060170 2020-11-20T12:04:36.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4424 (secure 0 0 0) 0x7f21e8011940 con 0x7f21f8060170 2020-11-20T12:04:36.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8060170 msgr2=0x7f21f8060580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21e5ffb700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8060170 0x7f21f8060580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f21f80ab760 tx=0x7f21e801e000).stop 2020-11-20T12:04:36.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21e5ffb700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8060170 msgr2=0x7f21f8060580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:36.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.496+0000 7f21e5ffb700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f8060170 0x7f21f8060580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21d0056cd0 msgr2=0x7f21d0059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21d0056cd0 0x7f21d0059190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f21f0001b10 tx=0x7f21f0001360).stop 2020-11-20T12:04:36.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f80aa400 msgr2=0x7f21f80a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 --2- 172.21.15.44:0/257510023 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f21f80aa400 0x7f21f80a2280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f21f40096b0 tx=0x7f21f4006dd0).stop 2020-11-20T12:04:36.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f21fe5c1700 1 -- 172.21.15.44:0/257510023 reap_dead start 2020-11-20T12:04:36.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f21fe5c1700 1 -- 172.21.15.44:0/257510023 reap_dead start 2020-11-20T12:04:36.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 shutdown_connections 2020-11-20T12:04:36.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 >> 172.21.15.44:0/257510023 conn(0x7f21f8062690 msgr2=0x7f21f8139420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:36.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 shutdown_connections 2020-11-20T12:04:36.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:36.497+0000 7f2200024700 1 -- 172.21.15.44:0/257510023 wait complete. 2020-11-20T12:04:36.511 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:36.511 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:36.512 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:36.512 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:04:36.512 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 14, 2020-11-20T12:04:36.512 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:36.512 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:36.513 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:36.513 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.513 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:04:36.513 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:36.513 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:36.514 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:36.514 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:36.514 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:36.515 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:36.515 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:36.515 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:36.515 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:36.515 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:36.516 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:36.516 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.516 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:36.516 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:36.516 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:36.517 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:36.517 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:36.517 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:36.517 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:36.517 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:36.518 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:36.518 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:36.518 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.518 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:36.519 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:36.519 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:36.519 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:36.519 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:36.519 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:36.520 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:36.520 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:36.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:36.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:36.520 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:36.521 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:36.521 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:36.521 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:36.521 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:36.521 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:36.522 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:36.522 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:36.522 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:36.522 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:36.522 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:36.523 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:36.523 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:36.523 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.523 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:36.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.524 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:36.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.525 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:36.525 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:36.525 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:36.525 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:36.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:36.526 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:36.526 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:36.527 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:36.527 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.528 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.528 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.528 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:36.528 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:36.529 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:36.529 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:36.529 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:36.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.530 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:36.530 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:36.530 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:36.530 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:36.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:36.531 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:36.531 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:36.532 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:36.532 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.532 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.532 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.533 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:36.533 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:36.533 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:36.533 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:36.534 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:36.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.534 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:36.535 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:36.535 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:36.535 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:36.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.535 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:36.536 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:36.536 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.536 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:36.537 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:36.537 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:36.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.537 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.538 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:36.538 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:36.538 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:36.538 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:36.538 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:36.539 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.539 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.539 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:36.539 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:36.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.540 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.540 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.540 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:36.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.541 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.541 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:36.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.541 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.541 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.542 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:04:36.542 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.542 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:36.542 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:04:36.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:36.543 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:36.543 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:36.543 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:36.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:36.543 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:36.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:36.544 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:36.544 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:36.545 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:04:36.908 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.904+0000 7fe7f37fe700 1 -- 172.21.15.44:0/2455386321 <== mon.2 v2:172.21.15.79:3301/0 5 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v9) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe80804c020 con 0x7fe8140619f0 2020-11-20T12:04:36.910 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe7ec056d20 msgr2=0x7fe7ec0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.911 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe7ec056d20 0x7fe7ec0591e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fe814060360 tx=0x7fe804006df0).stop 2020-11-20T12:04:36.911 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 msgr2=0x7fe7ec060a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:36.911 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe7ec060a50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe8140a5110 tx=0x7fe808007b90).stop 2020-11-20T12:04:36.911 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 shutdown_connections 2020-11-20T12:04:36.911 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe7ec056d20 0x7fe7ec0591e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.912 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe814061dd0 0x7fe7ec065a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.912 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7ec060f40 0x7fe7ec0633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:04:36.912 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 --2- 172.21.15.44:0/2455386321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe8140619f0 0x7fe7ec060a50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:36.912 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.906+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 >> 172.21.15.44:0/2455386321 conn(0x7fe8140633d0 msgr2=0x7fe8140612c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:36.912 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.907+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 shutdown_connections 2020-11-20T12:04:36.913 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:04:36.907+0000 7fe819acc700 1 -- 172.21.15.44:0/2455386321 wait complete. 2020-11-20T12:04:36.913 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:04:36.921 INFO:tasks.workunit.client.0.smithi044.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2020-11-20T12:04:56.546 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:04:56.547 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:04:56.547 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:04:56.548 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:04:56.551 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:04:56.551 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:04:56.551 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:04:56.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.689+0000 7f49957ec700 1 Processor -- start 2020-11-20T12:04:56.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.689+0000 7f49957ec700 1 -- start start 2020-11-20T12:04:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f49957ec700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f49900a9460 0x7f49900aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f49957ec700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 0x7f499013b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f49957ec700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f499013b560 0x7f4990137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:56.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f49957ec700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499013ac80 con 0x7f49900a4af0 2020-11-20T12:04:56.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f49957ec700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499013b9f0 con 0x7f49900a9460 2020-11-20T12:04:56.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f49957ec700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900ab310 con 0x7f499013b560 2020-11-20T12:04:56.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f4987fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 0x7f499013b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:56.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f4987fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 0x7f499013b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37450/0 (socket says 172.21.15.44:37450) 2020-11-20T12:04:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f4987fff700 1 -- 172.21.15.44:0/3486322294 learned_addr learned my addr 172.21.15.44:0/3486322294 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f4987fff700 1 -- 172.21.15.44:0/3486322294 >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 msgr2=0x7f499013b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:04:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f4987fff700 1 -- 172.21.15.44:0/3486322294 >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 msgr2=0x7f499013b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:04:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f4987fff700 1 --2- 172.21.15.44:0/3486322294 >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 0x7f499013b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:56.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f498effd700 1 --2- 172.21.15.44:0/3486322294 >> v2:172.21.15.79:3300/0 conn(0x7f49900a9460 0x7f49900aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f498f7fe700 1 --2- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f499013b560 0x7f4990137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.690+0000 7f4987fff700 1 --2- 172.21.15.44:0/3486322294 >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 0x7f499013b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498effd700 1 -- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f499013b560 msgr2=0x7f4990137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498effd700 1 --2- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f499013b560 0x7f4990137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:56.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498effd700 1 -- 172.21.15.44:0/3486322294 >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 msgr2=0x7f499013b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:04:56.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498effd700 1 --2- 172.21.15.44:0/3486322294 >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 0x7f499013b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:56.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498effd700 1 -- 172.21.15.44:0/3486322294 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900ab5b0 con 0x7f49900a9460 2020-11-20T12:04:56.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498f7fe700 1 --2- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f499013b560 0x7f4990137c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:56.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498effd700 1 --2- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49900a9460 0x7f49900aad20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f4990060ef0 tx=0x7f4980006df0).ready entity=mon.1 client_cookie=79765d2eade7c16c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:56.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498e7fc700 1 -- 172.21.15.44:0/3486322294 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800097b0 con 0x7f49900a9460 2020-11-20T12:04:56.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498e7fc700 1 -- 172.21.15.44:0/3486322294 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980009910 con 0x7f49900a9460 2020-11-20T12:04:56.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.691+0000 7f498e7fc700 1 -- 172.21.15.44:0/3486322294 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980009c90 con 0x7f49900a9460 2020-11-20T12:04:56.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 -- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49900a9460 msgr2=0x7f49900aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:56.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 --2- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49900a9460 0x7f49900aad20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f4990060ef0 tx=0x7f4980006df0).stop 2020-11-20T12:04:56.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 -- 172.21.15.44:0/3486322294 shutdown_connections 2020-11-20T12:04:56.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 --2- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f499013b560 0x7f4990137c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:56.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 --2- 172.21.15.44:0/3486322294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49900a9460 0x7f49900aad20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:56.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 --2- 172.21.15.44:0/3486322294 >> v2:172.21.15.44:3300/0 conn(0x7f49900a4af0 0x7f499013b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:56.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 -- 172.21.15.44:0/3486322294 >> 172.21.15.44:0/3486322294 conn(0x7f49900621c0 msgr2=0x7f4990060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:56.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 -- 172.21.15.44:0/3486322294 shutdown_connections 2020-11-20T12:04:56.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.692+0000 7f49957ec700 1 -- 172.21.15.44:0/3486322294 wait complete. 2020-11-20T12:04:56.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 Processor -- start 2020-11-20T12:04:56.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 -- start start 2020-11-20T12:04:56.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 0x7f499013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:56.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f49900a9460 0x7f499013d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:56.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f499013b560 0x7f49901324a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:56.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499013c430 con 0x7f49900a4af0 2020-11-20T12:04:56.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499013c660 con 0x7f499013b560 2020-11-20T12:04:56.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.693+0000 7f49957ec700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499013c900 con 0x7f49900a9460 2020-11-20T12:04:56.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 0x7f499013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:56.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 0x7f499013ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37454/0 (socket says 172.21.15.44:37454) 2020-11-20T12:04:56.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498effd700 1 -- 172.21.15.44:0/405347806 learned_addr learned my addr 172.21.15.44:0/405347806 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:56.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4987fff700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f49900a9460 0x7f499013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:56.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498f7fe700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f499013b560 0x7f49901324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:56.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498effd700 1 -- 172.21.15.44:0/405347806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 msgr2=0x7f499013ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:04:56.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498effd700 1 -- 172.21.15.44:0/405347806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 msgr2=0x7f499013ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:04:56.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498effd700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 0x7f499013ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:04:56.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498effd700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 0x7f499013ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:04:56.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4987fff700 1 -- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f499013b560 msgr2=0x7f49901324a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:56.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4987fff700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f499013b560 0x7f49901324a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:56.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4987fff700 1 -- 172.21.15.44:0/405347806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 msgr2=0x7f499013ed40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:04:56.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4987fff700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 0x7f499013ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:56.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4987fff700 1 -- 172.21.15.44:0/405347806 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499013d720 con 0x7f49900a9460 2020-11-20T12:04:56.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f498f7fe700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f499013b560 0x7f49901324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:04:56.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4987fff700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f49900a9460 0x7f499013d230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f497800d930 tx=0x7f4978008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:56.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4986ffd700 1 -- 172.21.15.44:0/405347806 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978017070 con 0x7f49900a9460 2020-11-20T12:04:56.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4986ffd700 1 -- 172.21.15.44:0/405347806 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497800de70 con 0x7f49900a9460 2020-11-20T12:04:56.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f4986ffd700 1 -- 172.21.15.44:0/405347806 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780099e0 con 0x7f49900a9460 2020-11-20T12:04:56.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4990138850 con 0x7f49900a9460 2020-11-20T12:04:56.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.694+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499013cda0 con 0x7f49900a9460 2020-11-20T12:04:56.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.695+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499013d020 con 0x7f49900a9460 2020-11-20T12:04:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.696+0000 7f4986ffd700 1 -- 172.21.15.44:0/405347806 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4978013480 con 0x7f49900a9460 2020-11-20T12:04:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.696+0000 7f4986ffd700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f496c056d20 0x7f496c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.696+0000 7f4986ffd700 1 -- 172.21.15.44:0/405347806 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4978070240 con 0x7f49900a9460 2020-11-20T12:04:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.696+0000 7f498effd700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f496c056d20 0x7f496c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:56.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.697+0000 7f498effd700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f496c056d20 0x7f496c0591e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4980002fc0 tx=0x7f4980015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:56.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.700+0000 7f4986ffd700 1 -- 172.21.15.44:0/405347806 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f499013d020 con 0x7f49900a9460 2020-11-20T12:04:56.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:56.947+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f499013d020 con 0x7f49900a9460 2020-11-20T12:04:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.269+0000 7f4986ffd700 1 -- 172.21.15.44:0/405347806 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f499013d020 con 0x7f49900a9460 2020-11-20T12:04:57.273 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":18,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:04:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.270+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f496c056d20 msgr2=0x7f496c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.270+0000 7f49957ec700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f496c056d20 0x7f496c0591e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4980002fc0 tx=0x7f4980015000).stop 2020-11-20T12:04:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.270+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f49900a9460 msgr2=0x7f499013d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.270+0000 7f49957ec700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f49900a9460 0x7f499013d230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f497800d930 tx=0x7f4978008040).stop 2020-11-20T12:04:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 shutdown_connections 2020-11-20T12:04:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f496c056d20 0x7f496c0591e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f499013b560 0x7f49901324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f49900a4af0 0x7f499013ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 --2- 172.21.15.44:0/405347806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f49900a9460 0x7f499013d230 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 >> 172.21.15.44:0/405347806 conn(0x7f49900621c0 msgr2=0x7f49901379e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:57.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 shutdown_connections 2020-11-20T12:04:57.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.271+0000 7f49957ec700 1 -- 172.21.15.44:0/405347806 wait complete. 2020-11-20T12:04:57.286 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:04:57.287 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:04:57.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.411+0000 7fe403b5b700 1 Processor -- start 2020-11-20T12:04:57.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.412+0000 7fe403b5b700 1 -- start start 2020-11-20T12:04:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.412+0000 7fe403b5b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe3fc0ab620 0x7fe3fc137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.412+0000 7fe403b5b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc067350 con 0x7fe3fc0abc30 2020-11-20T12:04:57.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.412+0000 7fe403b5b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc0ab040 con 0x7fe3fc1380b0 2020-11-20T12:04:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.412+0000 7fe403b5b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc0a9b40 con 0x7fe3fc0ab620 2020-11-20T12:04:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.412+0000 7fe4010f6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe3fc0ab620 0x7fe3fc137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4018f7700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe3fc0abc30 0x7fe3fc0666d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41214/0 (socket says 172.21.15.44:41214) 2020-11-20T12:04:57.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4018f7700 1 -- 172.21.15.44:0/3196476486 learned_addr learned my addr 172.21.15.44:0/3196476486 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:57.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4010f6700 1 -- 172.21.15.44:0/3196476486 >> v1:172.21.15.79:6789/0 conn(0x7fe3fc1380b0 legacy=0x7fe3fc145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4010f6700 1 -- 172.21.15.44:0/3196476486 >> v1:172.21.15.44:6789/0 conn(0x7fe3fc0abc30 legacy=0x7fe3fc0666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4010f6700 1 -- 172.21.15.44:0/3196476486 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc067880 con 0x7fe3fc0ab620 2020-11-20T12:04:57.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4008f5700 1 -- 172.21.15.44:0/3196476486 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189446738 0 0) 0x7fe3fc067350 con 0x7fe3fc0abc30 2020-11-20T12:04:57.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4010f6700 1 --2- 172.21.15.44:0/3196476486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc137bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe3ec00b9a0 tx=0x7fe3ec006df0).ready entity=mon.0 client_cookie=195d844bafbb7da4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4008f5700 1 -- 172.21.15.44:0/3196476486 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0097b0 con 0x7fe3fc0ab620 2020-11-20T12:04:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4008f5700 1 -- 172.21.15.44:0/3196476486 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec009910 con 0x7fe3fc0ab620 2020-11-20T12:04:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe4008f5700 1 -- 172.21.15.44:0/3196476486 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec009c90 con 0x7fe3fc0ab620 2020-11-20T12:04:57.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe403b5b700 1 -- 172.21.15.44:0/3196476486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 msgr2=0x7fe3fc137bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.413+0000 7fe403b5b700 1 --2- 172.21.15.44:0/3196476486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc137bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe3ec00b9a0 tx=0x7fe3ec006df0).stop 2020-11-20T12:04:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.414+0000 7fe403b5b700 1 -- 172.21.15.44:0/3196476486 shutdown_connections 2020-11-20T12:04:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.414+0000 7fe403b5b700 1 --2- 172.21.15.44:0/3196476486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc137bc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.414+0000 7fe403b5b700 1 -- 172.21.15.44:0/3196476486 >> 172.21.15.44:0/3196476486 conn(0x7fe3fc065290 msgr2=0x7fe3fc060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:57.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.414+0000 7fe403b5b700 1 -- 172.21.15.44:0/3196476486 shutdown_connections 2020-11-20T12:04:57.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.414+0000 7fe403b5b700 1 -- 172.21.15.44:0/3196476486 wait complete. 2020-11-20T12:04:57.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 Processor -- start 2020-11-20T12:04:57.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 -- start start 2020-11-20T12:04:57.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc13c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe3fc0abc30 0x7fe3fc1325c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc1380b0 0x7fe3fc13e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc13dca0 con 0x7fe3fc0ab620 2020-11-20T12:04:57.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc13df40 con 0x7fe3fc1380b0 2020-11-20T12:04:57.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe403b5b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc061930 con 0x7fe3fc0abc30 2020-11-20T12:04:57.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe4018f7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc13c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe4018f7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc13c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37472/0 (socket says 172.21.15.44:37472) 2020-11-20T12:04:57.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe4018f7700 1 -- 172.21.15.44:0/3300305259 learned_addr learned my addr 172.21.15.44:0/3300305259 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe4010f6700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe3fc0abc30 0x7fe3fc1325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.415+0000 7fe4020f8700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc1380b0 0x7fe3fc13e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe4018f7700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe3fc0abc30 msgr2=0x7fe3fc1325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe4018f7700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe3fc0abc30 0x7fe3fc1325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe4018f7700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc1380b0 msgr2=0x7fe3fc13e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe4018f7700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3fc1380b0 0x7fe3fc13e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe4018f7700 1 -- 172.21.15.44:0/3300305259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc13d990 con 0x7fe3fc0ab620 2020-11-20T12:04:57.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe4018f7700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc13c150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe3f800c760 tx=0x7fe3f8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80072e0 con 0x7fe3fc0ab620 2020-11-20T12:04:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8007440 con 0x7fe3fc0ab620 2020-11-20T12:04:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc0632b0 con 0x7fe3fc0ab620 2020-11-20T12:04:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8005840 con 0x7fe3fc0ab620 2020-11-20T12:04:57.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.416+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc063800 con 0x7fe3fc0ab620 2020-11-20T12:04:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.417+0000 7fe403b5b700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0643c0 0x7fe3fc13e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.417+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc13c640 con 0x7fe3fc0643c0 2020-11-20T12:04:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.417+0000 7fe4010f6700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0643c0 0x7fe3fc13e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.417+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe3f801a060 con 0x7fe3fc0ab620 2020-11-20T12:04:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe3d4056d20 0x7fe3d40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe4010f6700 1 -- 172.21.15.44:0/3300305259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3fc13ee40 con 0x7fe3fc0643c0 2020-11-20T12:04:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe3f806f8d0 con 0x7fe3fc0ab620 2020-11-20T12:04:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe4010f6700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0643c0 0x7fe3fc13e950 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe3fc0a95f0 tx=0x7fe3ec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe4020f8700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe3d4056d20 0x7fe3d40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec011e00 con 0x7fe3fc0643c0 2020-11-20T12:04:57.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fe3ec01e420 con 0x7fe3fc0643c0 2020-11-20T12:04:57.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0643c0 msgr2=0x7fe3fc13e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0643c0 0x7fe3fc13e950 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fe3fc0a95f0 tx=0x7fe3ec015000).stop 2020-11-20T12:04:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0643c0 msgr2=0x7fe3fc13e950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe3e9ffb700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0643c0 0x7fe3fc13e950 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.418+0000 7fe4020f8700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe3d4056d20 0x7fe3d40591e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe3f40012a0 tx=0x7fe3f4007f10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.426+0000 7fe403b5b700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc14dc10 0x7fe3fc14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.426+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc13dde0 con 0x7fe3fc14dc10 2020-11-20T12:04:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.426+0000 7fe4010f6700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc14dc10 0x7fe3fc14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe4010f6700 1 -- 172.21.15.44:0/3300305259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe3fc062b60 con 0x7fe3fc14dc10 2020-11-20T12:04:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe4010f6700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc14dc10 0x7fe3fc14dff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe3ec006e30 tx=0x7fe3ec021070).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec029440 con 0x7fe3fc14dc10 2020-11-20T12:04:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7fe3ec01f440 con 0x7fe3fc14dc10 2020-11-20T12:04:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc14dc10 msgr2=0x7fe3fc14dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe3e9ffb700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc14dc10 0x7fe3fc14dff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe3ec006e30 tx=0x7fe3ec021070).stop 2020-11-20T12:04:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe3e9ffb700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc14dc10 msgr2=0x7fe3fc14dff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe3e9ffb700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc14dc10 0x7fe3fc14dff0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe3d4056d20 msgr2=0x7fe3d40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe403b5b700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe3d4056d20 0x7fe3d40591e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fe3f40012a0 tx=0x7fe3f4007f10).stop 2020-11-20T12:04:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 msgr2=0x7fe3fc13c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe403b5b700 1 --2- 172.21.15.44:0/3300305259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ab620 0x7fe3fc13c150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe3f800c760 tx=0x7fe3f8007a60).stop 2020-11-20T12:04:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe4020f8700 1 -- 172.21.15.44:0/3300305259 reap_dead start 2020-11-20T12:04:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe4020f8700 1 -- 172.21.15.44:0/3300305259 reap_dead start 2020-11-20T12:04:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 shutdown_connections 2020-11-20T12:04:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.427+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 >> 172.21.15.44:0/3300305259 conn(0x7fe3fc065290 msgr2=0x7fe3fc137990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.428+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 shutdown_connections 2020-11-20T12:04:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.428+0000 7fe403b5b700 1 -- 172.21.15.44:0/3300305259 wait complete. 2020-11-20T12:04:57.443 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:57.443 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:57.443 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:57.443 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:04:57.444 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 18, 2020-11-20T12:04:57.444 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:57.444 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:57.444 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:57.444 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:57.445 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.445 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:04:57.445 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:57.445 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:57.446 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:57.446 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.446 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.446 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.446 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.447 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.447 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.447 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.447 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.448 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.448 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:57.448 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:57.448 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.448 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.449 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.450 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:57.451 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:57.451 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:57.451 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:57.451 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:57.452 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:57.452 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:57.452 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:57.452 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:57.453 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:57.453 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:57.453 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:57.453 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:57.454 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:57.454 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:57.454 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.454 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.454 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.455 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.455 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.455 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.455 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.456 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.456 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.456 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:57.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.457 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:57.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.457 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:57.457 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:57.458 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.458 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.458 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.458 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:57.459 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.459 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.460 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:57.460 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.461 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.461 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:57.461 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:57.462 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.462 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.462 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.463 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:57.463 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:57.463 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.463 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.464 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.464 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:57.464 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.465 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.465 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:57.465 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.466 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.466 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.466 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:57.467 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:57.467 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.467 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.467 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.469 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:57.469 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:57.470 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.470 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.470 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.470 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:57.471 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.471 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.471 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:57.472 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.472 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.472 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:57.473 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:57.473 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.473 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.473 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.474 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.474 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.474 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:57.474 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:57.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.475 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.475 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.475 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:57.475 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.476 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.476 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:57.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.476 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.476 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.477 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:04:57.477 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.477 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.477 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:57.478 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:57.479 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:04:57.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.603+0000 7fee6cb9e700 1 Processor -- start 2020-11-20T12:04:57.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.603+0000 7fee6cb9e700 1 -- start start 2020-11-20T12:04:57.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee6cb9e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680a1a40 0x7fee680aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee6cb9e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fee680666d0 0x7fee68066b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee6cb9e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fee68137bb0 0x7fee68138060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee6cb9e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee680a9b30 con 0x7fee680666d0 2020-11-20T12:04:57.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee6cb9e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee680ab240 con 0x7fee68137bb0 2020-11-20T12:04:57.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee6cb9e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee68067470 con 0x7fee680a1a40 2020-11-20T12:04:57.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fee680666d0 0x7fee68066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fee680666d0 0x7fee68066b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37486/0 (socket says 172.21.15.44:37486) 2020-11-20T12:04:57.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 -- 172.21.15.44:0/2919939945 learned_addr learned my addr 172.21.15.44:0/2919939945 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:57.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee66d9d700 1 --2- 172.21.15.44:0/2919939945 >> v2:172.21.15.79:3300/0 conn(0x7fee68137bb0 0x7fee68138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee6659c700 1 --2- 172.21.15.44:0/2919939945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680a1a40 0x7fee680aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 -- 172.21.15.44:0/2919939945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680a1a40 msgr2=0x7fee680aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:57.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 --2- 172.21.15.44:0/2919939945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680a1a40 0x7fee680aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 -- 172.21.15.44:0/2919939945 >> v2:172.21.15.79:3300/0 conn(0x7fee68137bb0 msgr2=0x7fee68138060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:57.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 --2- 172.21.15.44:0/2919939945 >> v2:172.21.15.79:3300/0 conn(0x7fee68137bb0 0x7fee68138060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 -- 172.21.15.44:0/2919939945 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68067710 con 0x7fee680666d0 2020-11-20T12:04:57.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee66d9d700 1 --2- 172.21.15.44:0/2919939945 >> v2:172.21.15.79:3300/0 conn(0x7fee68137bb0 0x7fee68138060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:57.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.604+0000 7fee65d9b700 1 --2- 172.21.15.44:0/2919939945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680666d0 0x7fee68066b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fee6813ab70 tx=0x7fee5c006eb0).ready entity=mon.0 client_cookie=f8c85dd8f1f4fd0d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6559a700 1 -- 172.21.15.44:0/2919939945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c009890 con 0x7fee680666d0 2020-11-20T12:04:57.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6559a700 1 -- 172.21.15.44:0/2919939945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c0099f0 con 0x7fee680666d0 2020-11-20T12:04:57.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6559a700 1 -- 172.21.15.44:0/2919939945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c009d70 con 0x7fee680666d0 2020-11-20T12:04:57.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2919939945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680666d0 msgr2=0x7fee68066b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2919939945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680666d0 0x7fee68066b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fee6813ab70 tx=0x7fee5c006eb0).stop 2020-11-20T12:04:57.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2919939945 shutdown_connections 2020-11-20T12:04:57.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2919939945 >> v2:172.21.15.79:3300/0 conn(0x7fee68137bb0 0x7fee68138060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2919939945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680a1a40 0x7fee680aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2919939945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680666d0 0x7fee68066b30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2919939945 >> 172.21.15.44:0/2919939945 conn(0x7fee680af030 msgr2=0x7fee680af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:57.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2919939945 shutdown_connections 2020-11-20T12:04:57.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.605+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2919939945 wait complete. 2020-11-20T12:04:57.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.606+0000 7fee6cb9e700 1 Processor -- start 2020-11-20T12:04:57.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.606+0000 7fee6cb9e700 1 -- start start 2020-11-20T12:04:57.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee6cb9e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680666d0 0x7fee68139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee6cb9e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680a1a40 0x7fee680630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee6cb9e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68137bb0 0x7fee6814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee6cb9e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68135e00 con 0x7fee680a1a40 2020-11-20T12:04:57.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee6cb9e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681360a0 con 0x7fee68137bb0 2020-11-20T12:04:57.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee6cb9e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee68136340 con 0x7fee680666d0 2020-11-20T12:04:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680a1a40 0x7fee680630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680a1a40 0x7fee680630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37494/0 (socket says 172.21.15.44:37494) 2020-11-20T12:04:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 -- 172.21.15.44:0/2247667465 learned_addr learned my addr 172.21.15.44:0/2247667465 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:57.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee66d9d700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68137bb0 0x7fee6814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee6659c700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680666d0 0x7fee68139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680666d0 msgr2=0x7fee68139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee680666d0 0x7fee68139e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68137bb0 msgr2=0x7fee6814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68137bb0 0x7fee6814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 -- 172.21.15.44:0/2247667465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee680635b0 con 0x7fee680a1a40 2020-11-20T12:04:57.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.607+0000 7fee65d9b700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680a1a40 0x7fee680630c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fee5c006ef0 tx=0x7fee5c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.608+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c011e50 con 0x7fee680a1a40 2020-11-20T12:04:57.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.608+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c01e460 con 0x7fee680a1a40 2020-11-20T12:04:57.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.608+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6813a700 con 0x7fee680a1a40 2020-11-20T12:04:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.608+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c01e800 con 0x7fee680a1a40 2020-11-20T12:04:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.608+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6814e130 con 0x7fee680a1a40 2020-11-20T12:04:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68130200 0x7fee681305e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee681382a0 con 0x7fee68130200 2020-11-20T12:04:57.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee6659c700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68130200 0x7fee681305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fee5c025030 con 0x7fee680a1a40 2020-11-20T12:04:57.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee6659c700 1 -- 172.21.15.44:0/2247667465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee680a2930 con 0x7fee68130200 2020-11-20T12:04:57.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee567fc700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee3c056d20 0x7fee3c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee6659c700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68130200 0x7fee681305e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fee5800f760 tx=0x7fee58008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fee5c071880 con 0x7fee680a1a40 2020-11-20T12:04:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.609+0000 7fee66d9d700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee3c056d20 0x7fee3c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.610+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800b6b0 con 0x7fee68130200 2020-11-20T12:04:57.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.610+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fee5800b810 con 0x7fee68130200 2020-11-20T12:04:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.610+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68130200 msgr2=0x7fee681305e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.610+0000 7fee567fc700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68130200 0x7fee681305e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fee5800f760 tx=0x7fee58008040).stop 2020-11-20T12:04:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.610+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68130200 msgr2=0x7fee681305e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:57.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.610+0000 7fee567fc700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68130200 0x7fee681305e0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0x7fee5800f760 tx=0x7fee58008040).stop 2020-11-20T12:04:57.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.610+0000 7fee66d9d700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee3c056d20 0x7fee3c0591e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fee60006f50 tx=0x7fee60008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.617+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68061880 0x7fee68061c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.617+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68135f40 con 0x7fee68061880 2020-11-20T12:04:57.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.618+0000 7fee6659c700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68061880 0x7fee68061c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.618+0000 7fee6659c700 1 -- 172.21.15.44:0/2247667465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fee680637a0 con 0x7fee68061880 2020-11-20T12:04:57.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.618+0000 7fee6659c700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68061880 0x7fee68061c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fee580067d0 tx=0x7fee58008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.618+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5801cde0 con 0x7fee68061880 2020-11-20T12:04:57.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4775 (secure 0 0 0) 0x7fee5800b4a0 con 0x7fee68061880 2020-11-20T12:04:57.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68061880 msgr2=0x7fee68061c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee567fc700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68061880 0x7fee68061c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fee580067d0 tx=0x7fee58008040).stop 2020-11-20T12:04:57.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee567fc700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68061880 msgr2=0x7fee68061c60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:57.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee567fc700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee68061880 0x7fee68061c60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee3c056d20 msgr2=0x7fee3c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee3c056d20 0x7fee3c0591e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fee60006f50 tx=0x7fee60008040).stop 2020-11-20T12:04:57.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680a1a40 msgr2=0x7fee680630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 --2- 172.21.15.44:0/2247667465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee680a1a40 0x7fee680630c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fee5c006ef0 tx=0x7fee5c015000).stop 2020-11-20T12:04:57.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee66d9d700 1 -- 172.21.15.44:0/2247667465 reap_dead start 2020-11-20T12:04:57.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee66d9d700 1 -- 172.21.15.44:0/2247667465 reap_dead start 2020-11-20T12:04:57.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 shutdown_connections 2020-11-20T12:04:57.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 >> 172.21.15.44:0/2247667465 conn(0x7fee680af030 msgr2=0x7fee68140df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:57.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 shutdown_connections 2020-11-20T12:04:57.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.619+0000 7fee6cb9e700 1 -- 172.21.15.44:0/2247667465 wait complete. 2020-11-20T12:04:57.639 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:57.639 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:57.639 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:57.639 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:04:57.639 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 18, 2020-11-20T12:04:57.640 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:57.640 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:57.640 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:57.640 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:57.641 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.641 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:04:57.641 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:57.641 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:57.641 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:57.642 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.642 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.642 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.642 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.642 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.643 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.643 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.643 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.643 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.643 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:57.644 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:57.644 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.644 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.644 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.645 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.645 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.645 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.645 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.646 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.646 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:57.646 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:57.646 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:57.647 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:57.647 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:57.647 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:57.647 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:57.647 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:57.648 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:57.648 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:57.648 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:57.648 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:57.648 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:57.649 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:57.649 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:57.649 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.649 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.649 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.650 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.650 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.650 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.650 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.650 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.651 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.651 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:57.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.651 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:57.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.652 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:57.652 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:57.652 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.652 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.653 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.653 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:57.653 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.654 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:57.654 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.655 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.655 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:57.655 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:57.655 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.656 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.656 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.656 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:57.657 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:57.657 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.657 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.658 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:57.658 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.659 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:57.659 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.659 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.660 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:57.660 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:57.660 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.660 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.660 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.661 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:57.661 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:57.661 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.662 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.662 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:57.662 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.663 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:57.663 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.664 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.664 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:57.664 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:57.665 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.665 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.665 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.666 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:57.666 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:57.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.666 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.667 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.667 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:57.667 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.667 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.667 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:04:57.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.668 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.668 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.668 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:04:57.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.669 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.669 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:04:57.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.669 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.669 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.670 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 8 2020-11-20T12:04:57.670 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.670 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.670 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:57.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.671 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.671 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.671 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:04:57.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.671 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.672 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:04:57.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.672 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.672 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.672 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:57.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.673 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.673 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:57.673 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:57.674 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:04:57.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.798+0000 7f635f4d9700 1 Processor -- start 2020-11-20T12:04:57.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635f4d9700 1 -- start start 2020-11-20T12:04:57.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635f4d9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6358060330 0x7f63580a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635f4d9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f635813b0a0 0x7f6358141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635f4d9700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63580a6260 con 0x7f63580a4c30 2020-11-20T12:04:57.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635f4d9700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63580a52e0 con 0x7f6358060330 2020-11-20T12:04:57.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635f4d9700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63580a6490 con 0x7f635813b0a0 2020-11-20T12:04:57.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635d275700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6358060330 0x7f63580a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635ca74700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f63580a4c30 0x7f635813a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41252/0 (socket says 172.21.15.44:41252) 2020-11-20T12:04:57.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.799+0000 7f635ca74700 1 -- 172.21.15.44:0/1467682321 learned_addr learned my addr 172.21.15.44:0/1467682321 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:04:57.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f635da76700 1 --2- 172.21.15.44:0/1467682321 >> v2:172.21.15.79:3300/0 conn(0x7f635813b0a0 0x7f6358141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f634bfff700 1 -- 172.21.15.44:0/1467682321 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594254955 0 0) 0x7f63580a6260 con 0x7f63580a4c30 2020-11-20T12:04:57.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f634bfff700 1 -- 172.21.15.44:0/1467682321 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6340003240 con 0x7f63580a4c30 2020-11-20T12:04:57.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f635d275700 1 -- 172.21.15.44:0/1467682321 >> v2:172.21.15.79:3300/0 conn(0x7f635813b0a0 msgr2=0x7f6358141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:57.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f635d275700 1 --2- 172.21.15.44:0/1467682321 >> v2:172.21.15.79:3300/0 conn(0x7f635813b0a0 0x7f6358141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f635d275700 1 -- 172.21.15.44:0/1467682321 >> v1:172.21.15.44:6789/0 conn(0x7f63580a4c30 legacy=0x7f635813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f635d275700 1 -- 172.21.15.44:0/1467682321 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63580a6730 con 0x7f6358060330 2020-11-20T12:04:57.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f635d275700 1 --2- 172.21.15.44:0/1467682321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a4740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6358061250 tx=0x7f634c006e80).ready entity=mon.0 client_cookie=d79a871265cd323 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f634bfff700 1 -- 172.21.15.44:0/1467682321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c014070 con 0x7f6358060330 2020-11-20T12:04:57.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f634bfff700 1 -- 172.21.15.44:0/1467682321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c00a6b0 con 0x7f6358060330 2020-11-20T12:04:57.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.800+0000 7f634bfff700 1 -- 172.21.15.44:0/1467682321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c00aa30 con 0x7f6358060330 2020-11-20T12:04:57.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 -- 172.21.15.44:0/1467682321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 msgr2=0x7f63580a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 --2- 172.21.15.44:0/1467682321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a4740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6358061250 tx=0x7f634c006e80).stop 2020-11-20T12:04:57.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 -- 172.21.15.44:0/1467682321 shutdown_connections 2020-11-20T12:04:57.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 --2- 172.21.15.44:0/1467682321 >> v2:172.21.15.79:3300/0 conn(0x7f635813b0a0 0x7f6358141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 --2- 172.21.15.44:0/1467682321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a4740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 -- 172.21.15.44:0/1467682321 >> 172.21.15.44:0/1467682321 conn(0x7f6358062690 msgr2=0x7f63580a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 -- 172.21.15.44:0/1467682321 shutdown_connections 2020-11-20T12:04:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.801+0000 7f635f4d9700 1 -- 172.21.15.44:0/1467682321 wait complete. 2020-11-20T12:04:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 Processor -- start 2020-11-20T12:04:57.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 -- start start 2020-11-20T12:04:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580a4c30 0x7f63581376d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f635813b0a0 0x7f63580ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635809f460 con 0x7f6358060330 2020-11-20T12:04:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635809f700 con 0x7f635813b0a0 2020-11-20T12:04:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.802+0000 7f635f4d9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63580a2f60 con 0x7f63580a4c30 2020-11-20T12:04:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37508/0 (socket says 172.21.15.44:37508) 2020-11-20T12:04:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 -- 172.21.15.44:0/3757928726 learned_addr learned my addr 172.21.15.44:0/3757928726 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635ca74700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580a4c30 0x7f63581376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635da76700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f635813b0a0 0x7f63580ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580a4c30 msgr2=0x7f63581376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580a4c30 0x7f63581376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f635813b0a0 msgr2=0x7f63580ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f635813b0a0 0x7f63580ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 -- 172.21.15.44:0/3757928726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635809da30 con 0x7f6358060330 2020-11-20T12:04:57.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635da76700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f635813b0a0 0x7f63580ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:57.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635ca74700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580a4c30 0x7f63581376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:57.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635d275700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a3400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f63580aa8b0 tx=0x7f6358137f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c010e00 con 0x7f6358060330 2020-11-20T12:04:57.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c020070 con 0x7f6358060330 2020-11-20T12:04:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635809efc0 con 0x7f6358060330 2020-11-20T12:04:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c01c440 con 0x7f6358060330 2020-11-20T12:04:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.803+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63580adc80 con 0x7f6358060330 2020-11-20T12:04:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.804+0000 7f635f4d9700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63581320d0 0x7f63581324b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.804+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635809f5a0 con 0x7f63581320d0 2020-11-20T12:04:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.804+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f634c00e040 con 0x7f6358060330 2020-11-20T12:04:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.804+0000 7f635ca74700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63581320d0 0x7f63581324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6330056d20 0x7f63300591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f634c071870 con 0x7f6358060330 2020-11-20T12:04:57.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f635ca74700 1 -- 172.21.15.44:0/3757928726 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f635809f2f0 con 0x7f63581320d0 2020-11-20T12:04:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f635da76700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6330056d20 0x7f63300591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f635ca74700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63581320d0 0x7f63581324b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f63580a4a30 tx=0x7f6354009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354002040 con 0x7f63581320d0 2020-11-20T12:04:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f635da76700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6330056d20 0x7f63300591e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6350006fd0 tx=0x7f6350008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f635400cbd0 con 0x7f63581320d0 2020-11-20T12:04:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63581320d0 msgr2=0x7f63581324b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63581320d0 0x7f63581324b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f63580a4a30 tx=0x7f6354009040).stop 2020-11-20T12:04:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63581320d0 msgr2=0x7f63581324b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:57.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.805+0000 7f63497fa700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63581320d0 0x7f63581324b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.813+0000 7f635f4d9700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580aac60 0x7f63580ab040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:57.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.813+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635809dc20 con 0x7f63580aac60 2020-11-20T12:04:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.813+0000 7f635ca74700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580aac60 0x7f63580ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f635ca74700 1 -- 172.21.15.44:0/3757928726 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6358061650 con 0x7f63580aac60 2020-11-20T12:04:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f635ca74700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580aac60 0x7f63580ab040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f635809f270 tx=0x7f6354009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:57.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354007770 con 0x7f63580aac60 2020-11-20T12:04:57.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4435 (secure 0 0 0) 0x7f635400c9c0 con 0x7f63580aac60 2020-11-20T12:04:57.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580aac60 msgr2=0x7f63580ab040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f63497fa700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580aac60 0x7f63580ab040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f635809f270 tx=0x7f6354009040).stop 2020-11-20T12:04:57.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f63497fa700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580aac60 msgr2=0x7f63580ab040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:04:57.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.814+0000 7f63497fa700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63580aac60 0x7f63580ab040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6330056d20 msgr2=0x7f63300591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6330056d20 0x7f63300591e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6350006fd0 tx=0x7f6350008040).stop 2020-11-20T12:04:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 msgr2=0x7f63580a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a3400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f63580aa8b0 tx=0x7f6358137f50).stop 2020-11-20T12:04:57.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635da76700 1 -- 172.21.15.44:0/3757928726 reap_dead start 2020-11-20T12:04:57.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 shutdown_connections 2020-11-20T12:04:57.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 --2- 172.21.15.44:0/3757928726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6358060330 0x7f63580a3400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:57.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 >> 172.21.15.44:0/3757928726 conn(0x7f6358062690 msgr2=0x7f6358134540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:57.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 shutdown_connections 2020-11-20T12:04:57.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.815+0000 7f635f4d9700 1 -- 172.21.15.44:0/3757928726 wait complete. 2020-11-20T12:04:57.830 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:04:57.830 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:57.830 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:57.830 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:04:57.831 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 18, 2020-11-20T12:04:57.831 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:04:57.831 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:04:57.831 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:04:57.831 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:04:57.832 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.832 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:04:57.832 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:57.832 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:04:57.832 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:04:57.833 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.833 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.833 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.833 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.833 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.834 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.834 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.834 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.834 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.835 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:04:57.835 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:04:57.835 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.835 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.835 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.836 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.836 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.836 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.836 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.836 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.837 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.837 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:04:57.837 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:04:57.837 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:04:57.838 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:04:57.838 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:04:57.838 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:04:57.838 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:57.838 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:04:57.839 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:04:57.839 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:04:57.839 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:04:57.839 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:04:57.839 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:04:57.840 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:04:57.840 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:04:57.840 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:04:57.840 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:04:57.840 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:04:57.841 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:04:57.841 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:04:57.841 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:04:57.841 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:04:57.842 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:04:57.842 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.842 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:04:57.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.842 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:04:57.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.843 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:04:57.843 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:04:57.843 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.843 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.844 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:04:57.844 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.845 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:04:57.845 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.845 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.846 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.846 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:04:57.846 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:04:57.846 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.847 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.847 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.847 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:04:57.848 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:04:57.848 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.848 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.849 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:04:57.849 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.850 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:04:57.850 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.850 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.850 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.851 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:04:57.851 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:04:57.851 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.851 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.852 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.852 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:04:57.853 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:04:57.853 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:04:57.853 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:04:57.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:04:57.854 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:04:57.854 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:04:57.855 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:04:57.855 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:04:57.855 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.855 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.856 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:04:57.856 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:04:57.856 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:04:57.856 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:04:57.856 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:04:57.857 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.857 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.857 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:04:57.857 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:04:57.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.858 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.858 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.858 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:57.858 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.859 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.859 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:04:57.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.859 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.859 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.860 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:57.860 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.860 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:04:57.860 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:04:57.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:04:57.861 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:04:57.861 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:04:57.861 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:04:57.861 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:04:57.861 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:04:57.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:04:57.862 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:04:57.862 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:04:57.863 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:04:57.863 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:04:57.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.988+0000 7effbc7d4700 1 Processor -- start 2020-11-20T12:04:57.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.988+0000 7effbc7d4700 1 -- start start 2020-11-20T12:04:58.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.988+0000 7effbc7d4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7effb4060330 0x7effb40a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:58.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effbc7d4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7effb413b0a0 0x7effb4141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:58.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effbc7d4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb40a6260 con 0x7effb40a4c30 2020-11-20T12:04:58.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effbc7d4700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb40a52e0 con 0x7effb4060330 2020-11-20T12:04:58.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effbc7d4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb40a6490 con 0x7effb413b0a0 2020-11-20T12:04:58.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7effb4060330 0x7effb40a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:58.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7effb4060330 0x7effb40a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37520/0 (socket says 172.21.15.44:37520) 2020-11-20T12:04:58.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 -- 172.21.15.44:0/2632185536 learned_addr learned my addr 172.21.15.44:0/2632185536 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:58.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effbad71700 1 --2- 172.21.15.44:0/2632185536 >> v2:172.21.15.79:3300/0 conn(0x7effb413b0a0 0x7effb4141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:58.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 -- 172.21.15.44:0/2632185536 >> v2:172.21.15.79:3300/0 conn(0x7effb413b0a0 msgr2=0x7effb4141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:58.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 --2- 172.21.15.44:0/2632185536 >> v2:172.21.15.79:3300/0 conn(0x7effb413b0a0 0x7effb4141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 -- 172.21.15.44:0/2632185536 >> v1:172.21.15.79:6789/0 conn(0x7effb40a4c30 legacy=0x7effb413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:04:58.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 -- 172.21.15.44:0/2632185536 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40a6730 con 0x7effb4060330 2020-11-20T12:04:58.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effba570700 1 --2- 172.21.15.44:0/2632185536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb4060330 0x7effb40a4740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7effb4061250 tx=0x7effb0006eb0).ready entity=mon.0 client_cookie=98ba59e081de4499 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:58.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effb956e700 1 -- 172.21.15.44:0/2632185536 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0009890 con 0x7effb4060330 2020-11-20T12:04:58.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effb956e700 1 -- 172.21.15.44:0/2632185536 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00099f0 con 0x7effb4060330 2020-11-20T12:04:58.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.989+0000 7effb956e700 1 -- 172.21.15.44:0/2632185536 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0009d70 con 0x7effb4060330 2020-11-20T12:04:58.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 -- 172.21.15.44:0/2632185536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb4060330 msgr2=0x7effb40a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:58.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 --2- 172.21.15.44:0/2632185536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb4060330 0x7effb40a4740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7effb4061250 tx=0x7effb0006eb0).stop 2020-11-20T12:04:58.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 -- 172.21.15.44:0/2632185536 shutdown_connections 2020-11-20T12:04:58.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 --2- 172.21.15.44:0/2632185536 >> v2:172.21.15.79:3300/0 conn(0x7effb413b0a0 0x7effb4141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 --2- 172.21.15.44:0/2632185536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb4060330 0x7effb40a4740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 -- 172.21.15.44:0/2632185536 >> 172.21.15.44:0/2632185536 conn(0x7effb4062690 msgr2=0x7effb40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:58.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 -- 172.21.15.44:0/2632185536 shutdown_connections 2020-11-20T12:04:58.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.990+0000 7effbc7d4700 1 -- 172.21.15.44:0/2632185536 wait complete. 2020-11-20T12:04:58.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 Processor -- start 2020-11-20T12:04:58.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 -- start start 2020-11-20T12:04:58.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effb4060330 0x7effb41353d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:58.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb40a4c30 0x7effb4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:58.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7effb413b0a0 0x7effb4131500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:58.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb41372c0 con 0x7effb40a4c30 2020-11-20T12:04:58.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb4137800 con 0x7effb4060330 2020-11-20T12:04:58.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effbc7d4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb40acb70 con 0x7effb413b0a0 2020-11-20T12:04:58.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.991+0000 7effb9d6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb40a4c30 0x7effb4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:58.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb40a4c30 0x7effb4136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37528/0 (socket says 172.21.15.44:37528) 2020-11-20T12:04:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 -- 172.21.15.44:0/3114566423 learned_addr learned my addr 172.21.15.44:0/3114566423 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:04:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effba570700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effb4060330 0x7effb41353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effbad71700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7effb413b0a0 0x7effb4131500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 -- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7effb413b0a0 msgr2=0x7effb4131500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:58.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7effb413b0a0 0x7effb4131500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 -- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effb4060330 msgr2=0x7effb41353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:58.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effb4060330 0x7effb41353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 -- 172.21.15.44:0/3114566423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40acd30 con 0x7effb40a4c30 2020-11-20T12:04:58.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effbad71700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7effb413b0a0 0x7effb4131500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:58.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effba570700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effb4060330 0x7effb41353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:04:58.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effb9d6f700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb40a4c30 0x7effb4136c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7effa400c730 tx=0x7effa4007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:58.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effa27fc700 1 -- 172.21.15.44:0/3114566423 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400e560 con 0x7effb40a4c30 2020-11-20T12:04:58.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effa27fc700 1 -- 172.21.15.44:0/3114566423 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa400e6c0 con 0x7effb40a4c30 2020-11-20T12:04:58.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb4137ca0 con 0x7effb40a4c30 2020-11-20T12:04:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effa27fc700 1 -- 172.21.15.44:0/3114566423 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400ea80 con 0x7effb40a4c30 2020-11-20T12:04:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.992+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb412f170 con 0x7effb40a4c30 2020-11-20T12:04:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.993+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb4137110 con 0x7effb40a4c30 2020-11-20T12:04:58.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.993+0000 7effa27fc700 1 -- 172.21.15.44:0/3114566423 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7effa4010070 con 0x7effb40a4c30 2020-11-20T12:04:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.994+0000 7effa27fc700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff8c056d50 0x7eff8c059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:04:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.994+0000 7effa27fc700 1 -- 172.21.15.44:0/3114566423 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7effa406fc60 con 0x7effb40a4c30 2020-11-20T12:04:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.994+0000 7effba570700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff8c056d50 0x7eff8c059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:04:58.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.994+0000 7effba570700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff8c056d50 0x7eff8c059210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7effb0006ef0 tx=0x7effb0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:04:58.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:57.998+0000 7effa27fc700 1 -- 172.21.15.44:0/3114566423 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7effa4044f50 con 0x7effb40a4c30 2020-11-20T12:04:58.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.242+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7effb41379c0 con 0x7effb40a4c30 2020-11-20T12:04:58.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.243+0000 7effa27fc700 1 -- 172.21.15.44:0/3114566423 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7effa4048570 con 0x7effb40a4c30 2020-11-20T12:04:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff8c056d50 msgr2=0x7eff8c059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff8c056d50 0x7eff8c059210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7effb0006ef0 tx=0x7effb0015000).stop 2020-11-20T12:04:58.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb40a4c30 msgr2=0x7effb4136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:04:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb40a4c30 0x7effb4136c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7effa400c730 tx=0x7effa4007c30).stop 2020-11-20T12:04:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 shutdown_connections 2020-11-20T12:04:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff8c056d50 0x7eff8c059210 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7effb413b0a0 0x7effb4131500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7effb4060330 0x7effb41353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 --2- 172.21.15.44:0/3114566423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effb40a4c30 0x7effb4136c20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:04:58.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 >> 172.21.15.44:0/3114566423 conn(0x7effb4062690 msgr2=0x7effb4138390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:04:58.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 shutdown_connections 2020-11-20T12:04:58.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:04:58.245+0000 7effbc7d4700 1 -- 172.21.15.44:0/3114566423 wait complete. 2020-11-20T12:04:58.260 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:05:05.720 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:05:05.723 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:05:08.262 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:05:08.262 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:05:08.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.386+0000 7fb24cac0700 1 Processor -- start 2020-11-20T12:05:08.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.386+0000 7fb24cac0700 1 -- start start 2020-11-20T12:05:08.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24cac0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb2480aa020 0x7fb2480a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24cac0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 0x7fb2480a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24cac0700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2480a9770 con 0x7fb2480aa400 2020-11-20T12:05:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24cac0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2480a56c0 con 0x7fb2480aa020 2020-11-20T12:05:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24cac0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480a6da0 con 0x7fb24813ac60 2020-11-20T12:05:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb245d9b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb2480aa400 0x7fb2480a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41286/0 (socket says 172.21.15.44:41286) 2020-11-20T12:05:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb245d9b700 1 -- 172.21.15.44:0/3712187206 learned_addr learned my addr 172.21.15.44:0/3712187206 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb246d9d700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 0x7fb2480a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24659c700 1 --2- 172.21.15.44:0/3712187206 >> v2:172.21.15.79:3300/0 conn(0x7fb2480aa020 0x7fb2480a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066061371 0 0) 0x7fb2380010d0 con 0x7fb2480aa400 2020-11-20T12:05:08.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.387+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb228003240 con 0x7fb2480aa400 2020-11-20T12:05:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3400363682 0 0) 0x7fb228003240 con 0x7fb2480aa400 2020-11-20T12:05:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 msgr2=0x7fb2480a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 0x7fb2480a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 >> v2:172.21.15.79:3300/0 conn(0x7fb2480aa020 msgr2=0x7fb2480a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 --2- 172.21.15.44:0/3712187206 >> v2:172.21.15.79:3300/0 conn(0x7fb2480aa020 0x7fb2480a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480a7040 con 0x7fb2480aa400 2020-11-20T12:05:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fb238001670 con 0x7fb2480aa400 2020-11-20T12:05:08.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 >> v1:172.21.15.44:6789/0 conn(0x7fb2480aa400 legacy=0x7fb2480a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb228003730 0x7fb228003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2280040b0 0x7fb228004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 0x7fb228004be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480a56c0 con 0x7fb228003730 2020-11-20T12:05:08.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480a6da0 con 0x7fb2280040b0 2020-11-20T12:05:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280051a0 con 0x7fb24813ac60 2020-11-20T12:05:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24659c700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb228003730 0x7fb228003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb246d9d700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 0x7fb228004be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb245d9b700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2280040b0 0x7fb228004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24659c700 1 -- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 msgr2=0x7fb228004be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24659c700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb24813ac60 0x7fb228004be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24659c700 1 -- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2280040b0 msgr2=0x7fb228004580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24659c700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb2280040b0 0x7fb228004580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24659c700 1 -- 172.21.15.44:0/3712187206 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480a7040 con 0x7fb228003730 2020-11-20T12:05:08.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24659c700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb228003730 0x7fb228003b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb23000bc00 tx=0x7fb230006df0).ready entity=mon.0 client_cookie=6dc359f83183a285 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000b310 con 0x7fb228003730 2020-11-20T12:05:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23000ea90 con 0x7fb228003730 2020-11-20T12:05:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.388+0000 7fb24559a700 1 -- 172.21.15.44:0/3712187206 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000ee10 con 0x7fb228003730 2020-11-20T12:05:08.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.389+0000 7fb24cac0700 1 -- 172.21.15.44:0/3712187206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb228003730 msgr2=0x7fb228003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.389+0000 7fb24cac0700 1 --2- 172.21.15.44:0/3712187206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb228003730 0x7fb228003b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb23000bc00 tx=0x7fb230006df0).stop 2020-11-20T12:05:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.389+0000 7fb246d9d700 1 -- 172.21.15.44:0/3712187206 reap_dead start 2020-11-20T12:05:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.389+0000 7fb24cac0700 1 -- 172.21.15.44:0/3712187206 shutdown_connections 2020-11-20T12:05:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.389+0000 7fb24cac0700 1 -- 172.21.15.44:0/3712187206 >> 172.21.15.44:0/3712187206 conn(0x7fb248062690 msgr2=0x7fb2480a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:08.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.389+0000 7fb24cac0700 1 -- 172.21.15.44:0/3712187206 shutdown_connections 2020-11-20T12:05:08.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.389+0000 7fb24cac0700 1 -- 172.21.15.44:0/3712187206 wait complete. 2020-11-20T12:05:08.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.390+0000 7fb24cac0700 1 Processor -- start 2020-11-20T12:05:08.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.390+0000 7fb24cac0700 1 -- start start 2020-11-20T12:05:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb228003730 0x7fb2480ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb248067ae0 0x7fb24812fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb248067f40 0x7fb248130140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480ac4b0 con 0x7fb248067f40 2020-11-20T12:05:08.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb248146d20 con 0x7fb248067ae0 2020-11-20T12:05:08.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb248138860 con 0x7fb228003730 2020-11-20T12:05:08.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb248067f40 0x7fb248130140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb248067f40 0x7fb248130140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37552/0 (socket says 172.21.15.44:37552) 2020-11-20T12:05:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 -- 172.21.15.44:0/3850583363 learned_addr learned my addr 172.21.15.44:0/3850583363 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb245d9b700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb248067ae0 0x7fb24812fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24659c700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb228003730 0x7fb2480ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 -- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb228003730 msgr2=0x7fb2480ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb228003730 0x7fb2480ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 -- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb248067ae0 msgr2=0x7fb24812fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb248067ae0 0x7fb24812fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 -- 172.21.15.44:0/3850583363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480a75a0 con 0x7fb248067f40 2020-11-20T12:05:08.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb246d9d700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb248067f40 0x7fb248130140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb2400090e0 tx=0x7fb2400091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb23e7fc700 1 -- 172.21.15.44:0/3850583363 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000faa0 con 0x7fb248067f40 2020-11-20T12:05:08.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb23e7fc700 1 -- 172.21.15.44:0/3850583363 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24000fc00 con 0x7fb248067f40 2020-11-20T12:05:08.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2480ad4a0 con 0x7fb248067f40 2020-11-20T12:05:08.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb23e7fc700 1 -- 172.21.15.44:0/3850583363 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240014440 con 0x7fb248067f40 2020-11-20T12:05:08.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.391+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2480ad7a0 con 0x7fb248067f40 2020-11-20T12:05:08.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.392+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb248130750 con 0x7fb248067f40 2020-11-20T12:05:08.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.392+0000 7fb23e7fc700 1 -- 172.21.15.44:0/3850583363 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb240014830 con 0x7fb248067f40 2020-11-20T12:05:08.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.393+0000 7fb23e7fc700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb224056d20 0x7fb2240591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.393+0000 7fb23e7fc700 1 -- 172.21.15.44:0/3850583363 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb240070450 con 0x7fb248067f40 2020-11-20T12:05:08.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.393+0000 7fb24659c700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb224056d20 0x7fb2240591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.393+0000 7fb24659c700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb224056d20 0x7fb2240591e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb230006e30 tx=0x7fb230019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.397+0000 7fb23e7fc700 1 -- 172.21.15.44:0/3850583363 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb240045740 con 0x7fb248067f40 2020-11-20T12:05:08.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.642+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb2480a75a0 con 0x7fb248067f40 2020-11-20T12:05:08.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.643+0000 7fb23e7fc700 1 -- 172.21.15.44:0/3850583363 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fb2480a75a0 con 0x7fb248067f40 2020-11-20T12:05:08.647 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":18,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:05:08.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb224056d20 msgr2=0x7fb2240591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb224056d20 0x7fb2240591e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb230006e30 tx=0x7fb230019000).stop 2020-11-20T12:05:08.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb248067f40 msgr2=0x7fb248130140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb248067f40 0x7fb248130140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb2400090e0 tx=0x7fb2400091d0).stop 2020-11-20T12:05:08.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 shutdown_connections 2020-11-20T12:05:08.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb224056d20 0x7fb2240591e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb248067f40 0x7fb248130140 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb228003730 0x7fb2480ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 --2- 172.21.15.44:0/3850583363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb248067ae0 0x7fb24812fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 >> 172.21.15.44:0/3850583363 conn(0x7fb248062690 msgr2=0x7fb2480a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:08.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 shutdown_connections 2020-11-20T12:05:08.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.644+0000 7fb24cac0700 1 -- 172.21.15.44:0/3850583363 wait complete. 2020-11-20T12:05:08.659 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:05:08.660 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:05:08.660 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:05:08.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.786+0000 7fa0fe869700 1 Processor -- start 2020-11-20T12:05:08.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.786+0000 7fa0fe869700 1 -- start start 2020-11-20T12:05:08.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fe869700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa0f80a7810 0x7fa0f813c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fe869700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa0f8066250 0x7fa0f813c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fe869700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f80a8bf0 con 0x7fa0f80aa1e0 2020-11-20T12:05:08.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fe869700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f80a86b0 con 0x7fa0f8066250 2020-11-20T12:05:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fe869700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f80a8950 con 0x7fa0f80a7810 2020-11-20T12:05:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fce06700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa0f8066250 0x7fa0f813c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0f7fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa0f80aa1e0 0x7fa0f8065b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41304/0 (socket says 172.21.15.44:41304) 2020-11-20T12:05:08.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0f7fff700 1 -- 172.21.15.44:0/2425140812 learned_addr learned my addr 172.21.15.44:0/2425140812 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0f77fe700 1 --2- 172.21.15.44:0/2425140812 >> v2:172.21.15.79:3300/0 conn(0x7fa0f80a7810 0x7fa0f813c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fce06700 1 -- 172.21.15.44:0/2425140812 >> v2:172.21.15.79:3300/0 conn(0x7fa0f80a7810 msgr2=0x7fa0f813c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fce06700 1 --2- 172.21.15.44:0/2425140812 >> v2:172.21.15.79:3300/0 conn(0x7fa0f80a7810 0x7fa0f813c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fce06700 1 -- 172.21.15.44:0/2425140812 >> v1:172.21.15.44:6789/0 conn(0x7fa0f80aa1e0 legacy=0x7fa0f8065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.787+0000 7fa0fce06700 1 -- 172.21.15.44:0/2425140812 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f80aa5c0 con 0x7fa0f8066250 2020-11-20T12:05:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0f77fe700 1 --2- 172.21.15.44:0/2425140812 >> v2:172.21.15.79:3300/0 conn(0x7fa0f80a7810 0x7fa0f813c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:05:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fce06700 1 --2- 172.21.15.44:0/2425140812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813c660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fa0f8062c40 tx=0x7fa0f00091d0).ready entity=mon.0 client_cookie=36761f2a30187cf3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0f6ffd700 1 -- 172.21.15.44:0/2425140812 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000bbc0 con 0x7fa0f8066250 2020-11-20T12:05:08.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0f6ffd700 1 -- 172.21.15.44:0/2425140812 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f000bd20 con 0x7fa0f8066250 2020-11-20T12:05:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0f6ffd700 1 -- 172.21.15.44:0/2425140812 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011610 con 0x7fa0f8066250 2020-11-20T12:05:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 -- 172.21.15.44:0/2425140812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 msgr2=0x7fa0f813c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 --2- 172.21.15.44:0/2425140812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813c660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fa0f8062c40 tx=0x7fa0f00091d0).stop 2020-11-20T12:05:08.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 -- 172.21.15.44:0/2425140812 shutdown_connections 2020-11-20T12:05:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 --2- 172.21.15.44:0/2425140812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813c660 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 --2- 172.21.15.44:0/2425140812 >> v2:172.21.15.79:3300/0 conn(0x7fa0f80a7810 0x7fa0f813c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 -- 172.21.15.44:0/2425140812 >> 172.21.15.44:0/2425140812 conn(0x7fa0f8062690 msgr2=0x7fa0f80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 -- 172.21.15.44:0/2425140812 shutdown_connections 2020-11-20T12:05:08.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.788+0000 7fa0fe869700 1 -- 172.21.15.44:0/2425140812 wait complete. 2020-11-20T12:05:08.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.789+0000 7fa0fe869700 1 Processor -- start 2020-11-20T12:05:08.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.789+0000 7fa0fe869700 1 -- start start 2020-11-20T12:05:08.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0fe869700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0fe869700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0f80a7810 0x7fa0f80a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0fe869700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0f80aa1e0 0x7fa0f80a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0fe869700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80a04c0 con 0x7fa0f8066250 2020-11-20T12:05:08.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0fe869700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80a0a00 con 0x7fa0f80a7810 2020-11-20T12:05:08.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0fe869700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80a0ca0 con 0x7fa0f80aa1e0 2020-11-20T12:05:08.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37562/0 (socket says 172.21.15.44:37562) 2020-11-20T12:05:08.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 -- 172.21.15.44:0/939126684 learned_addr learned my addr 172.21.15.44:0/939126684 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:08.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f77fe700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0f80a7810 0x7fa0f80a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0fce06700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0f80aa1e0 0x7fa0f80a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0f80aa1e0 msgr2=0x7fa0f80a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0f80aa1e0 0x7fa0f80a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0f80a7810 msgr2=0x7fa0f80a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0f80a7810 0x7fa0f80a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.790+0000 7fa0f7fff700 1 -- 172.21.15.44:0/939126684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f80a0f40 con 0x7fa0f8066250 2020-11-20T12:05:08.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.791+0000 7fa0f7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813fa80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa0e800c710 tx=0x7fa0e8007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.791+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80072e0 con 0x7fa0f8066250 2020-11-20T12:05:08.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.791+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e8007440 con 0x7fa0f8066250 2020-11-20T12:05:08.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.791+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f80a06f0 con 0x7fa0f8066250 2020-11-20T12:05:08.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.791+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8005800 con 0x7fa0f8066250 2020-11-20T12:05:08.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.791+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f8114aa0 con 0x7fa0f8066250 2020-11-20T12:05:08.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0fe869700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f81390a0 0x7fa0f8139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f809f970 con 0x7fa0f81390a0 2020-11-20T12:05:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0f77fe700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f81390a0 0x7fa0f8139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa0e801a060 con 0x7fa0f8066250 2020-11-20T12:05:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0f77fe700 1 -- 172.21.15.44:0/939126684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0f80a0880 con 0x7fa0f81390a0 2020-11-20T12:05:08.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0f77fe700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f81390a0 0x7fa0f8139480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa0ec006fd0 tx=0x7fa0ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa0d0056d20 0x7fa0d00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa0e806f370 con 0x7fa0f8066250 2020-11-20T12:05:08.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00d9e0 con 0x7fa0f81390a0 2020-11-20T12:05:08.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0fce06700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa0d0056d20 0x7fa0d00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fa0ec00c040 con 0x7fa0f81390a0 2020-11-20T12:05:08.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f81390a0 msgr2=0x7fa0f8139480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f81390a0 0x7fa0f8139480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa0ec006fd0 tx=0x7fa0ec008040).stop 2020-11-20T12:05:08.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f81390a0 msgr2=0x7fa0f8139480 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:08.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.792+0000 7fa0d7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f81390a0 0x7fa0f8139480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.793+0000 7fa0fce06700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa0d0056d20 0x7fa0d00591e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fa0f0009210 tx=0x7fa0f0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.800+0000 7fa0fe869700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f809e440 0x7fa0f809e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.800+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8114cf0 con 0x7fa0f809e440 2020-11-20T12:05:08.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.800+0000 7fa0f77fe700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f809e440 0x7fa0f809e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0f77fe700 1 -- 172.21.15.44:0/939126684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa0f80aade0 con 0x7fa0f809e440 2020-11-20T12:05:08.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0f77fe700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f809e440 0x7fa0f809e850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa0ec008010 tx=0x7fa0ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec013670 con 0x7fa0f809e440 2020-11-20T12:05:08.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4268 (secure 0 0 0) 0x7fa0ec00d7d0 con 0x7fa0f809e440 2020-11-20T12:05:08.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f809e440 msgr2=0x7fa0f809e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0d7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f809e440 0x7fa0f809e850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa0ec008010 tx=0x7fa0ec008040).stop 2020-11-20T12:05:08.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0d7fff700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f809e440 msgr2=0x7fa0f809e850 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:08.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0d7fff700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f809e440 0x7fa0f809e850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa0d0056d20 msgr2=0x7fa0d00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0fe869700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa0d0056d20 0x7fa0d00591e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fa0f0009210 tx=0x7fa0f0016000).stop 2020-11-20T12:05:08.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 msgr2=0x7fa0f813fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0fe869700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813fa80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa0e800c710 tx=0x7fa0e8007ca0).stop 2020-11-20T12:05:08.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.801+0000 7fa0fce06700 1 -- 172.21.15.44:0/939126684 reap_dead start 2020-11-20T12:05:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.802+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 shutdown_connections 2020-11-20T12:05:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.802+0000 7fa0fe869700 1 --2- 172.21.15.44:0/939126684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0f8066250 0x7fa0f813fa80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.802+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 >> 172.21.15.44:0/939126684 conn(0x7fa0f8062690 msgr2=0x7fa0f80a98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.802+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 shutdown_connections 2020-11-20T12:05:08.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.802+0000 7fa0fe869700 1 -- 172.21.15.44:0/939126684 wait complete. 2020-11-20T12:05:08.816 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:05:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:05:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:05:08.816 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:05:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 18, 2020-11-20T12:05:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:05:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:05:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:05:08.817 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:05:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 11, 2020-11-20T12:05:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:05:08.818 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:05:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:08.819 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:08.820 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:05:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:05:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:08.821 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:08.822 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:05:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:05:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:05:08.823 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:05:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:05:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:05:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:08.824 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:05:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:05:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:05:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:05:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:05:08.825 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:05:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:08.826 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:08.827 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:05:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:05:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.828 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:05:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:05:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.829 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:05:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.830 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:05:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:08.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:05:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:05:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:08.832 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:05:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:05:08.833 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:05:08.834 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:05:08.835 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:05:08.836 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:05:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:05:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:05:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:08.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:05:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:05:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:08.840 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:05:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:05:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:08.841 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.842 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:05:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:05:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:08.843 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:05:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:08.844 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:05:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:08.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:08.846 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:05:08.846 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:05:08.847 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:05:08.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.972+0000 7f75e7476700 1 Processor -- start 2020-11-20T12:05:08.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.972+0000 7f75e7476700 1 -- start start 2020-11-20T12:05:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e7476700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e7476700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f75e013ac60 0x7f75e00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e7476700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e00a9770 con 0x7f75e00aa020 2020-11-20T12:05:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e7476700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00a56c0 con 0x7f75e013ac60 2020-11-20T12:05:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e7476700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00a6da0 con 0x7f75e00aa400 2020-11-20T12:05:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e4a11700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e5a13700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f75e013ac60 0x7f75e00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e4a11700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e00a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49014/0 (socket says 172.21.15.44:49014) 2020-11-20T12:05:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e5212700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f75e00aa020 0x7f75e00a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44198/0 (socket says 172.21.15.44:44198) 2020-11-20T12:05:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e4a11700 1 -- 172.21.15.44:0/3053723629 learned_addr learned my addr 172.21.15.44:0/3053723629 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e4a11700 1 -- 172.21.15.44:0/3053723629 >> v2:172.21.15.79:3300/0 conn(0x7f75e013ac60 msgr2=0x7f75e00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e4a11700 1 --2- 172.21.15.44:0/3053723629 >> v2:172.21.15.79:3300/0 conn(0x7f75e013ac60 0x7f75e00a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e4a11700 1 -- 172.21.15.44:0/3053723629 >> v1:172.21.15.79:6789/0 conn(0x7f75e00aa020 legacy=0x7f75e00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.973+0000 7f75e4a11700 1 -- 172.21.15.44:0/3053723629 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00a7040 con 0x7f75e00aa400 2020-11-20T12:05:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e4a11700 1 --2- 172.21.15.44:0/3053723629 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e00a9280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f75d4006fd0 tx=0x7f75d4006f40).ready entity=mon.2 client_cookie=597bd1b93a1dee82 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75d3fff700 1 -- 172.21.15.44:0/3053723629 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400dc60 con 0x7f75e00aa400 2020-11-20T12:05:08.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75d3fff700 1 -- 172.21.15.44:0/3053723629 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d400ddc0 con 0x7f75e00aa400 2020-11-20T12:05:08.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75d3fff700 1 -- 172.21.15.44:0/3053723629 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4010680 con 0x7f75e00aa400 2020-11-20T12:05:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 -- 172.21.15.44:0/3053723629 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 msgr2=0x7f75e00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 --2- 172.21.15.44:0/3053723629 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e00a9280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f75d4006fd0 tx=0x7f75d4006f40).stop 2020-11-20T12:05:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 -- 172.21.15.44:0/3053723629 shutdown_connections 2020-11-20T12:05:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 --2- 172.21.15.44:0/3053723629 >> v2:172.21.15.79:3300/0 conn(0x7f75e013ac60 0x7f75e00a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 --2- 172.21.15.44:0/3053723629 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e00a9280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 -- 172.21.15.44:0/3053723629 >> 172.21.15.44:0/3053723629 conn(0x7f75e0062690 msgr2=0x7f75e00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 -- 172.21.15.44:0/3053723629 shutdown_connections 2020-11-20T12:05:08.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.974+0000 7f75e7476700 1 -- 172.21.15.44:0/3053723629 wait complete. 2020-11-20T12:05:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.975+0000 7f75e7476700 1 Processor -- start 2020-11-20T12:05:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.975+0000 7f75e7476700 1 -- start start 2020-11-20T12:05:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e7476700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e00aa020 0x7f75e01372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e7476700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e0067310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e7476700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e013ac60 0x7f75e00667c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e7476700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0067f10 con 0x7f75e00aa020 2020-11-20T12:05:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e7476700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00679d0 con 0x7f75e013ac60 2020-11-20T12:05:08.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e7476700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0067c70 con 0x7f75e00aa400 2020-11-20T12:05:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e5212700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e00aa020 0x7f75e01372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e5212700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e00aa020 0x7f75e01372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37580/0 (socket says 172.21.15.44:37580) 2020-11-20T12:05:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e5212700 1 -- 172.21.15.44:0/539128527 learned_addr learned my addr 172.21.15.44:0/539128527 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e4a11700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e0067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.976+0000 7f75e5a13700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e013ac60 0x7f75e00667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e5212700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 msgr2=0x7f75e0067310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e5212700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f75e00aa400 0x7f75e0067310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e5212700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e013ac60 msgr2=0x7f75e00667c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e5212700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e013ac60 0x7f75e00667c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:08.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e5212700 1 -- 172.21.15.44:0/539128527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00ac5a0 con 0x7f75e00aa020 2020-11-20T12:05:08.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e5212700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e00aa020 0x7f75e01372c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f75dc00c710 tx=0x7f75dc007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0072e0 con 0x7f75e00aa020 2020-11-20T12:05:08.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc005530 con 0x7f75e00aa020 2020-11-20T12:05:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e00ac7f0 con 0x7f75e00aa020 2020-11-20T12:05:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0058b0 con 0x7f75e00aa020 2020-11-20T12:05:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.977+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e0066ef0 con 0x7f75e00aa020 2020-11-20T12:05:08.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.978+0000 7f75e7476700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0131120 0x7f75e0131500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.978+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00a3f10 con 0x7f75e0131120 2020-11-20T12:05:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.978+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f75dc01a060 con 0x7f75e00aa020 2020-11-20T12:05:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.978+0000 7f75e4a11700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0131120 0x7f75e0131500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.978+0000 7f75d17fa700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75b8056d20 0x7f75b80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.978+0000 7f75e4a11700 1 -- 172.21.15.44:0/539128527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75e0066500 con 0x7f75e0131120 2020-11-20T12:05:08.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.978+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f75dc06f610 con 0x7f75e00aa020 2020-11-20T12:05:08.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75e4a11700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0131120 0x7f75e0131500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f75d4006fa0 tx=0x7f75e00a6970).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75e5a13700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75b8056d20 0x7f75b80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4018610 con 0x7f75e0131120 2020-11-20T12:05:08.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75e5a13700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75b8056d20 0x7f75b80591e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f75e00ac090 tx=0x7f75d80091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:08.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f75d40243f0 con 0x7f75e0131120 2020-11-20T12:05:08.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0131120 msgr2=0x7f75e0131500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:08.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75d17fa700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0131120 0x7f75e0131500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f75d4006fa0 tx=0x7f75e00a6970).stop 2020-11-20T12:05:08.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0131120 msgr2=0x7f75e0131500 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:08.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.979+0000 7f75d17fa700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0131120 0x7f75e0131500 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0x7f75d4006fa0 tx=0x7f75e00a6970).stop 2020-11-20T12:05:08.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.987+0000 7f75e7476700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0149220 0x7f75e0149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:08.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.987+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00a9770 con 0x7f75e0149220 2020-11-20T12:05:08.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.987+0000 7f75e4a11700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0149220 0x7f75e0149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:08.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75e4a11700 1 -- 172.21.15.44:0/539128527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f75e0067800 con 0x7f75e0149220 2020-11-20T12:05:09.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75e4a11700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0149220 0x7f75e0149630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f75e00a40a0 tx=0x7f75d4023000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:09.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4026ef0 con 0x7f75e0149220 2020-11-20T12:05:09.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f75d4018400 con 0x7f75e0149220 2020-11-20T12:05:09.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0149220 msgr2=0x7f75e0149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75d17fa700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0149220 0x7f75e0149630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f75e00a40a0 tx=0x7f75d4023000).stop 2020-11-20T12:05:09.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75d17fa700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0149220 msgr2=0x7f75e0149630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:09.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.988+0000 7f75d17fa700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75e0149220 0x7f75e0149630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:09.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75b8056d20 msgr2=0x7f75b80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75b8056d20 0x7f75b80591e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f75e00ac090 tx=0x7f75d80091d0).stop 2020-11-20T12:05:09.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e00aa020 msgr2=0x7f75e01372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 --2- 172.21.15.44:0/539128527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75e00aa020 0x7f75e01372c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f75dc00c710 tx=0x7f75dc007c40).stop 2020-11-20T12:05:09.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e5a13700 1 -- 172.21.15.44:0/539128527 reap_dead start 2020-11-20T12:05:09.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e5a13700 1 -- 172.21.15.44:0/539128527 reap_dead start 2020-11-20T12:05:09.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 shutdown_connections 2020-11-20T12:05:09.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 >> 172.21.15.44:0/539128527 conn(0x7f75e0062690 msgr2=0x7f75e00a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:09.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 shutdown_connections 2020-11-20T12:05:09.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:08.989+0000 7f75e7476700 1 -- 172.21.15.44:0/539128527 wait complete. 2020-11-20T12:05:09.004 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:05:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:05:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:05:09.004 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:05:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 18, 2020-11-20T12:05:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:05:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:05:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:05:09.005 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:05:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 11, 2020-11-20T12:05:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:09.006 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:05:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:05:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:09.007 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:09.008 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:05:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:05:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:09.009 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:09.010 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:05:09.011 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:05:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:05:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:05:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:05:09.012 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:05:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:05:09.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:05:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:05:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:05:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:05:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:09.014 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:05:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:09.015 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.016 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:05:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:05:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.017 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:05:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:05:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:05:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:05:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.020 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:05:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:05:09.021 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.022 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:05:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:05:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:05:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:05:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.025 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:05:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:05:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:09.026 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:05:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:05:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:05:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:05:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.030 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:05:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:05:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:09.031 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:05:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:05:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.033 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.034 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:05:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:05:09.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:05:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.036 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 8 2020-11-20T12:05:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.037 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:05:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.038 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:05:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.039 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:05:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:09.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.041 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:05:09.041 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:05:09.043 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:05:09.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.167+0000 7f3d93162700 1 Processor -- start 2020-11-20T12:05:09.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.167+0000 7f3d93162700 1 -- start start 2020-11-20T12:05:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.167+0000 7f3d93162700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3d8c0a9510 0x7f3d8c13b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.167+0000 7f3d93162700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0aabd0 0x7f3d8c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.167+0000 7f3d93162700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8c13ba40 con 0x7f3d8c0a9130 2020-11-20T12:05:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.167+0000 7f3d93162700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8c0ae3a0 con 0x7f3d8c0a9510 2020-11-20T12:05:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.167+0000 7f3d93162700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0ab630 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d8bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3d8c0a9510 0x7f3d8c13b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d916ff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0aabd0 0x7f3d8c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d90efe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3d8c0a9130 0x7f3d8c0adc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44216/0 (socket says 172.21.15.44:44216) 2020-11-20T12:05:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d90efe700 1 -- 172.21.15.44:0/4123114536 learned_addr learned my addr 172.21.15.44:0/4123114536 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d8b7fe700 1 -- 172.21.15.44:0/4123114536 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880700904 0 0) 0x7f3d800010d0 con 0x7f3d8c0a9130 2020-11-20T12:05:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d8b7fe700 1 -- 172.21.15.44:0/4123114536 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d70003240 con 0x7f3d8c0a9130 2020-11-20T12:05:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d916ff700 1 -- 172.21.15.44:0/4123114536 >> v2:172.21.15.79:3300/0 conn(0x7f3d8c0a9510 msgr2=0x7f3d8c13b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d916ff700 1 --2- 172.21.15.44:0/4123114536 >> v2:172.21.15.79:3300/0 conn(0x7f3d8c0a9510 0x7f3d8c13b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d916ff700 1 -- 172.21.15.44:0/4123114536 >> v1:172.21.15.79:6789/0 conn(0x7f3d8c0a9130 legacy=0x7f3d8c0adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d916ff700 1 -- 172.21.15.44:0/4123114536 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0ab110 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.168+0000 7f3d916ff700 1 --2- 172.21.15.44:0/4123114536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0aabd0 0x7f3d8c13b550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3d84009c90 tx=0x7f3d840091d0).ready entity=mon.2 client_cookie=2c8723ae06530b45 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d8b7fe700 1 -- 172.21.15.44:0/4123114536 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400b9c0 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d8b7fe700 1 -- 172.21.15.44:0/4123114536 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8400bb20 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d8b7fe700 1 -- 172.21.15.44:0/4123114536 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400bea0 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 -- 172.21.15.44:0/4123114536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0aabd0 msgr2=0x7f3d8c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 --2- 172.21.15.44:0/4123114536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0aabd0 0x7f3d8c13b550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f3d84009c90 tx=0x7f3d840091d0).stop 2020-11-20T12:05:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 -- 172.21.15.44:0/4123114536 shutdown_connections 2020-11-20T12:05:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 --2- 172.21.15.44:0/4123114536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0aabd0 0x7f3d8c13b550 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 --2- 172.21.15.44:0/4123114536 >> v2:172.21.15.79:3300/0 conn(0x7f3d8c0a9510 0x7f3d8c13b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 -- 172.21.15.44:0/4123114536 >> 172.21.15.44:0/4123114536 conn(0x7f3d8c060170 msgr2=0x7f3d8c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:09.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 -- 172.21.15.44:0/4123114536 shutdown_connections 2020-11-20T12:05:09.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.169+0000 7f3d93162700 1 -- 172.21.15.44:0/4123114536 wait complete. 2020-11-20T12:05:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.170+0000 7f3d93162700 1 Processor -- start 2020-11-20T12:05:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.170+0000 7f3d93162700 1 -- start start 2020-11-20T12:05:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d93162700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0a9130 0x7f3d8c0b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d93162700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d8c0a9510 0x7f3d8c09e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d93162700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d8c0aabd0 0x7f3d8c09ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d93162700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c138810 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d93162700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c09fb30 con 0x7f3d8c0a9510 2020-11-20T12:05:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d93162700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c09fdd0 con 0x7f3d8c0a9130 2020-11-20T12:05:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d8c0aabd0 0x7f3d8c09ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d8c0aabd0 0x7f3d8c09ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37600/0 (socket says 172.21.15.44:37600) 2020-11-20T12:05:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 -- 172.21.15.44:0/4052665380 learned_addr learned my addr 172.21.15.44:0/4052665380 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d90efe700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0a9130 0x7f3d8c0b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d8bfff700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d8c0a9510 0x7f3d8c09e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0a9130 msgr2=0x7f3d8c0b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d8c0a9130 0x7f3d8c0b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d8c0a9510 msgr2=0x7f3d8c09e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d8c0a9510 0x7f3d8c09e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 -- 172.21.15.44:0/4052665380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0a0070 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.171+0000 7f3d916ff700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d8c0aabd0 0x7f3d8c09ee90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3d84003340 tx=0x7f3d84016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.172+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8401c6c0 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.172+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8401c820 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.172+0000 7f3d93162700 1 -- 172.21.15.44:0/4052665380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c0a0310 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.172+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8401cbc0 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.172+0000 7f3d93162700 1 -- 172.21.15.44:0/4052665380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c0a1f80 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.172+0000 7f3d6e7fc700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c002a10 0x7f3d5c004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.172+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c007b70 con 0x7f3d5c002a10 2020-11-20T12:05:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d90efe700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c002a10 0x7f3d5c004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3d84017070 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d90efe700 1 -- 172.21.15.44:0/4052665380 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d5c007d30 con 0x7f3d5c002a10 2020-11-20T12:05:09.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d88ff9700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d64056d50 0x7f3d64059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3d840723f0 con 0x7f3d8c0aabd0 2020-11-20T12:05:09.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d90efe700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c002a10 0x7f3d5c004ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3d800055a0 tx=0x7f3d800078c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:09.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d8bfff700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d64056d50 0x7f3d64059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.173+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000f660 con 0x7f3d5c002a10 2020-11-20T12:05:09.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.174+0000 7f3d8bfff700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d64056d50 0x7f3d64059210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3d8c13f0a0 tx=0x7f3d78006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:09.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.174+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f3d8000f7c0 con 0x7f3d5c002a10 2020-11-20T12:05:09.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.174+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c002a10 msgr2=0x7f3d5c004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.174+0000 7f3d88ff9700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c002a10 0x7f3d5c004ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f3d800055a0 tx=0x7f3d800078c0).stop 2020-11-20T12:05:09.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.174+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c002a10 msgr2=0x7f3d5c004ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:09.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.174+0000 7f3d88ff9700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c002a10 0x7f3d5c004ed0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0x7f3d800055a0 tx=0x7f3d800078c0).stop 2020-11-20T12:05:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.181+0000 7f3d6e7fc700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c006c00 0x7f3d5c007090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.182+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0076a0 con 0x7f3d5c006c00 2020-11-20T12:05:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.182+0000 7f3d90efe700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c006c00 0x7f3d5c007090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.182+0000 7f3d90efe700 1 -- 172.21.15.44:0/4052665380 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3d5c007860 con 0x7f3d5c006c00 2020-11-20T12:05:09.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.182+0000 7f3d90efe700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c006c00 0x7f3d5c007090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3d80008fd0 tx=0x7f3d80019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:09.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.182+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800135c0 con 0x7f3d5c006c00 2020-11-20T12:05:09.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4436 (secure 0 0 0) 0x7f3d8000f450 con 0x7f3d5c006c00 2020-11-20T12:05:09.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c006c00 msgr2=0x7f3d5c007090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d88ff9700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c006c00 0x7f3d5c007090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f3d80008fd0 tx=0x7f3d80019000).stop 2020-11-20T12:05:09.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d88ff9700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c006c00 msgr2=0x7f3d5c007090 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:09.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d88ff9700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d5c006c00 0x7f3d5c007090 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:09.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d64056d50 msgr2=0x7f3d64059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d64056d50 0x7f3d64059210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3d8c13f0a0 tx=0x7f3d78006df0).stop 2020-11-20T12:05:09.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d8c0aabd0 msgr2=0x7f3d8c09ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 --2- 172.21.15.44:0/4052665380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d8c0aabd0 0x7f3d8c09ee90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3d84003340 tx=0x7f3d84016000).stop 2020-11-20T12:05:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d916ff700 1 -- 172.21.15.44:0/4052665380 reap_dead start 2020-11-20T12:05:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d916ff700 1 -- 172.21.15.44:0/4052665380 reap_dead start 2020-11-20T12:05:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 shutdown_connections 2020-11-20T12:05:09.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 >> 172.21.15.44:0/4052665380 conn(0x7f3d8c060170 msgr2=0x7f3d8c05ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 shutdown_connections 2020-11-20T12:05:09.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:09.183+0000 7f3d6e7fc700 1 -- 172.21.15.44:0/4052665380 wait complete. 2020-11-20T12:05:09.196 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:05:09.196 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:05:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:05:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:05:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 18, 2020-11-20T12:05:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:05:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:05:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:05:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:05:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.198 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 11, 2020-11-20T12:05:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:05:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:05:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:09.199 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:09.200 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:05:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:05:09.201 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:09.202 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:05:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:05:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:05:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:05:09.204 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:05:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:05:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:05:09.205 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:05:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:05:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:05:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:05:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:09.206 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:05:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:09.207 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:09.208 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:05:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:05:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.209 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:05:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:05:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:05:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:05:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:05:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:05:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:05:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:05:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:09.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:05:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:09.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:05:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.217 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:05:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:05:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:09.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:05:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:05:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:09.219 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:05:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:05:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:09.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:05:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:05:09.222 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:05:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:05:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.224 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.225 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:05:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:09.226 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:05:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:09.227 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:09.228 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:05:09.229 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:05:09.230 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:05:09.230 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2020-11-20T12:05:09.231 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:05:09.231 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:05:09.231 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:05:09.231 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:05:09.246 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:05:09.247 INFO:tasks.mon_thrash:Sending STOP to mon c 2020-11-20T12:05:09.247 INFO:tasks.ceph.mon.c:Sent signal 19 2020-11-20T12:05:09.247 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2020-11-20T12:05:24.248 INFO:tasks.mon_thrash:Sending CONT to mon c 2020-11-20T12:05:24.249 INFO:tasks.ceph.mon.c:Sent signal 18 2020-11-20T12:05:24.249 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:05:24.249 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:05:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.378+0000 7fa7837e8700 1 Processor -- start 2020-11-20T12:05:24.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.378+0000 7fa7837e8700 1 -- start start 2020-11-20T12:05:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa7837e8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa7837e8700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0a9770 con 0x7fa77c13ac60 2020-11-20T12:05:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa7837e8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0a6da0 con 0x7fa77c0aa020 2020-11-20T12:05:24.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa7837e8700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0a56c0 con 0x7fa77c0aa400 2020-11-20T12:05:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa780d83700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa781584700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fa77c0aa020 legacy=0x7fa77c0a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa780d83700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37666/0 (socket says 172.21.15.44:37666) 2020-11-20T12:05:24.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 learned_addr learned my addr 172.21.15.44:0/1009110501 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa781d85700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa77c13ac60 0x7fa77c0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41416/0 (socket says 172.21.15.44:41416) 2020-11-20T12:05:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:05:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:24.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.379+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.580+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.79:6789/0 conn(0x7fa77c0aa020 legacy=0x7fa77c0a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.580+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.580+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:05:24.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.580+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.580+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:24.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.580+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:05:24.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.981+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:24.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.981+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.79:6789/0 conn(0x7fa77c0aa020 legacy=0x7fa77c0a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:24.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.981+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:05:24.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.981+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:24.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.981+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:24.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:24.981+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:05:25.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:25.782+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.79:6789/0 conn(0x7fa77c0aa020 legacy=0x7fa77c0a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:25.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:25.782+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:25.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:25.782+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:05:25.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:25.782+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:25.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:25.782+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:25.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:25.782+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:05:27.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 msgr2=0x7fa77c0a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.44:3300/0 conn(0x7fa77c0aa400 0x7fa77c0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:27.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.79:6789/0 conn(0x7fa77c0aa020 legacy=0x7fa77c0a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.44:6789/0 conn(0x7fa77c13ac60 legacy=0x7fa77c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c005810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:27.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0a56c0 con 0x7fa75c0033b0 2020-11-20T12:05:27.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c008950 con 0x7fa75c005db0 2020-11-20T12:05:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 msgr2=0x7fa75c005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.379+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:27.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.380+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.380+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:05:27.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.380+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:27.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.380+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:27.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.380+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:27.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.581+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 msgr2=0x7fa75c005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:27.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.581+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:05:27.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.581+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.581+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:05:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.581+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:27.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.581+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:27.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.581+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:05:27.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.982+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 msgr2=0x7fa75c005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:27.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.982+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:05:27.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.982+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:27.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.982+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:05:27.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.982+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:27.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.982+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:27.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:27.982+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:05:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:28.783+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 msgr2=0x7fa75c005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:28.783+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:05:28.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:28.783+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:28.783+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:05:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:28.783+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:05:28.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:28.783+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:05:28.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:28.783+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:05:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.380+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 msgr2=0x7fa75c008270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.380+0000 7fa76effd700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa75c005db0 0x7fa75c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 msgr2=0x7fa75c005810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:30.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76effd700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa781d85700 1 -- 172.21.15.44:0/1009110501 reap_dead start 2020-11-20T12:05:30.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76effd700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c0057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0a7040 con 0x7fa75c00a490 2020-11-20T12:05:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0a6da0 con 0x7fa75c006260 2020-11-20T12:05:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76effd700 1 -- 172.21.15.44:0/1009110501 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0a56c0 con 0x7fa75c0033b0 2020-11-20T12:05:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa780d83700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.79:6789/0 conn(0x7fa75c006260 legacy=0x7fa75c008d70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:30.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 msgr2=0x7fa75c0057d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:30.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c0057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:30.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536549699 0 0) 0x7fa77c0a7040 con 0x7fa75c00a490 2020-11-20T12:05:30.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.381+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa764003240 con 0x7fa75c00a490 2020-11-20T12:05:30.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1803839635 0 0) 0x7fa764003240 con 0x7fa75c00a490 2020-11-20T12:05:30.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 msgr2=0x7fa75c0057d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:30.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 --2- 172.21.15.44:0/1009110501 >> v2:172.21.15.79:3300/0 conn(0x7fa75c0033b0 0x7fa75c0057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.79:6789/0 conn(0x7fa75c006260 legacy=0x7fa75c008d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:30.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c010320 con 0x7fa75c00a490 2020-11-20T12:05:30.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fa7740016d0 con 0x7fa75c00a490 2020-11-20T12:05:30.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 >> v1:172.21.15.44:6789/0 conn(0x7fa75c00a490 legacy=0x7fa75c00c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa764003730 0x7fa764003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa764004040 0x7fa764004520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764004bc0 0x7fa76400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0a56c0 con 0x7fa764003730 2020-11-20T12:05:30.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764005260 con 0x7fa764004bc0 2020-11-20T12:05:30.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76400bc20 con 0x7fa764004040 2020-11-20T12:05:30.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa764003730 0x7fa764003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781d85700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764004bc0 msgr2=0x7fa76400b730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:30.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781d85700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764004bc0 0x7fa76400b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:30.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa780d83700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa764004040 0x7fa764004520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa764004040 msgr2=0x7fa764004520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:30.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa764004040 0x7fa764004520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764004bc0 msgr2=0x7fa76400b730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:30.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764004bc0 0x7fa76400b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781584700 1 -- 172.21.15.44:0/1009110501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764003240 con 0x7fa764003730 2020-11-20T12:05:30.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781d85700 1 -- 172.21.15.44:0/1009110501 reap_dead start 2020-11-20T12:05:30.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.382+0000 7fa781584700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa764003730 0x7fa764003b30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa77800b550 tx=0x7fa7780096e0).ready entity=mon.0 client_cookie=e47f9b76d9745045 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778006170 con 0x7fa764003730 2020-11-20T12:05:30.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7780062f0 con 0x7fa764003730 2020-11-20T12:05:30.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa76ffff700 1 -- 172.21.15.44:0/1009110501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780066b0 con 0x7fa764003730 2020-11-20T12:05:30.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa7837e8700 1 -- 172.21.15.44:0/1009110501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa764003730 msgr2=0x7fa764003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa764003730 0x7fa764003b30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa77800b550 tx=0x7fa7780096e0).stop 2020-11-20T12:05:30.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa7837e8700 1 -- 172.21.15.44:0/1009110501 shutdown_connections 2020-11-20T12:05:30.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1009110501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa764003730 0x7fa764003b30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.383+0000 7fa7837e8700 1 -- 172.21.15.44:0/1009110501 >> 172.21.15.44:0/1009110501 conn(0x7fa77c062690 msgr2=0x7fa77c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:30.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.384+0000 7fa7837e8700 1 -- 172.21.15.44:0/1009110501 shutdown_connections 2020-11-20T12:05:30.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.384+0000 7fa7837e8700 1 -- 172.21.15.44:0/1009110501 wait complete. 2020-11-20T12:05:30.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.385+0000 7fa7837e8700 1 Processor -- start 2020-11-20T12:05:30.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.385+0000 7fa7837e8700 1 -- start start 2020-11-20T12:05:30.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa7837e8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764003730 0x7fa77c0a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa7837e8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa77c0ac000 0x7fa77c0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa7837e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77c0acea0 0x7fa77c12ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa7837e8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c066b60 con 0x7fa77c0acea0 2020-11-20T12:05:30.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa7837e8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0adb70 con 0x7fa764003730 2020-11-20T12:05:30.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa7837e8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c066620 con 0x7fa77c0ac000 2020-11-20T12:05:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781584700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764003730 msgr2=0x7fa77c0a76f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77c0acea0 0x7fa77c12ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781584700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764003730 0x7fa77c0a76f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:30.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77c0acea0 0x7fa77c12ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37714/0 (socket says 172.21.15.44:37714) 2020-11-20T12:05:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 -- 172.21.15.44:0/1465535578 learned_addr learned my addr 172.21.15.44:0/1465535578 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa780d83700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa77c0ac000 0x7fa77c0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 -- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa77c0ac000 msgr2=0x7fa77c0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa77c0ac000 0x7fa77c0ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 -- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764003730 msgr2=0x7fa77c0a76f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:05:30.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764003730 0x7fa77c0a76f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.386+0000 7fa781d85700 1 -- 172.21.15.44:0/1465535578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0668c0 con 0x7fa77c0acea0 2020-11-20T12:05:30.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.387+0000 7fa781d85700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77c0acea0 0x7fa77c12ebc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa77c13b040 tx=0x7fa774007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.387+0000 7fa76d7fa700 1 -- 172.21.15.44:0/1465535578 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774004a90 con 0x7fa77c0acea0 2020-11-20T12:05:30.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.387+0000 7fa76d7fa700 1 -- 172.21.15.44:0/1465535578 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa774004bf0 con 0x7fa77c0acea0 2020-11-20T12:05:30.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.387+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c0662e0 con 0x7fa77c0acea0 2020-11-20T12:05:30.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.387+0000 7fa76d7fa700 1 -- 172.21.15.44:0/1465535578 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774007030 con 0x7fa77c0acea0 2020-11-20T12:05:30.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.387+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c12f0b0 con 0x7fa77c0acea0 2020-11-20T12:05:30.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.388+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c0a8ae0 con 0x7fa77c0acea0 2020-11-20T12:05:30.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.388+0000 7fa76d7fa700 1 -- 172.21.15.44:0/1465535578 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa77401b060 con 0x7fa77c0acea0 2020-11-20T12:05:30.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.388+0000 7fa76d7fa700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa754056d00 0x7fa7540591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.389+0000 7fa76d7fa700 1 -- 172.21.15.44:0/1465535578 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa77406f790 con 0x7fa77c0acea0 2020-11-20T12:05:30.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.389+0000 7fa781584700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa754056d00 0x7fa7540591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.389+0000 7fa781584700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa754056d00 0x7fa7540591c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa778003940 tx=0x7fa778015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.393+0000 7fa76d7fa700 1 -- 172.21.15.44:0/1465535578 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa774044a80 con 0x7fa77c0acea0 2020-11-20T12:05:30.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.636+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa77c0a9770 con 0x7fa77c0acea0 2020-11-20T12:05:30.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.637+0000 7fa76d7fa700 1 -- 172.21.15.44:0/1465535578 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fa77c0a9770 con 0x7fa77c0acea0 2020-11-20T12:05:30.640 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":20,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:05:30.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa754056d00 msgr2=0x7fa7540591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa754056d00 0x7fa7540591c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa778003940 tx=0x7fa778015000).stop 2020-11-20T12:05:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77c0acea0 msgr2=0x7fa77c12ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77c0acea0 0x7fa77c12ebc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa77c13b040 tx=0x7fa774007a90).stop 2020-11-20T12:05:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 shutdown_connections 2020-11-20T12:05:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa754056d00 0x7fa7540591c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa77c0acea0 0x7fa77c12ebc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa764003730 0x7fa77c0a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 --2- 172.21.15.44:0/1465535578 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa77c0ac000 0x7fa77c0ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 >> 172.21.15.44:0/1465535578 conn(0x7fa77c062690 msgr2=0x7fa77c060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:30.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 shutdown_connections 2020-11-20T12:05:30.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.638+0000 7fa7837e8700 1 -- 172.21.15.44:0/1465535578 wait complete. 2020-11-20T12:05:30.652 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:05:30.653 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:05:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.778+0000 7f9803e1b700 1 Processor -- start 2020-11-20T12:05:30.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.778+0000 7f9803e1b700 1 -- start start 2020-11-20T12:05:30.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9803e1b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9803e1b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc1385f0 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9803e1b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc0aca50 con 0x7f97fc0a86d0 2020-11-20T12:05:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9803e1b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc13e250 con 0x7f97fc0aad00 2020-11-20T12:05:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f98023b8700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f97fc0a86d0 legacy=0x7f97fc0ac340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f98013b6700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f97fc0ab0e0 0x7f97fc0abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41468/0 (socket says 172.21.15.44:41468) 2020-11-20T12:05:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f98013b6700 1 -- 172.21.15.44:0/3436298414 learned_addr learned my addr 172.21.15.44:0/3436298414 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9801bb7700 1 --2- 172.21.15.44:0/3436298414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9800bb5700 1 -- 172.21.15.44:0/3436298414 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511134309 0 0) 0x7f97fc1385f0 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9800bb5700 1 -- 172.21.15.44:0/3436298414 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e4003240 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9801bb7700 1 -- 172.21.15.44:0/3436298414 >> v1:172.21.15.79:6789/0 conn(0x7f97fc0a86d0 legacy=0x7f97fc0ac340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9801bb7700 1 -- 172.21.15.44:0/3436298414 >> v1:172.21.15.44:6789/0 conn(0x7f97fc0ab0e0 legacy=0x7f97fc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9801bb7700 1 -- 172.21.15.44:0/3436298414 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc13e4f0 con 0x7f97fc0aad00 2020-11-20T12:05:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9801bb7700 1 --2- 172.21.15.44:0/3436298414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a94d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f97f800b9d0 tx=0x7f97f8006df0).ready entity=mon.2 client_cookie=d2d25920146c8bfd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9800bb5700 1 -- 172.21.15.44:0/3436298414 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8009800 con 0x7f97fc0aad00 2020-11-20T12:05:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.779+0000 7f9800bb5700 1 -- 172.21.15.44:0/3436298414 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f8009960 con 0x7f97fc0aad00 2020-11-20T12:05:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9800bb5700 1 -- 172.21.15.44:0/3436298414 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8009ce0 con 0x7f97fc0aad00 2020-11-20T12:05:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9803e1b700 1 -- 172.21.15.44:0/3436298414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 msgr2=0x7f97fc0a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9803e1b700 1 --2- 172.21.15.44:0/3436298414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a94d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f97f800b9d0 tx=0x7f97f8006df0).stop 2020-11-20T12:05:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9803e1b700 1 -- 172.21.15.44:0/3436298414 shutdown_connections 2020-11-20T12:05:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9803e1b700 1 --2- 172.21.15.44:0/3436298414 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a94d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9803e1b700 1 -- 172.21.15.44:0/3436298414 >> 172.21.15.44:0/3436298414 conn(0x7f97fc060170 msgr2=0x7f97fc060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9803e1b700 1 -- 172.21.15.44:0/3436298414 shutdown_connections 2020-11-20T12:05:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.780+0000 7f9803e1b700 1 -- 172.21.15.44:0/3436298414 wait complete. 2020-11-20T12:05:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.781+0000 7f9803e1b700 1 Processor -- start 2020-11-20T12:05:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.781+0000 7f9803e1b700 1 -- start start 2020-11-20T12:05:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9803e1b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97fc0a86d0 0x7f97fc13a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9803e1b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9803e1b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc0ab0e0 0x7f97fc0a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9803e1b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0a7c90 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9803e1b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0a7f20 con 0x7f97fc0a86d0 2020-11-20T12:05:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9803e1b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0a6640 con 0x7f97fc0aad00 2020-11-20T12:05:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9801bb7700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97fc0a86d0 msgr2=0x7f97fc13a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc0ab0e0 0x7f97fc0a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f9801bb7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97fc0a86d0 0x7f97fc13a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc0ab0e0 0x7f97fc0a6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37728/0 (socket says 172.21.15.44:37728) 2020-11-20T12:05:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 -- 172.21.15.44:0/475083293 learned_addr learned my addr 172.21.15.44:0/475083293 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98013b6700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 msgr2=0x7f97fc0a77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f97fc0aad00 0x7f97fc0a77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97fc0a86d0 msgr2=0x7f97fc13a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:05:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97fc0a86d0 0x7f97fc13a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 -- 172.21.15.44:0/475083293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0a68e0 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f98023b8700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc0ab0e0 0x7f97fc0a6d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f97f400bae0 tx=0x7f97f4006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.782+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4003c10 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.783+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f4003d70 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.783+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc0a7270 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.783+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4006170 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.783+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc14a1b0 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.783+0000 7f9803e1b700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc09ee10 0x7f97fc14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.784+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc05c5f0 con 0x7f97fc09ee10 2020-11-20T12:05:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.784+0000 7f9801bb7700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc09ee10 0x7f97fc14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.784+0000 7f9801bb7700 1 -- 172.21.15.44:0/475083293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97fc069680 con 0x7f97fc09ee10 2020-11-20T12:05:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.784+0000 7f9801bb7700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc09ee10 0x7f97fc14a590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f97f800b9d0 tx=0x7f97f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.784+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8011dd0 con 0x7f97fc09ee10 2020-11-20T12:05:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.784+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f97f40065a0 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f97e9ffb700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f97d4056d60 0x7f97d4059220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f97f4011020 con 0x7f97fc0ab0e0 2020-11-20T12:05:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f97f801e460 con 0x7f97fc09ee10 2020-11-20T12:05:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc09ee10 msgr2=0x7f97fc14a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f97e9ffb700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc09ee10 0x7f97fc14a590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f97f800b9d0 tx=0x7f97f8015000).stop 2020-11-20T12:05:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc09ee10 msgr2=0x7f97fc14a590 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f97e9ffb700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc09ee10 0x7f97fc14a590 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f98013b6700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f97d4056d60 0x7f97d4059220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.785+0000 7f98013b6700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f97d4056d60 0x7f97d4059220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f97ec007fd0 tx=0x7f97ec009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f9803e1b700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc14aa80 0x7f97fc14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0446d0 con 0x7f97fc14aa80 2020-11-20T12:05:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f9801bb7700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc14aa80 0x7f97fc14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f9801bb7700 1 -- 172.21.15.44:0/475083293 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f97fc14ae60 con 0x7f97fc14aa80 2020-11-20T12:05:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f9801bb7700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc14aa80 0x7f97fc14d540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f97f8006e30 tx=0x7f97f801e000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f801d8b0 con 0x7f97fc14aa80 2020-11-20T12:05:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4766 (secure 0 0 0) 0x7f97f8011bc0 con 0x7f97fc14aa80 2020-11-20T12:05:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc14aa80 msgr2=0x7f97fc14d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f97e9ffb700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc14aa80 0x7f97fc14d540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f97f8006e30 tx=0x7f97f801e000).stop 2020-11-20T12:05:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f97e9ffb700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc14aa80 msgr2=0x7f97fc14d540 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.793+0000 7f97e9ffb700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc14aa80 0x7f97fc14d540 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0x7f97f8006e30 tx=0x7f97f801e000).stop 2020-11-20T12:05:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f97d4056d60 msgr2=0x7f97d4059220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f97d4056d60 0x7f97d4059220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f97ec007fd0 tx=0x7f97ec009040).stop 2020-11-20T12:05:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc0ab0e0 msgr2=0x7f97fc0a6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f98023b8700 1 -- 172.21.15.44:0/475083293 reap_dead start 2020-11-20T12:05:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc0ab0e0 0x7f97fc0a6d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f97f400bae0 tx=0x7f97f4006f90).stop 2020-11-20T12:05:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 shutdown_connections 2020-11-20T12:05:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 --2- 172.21.15.44:0/475083293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f97fc0ab0e0 0x7f97fc0a6d80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 >> 172.21.15.44:0/475083293 conn(0x7f97fc060170 msgr2=0x7f97fc0ad600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 shutdown_connections 2020-11-20T12:05:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.794+0000 7f9803e1b700 1 -- 172.21.15.44:0/475083293 wait complete. 2020-11-20T12:05:30.805 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:05:30.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:05:30.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:05:30.806 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:05:30.806 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 20, 2020-11-20T12:05:30.807 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:05:30.807 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:05:30.807 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:05:30.807 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.807 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:05:30.808 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:30.808 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:05:30.808 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:05:30.808 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:30.808 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:30.809 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:30.809 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:30.809 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:30.809 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:30.809 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:30.810 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:30.810 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.810 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:05:30.810 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:05:30.810 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:30.811 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:30.811 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:30.811 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:30.811 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:30.811 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.812 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:05:30.813 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:05:30.813 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:05:30.813 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:05:30.813 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:05:30.813 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:05:30.814 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:30.814 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:30.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:05:30.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:05:30.814 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:05:30.815 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:05:30.815 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:05:30.815 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:30.815 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:05:30.815 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:30.816 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:30.816 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:30.816 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:30.816 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:30.816 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:30.816 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:30.817 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:30.817 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.817 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:05:30.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.818 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:05:30.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.818 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:05:30.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:05:30.818 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:30.818 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:30.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:30.819 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:05:30.819 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:30.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:05:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.821 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:30.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.821 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:05:30.821 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:05:30.821 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:30.822 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:30.822 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:30.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.822 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:05:30.823 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:05:30.823 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:30.823 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:30.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.823 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:30.824 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:05:30.824 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:30.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.824 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:30.825 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:05:30.825 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:30.825 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.825 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:30.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.826 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:05:30.826 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:05:30.826 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:30.826 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:30.826 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:30.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.827 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:05:30.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:05:30.827 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:30.828 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:30.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:30.828 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:05:30.828 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:30.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.829 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:30.829 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:05:30.829 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:30.830 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.830 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:30.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.830 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:05:30.830 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:05:30.830 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:30.831 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:30.831 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:30.831 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.831 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:30.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.832 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:05:30.832 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:05:30.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.832 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:30.832 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:30.833 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:30.833 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.833 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.833 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:05:30.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.834 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:30.834 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:30.834 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:05:30.834 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.834 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.835 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:05:30.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.835 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:30.835 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:30.835 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:05:30.836 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.836 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.836 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:05:30.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.836 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:30.837 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:30.837 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:05:30.837 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.837 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.837 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:05:30.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:30.838 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:30.838 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:30.838 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:30.838 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:30.839 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:30.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:30.839 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:05:30.839 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:05:30.840 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:05:30.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.967+0000 7f66797dc700 1 Processor -- start 2020-11-20T12:05:30.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.967+0000 7f66797dc700 1 -- start start 2020-11-20T12:05:30.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66797dc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f66740aa400 0x7f66740a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66797dc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f667413ac60 0x7f66740a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66797dc700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66740a9770 con 0x7f66740aa020 2020-11-20T12:05:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66797dc700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66740a56c0 con 0x7f66740aa400 2020-11-20T12:05:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66797dc700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66740a6da0 con 0x7f667413ac60 2020-11-20T12:05:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66727fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f66740aa400 0x7f66740a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66737fe700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f667413ac60 msgr2=0x7f66740a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66737fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f667413ac60 0x7f66740a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f6672ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f66740aa020 0x7f66740a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41484/0 (socket says 172.21.15.44:41484) 2020-11-20T12:05:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f6672ffd700 1 -- 172.21.15.44:0/84855525 learned_addr learned my addr 172.21.15.44:0/84855525 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66727fc700 1 --2- 172.21.15.44:0/84855525 >> v2:172.21.15.44:3300/0 conn(0x7f66740aa400 0x7f66740a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37736/0 (socket says 172.21.15.44:37736) 2020-11-20T12:05:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66727fc700 1 -- 172.21.15.44:0/84855525 >> v2:172.21.15.79:3300/0 conn(0x7f667413ac60 msgr2=0x7f66740a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66727fc700 1 --2- 172.21.15.44:0/84855525 >> v2:172.21.15.79:3300/0 conn(0x7f667413ac60 0x7f66740a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66727fc700 1 -- 172.21.15.44:0/84855525 >> v1:172.21.15.44:6789/0 conn(0x7f66740aa020 legacy=0x7f66740a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f6671ffb700 1 -- 172.21.15.44:0/84855525 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353155887 0 0) 0x7f66680010d0 con 0x7f66740aa020 2020-11-20T12:05:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66727fc700 1 -- 172.21.15.44:0/84855525 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740a7040 con 0x7f66740aa400 2020-11-20T12:05:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f66727fc700 1 --2- 172.21.15.44:0/84855525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66740aa400 0x7f66740a9280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f66740a8f70 tx=0x7f6664006df0).ready entity=mon.0 client_cookie=9c7f24f54cacbb89 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f6671ffb700 1 -- 172.21.15.44:0/84855525 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640097b0 con 0x7f66740aa400 2020-11-20T12:05:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f6671ffb700 1 -- 172.21.15.44:0/84855525 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6664009910 con 0x7f66740aa400 2020-11-20T12:05:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.968+0000 7f6671ffb700 1 -- 172.21.15.44:0/84855525 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664009c90 con 0x7f66740aa400 2020-11-20T12:05:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 -- 172.21.15.44:0/84855525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66740aa400 msgr2=0x7f66740a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 --2- 172.21.15.44:0/84855525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66740aa400 0x7f66740a9280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f66740a8f70 tx=0x7f6664006df0).stop 2020-11-20T12:05:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 -- 172.21.15.44:0/84855525 shutdown_connections 2020-11-20T12:05:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 --2- 172.21.15.44:0/84855525 >> v2:172.21.15.79:3300/0 conn(0x7f667413ac60 0x7f66740a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 --2- 172.21.15.44:0/84855525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66740aa400 0x7f66740a9280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 -- 172.21.15.44:0/84855525 >> 172.21.15.44:0/84855525 conn(0x7f6674062690 msgr2=0x7f66740a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 -- 172.21.15.44:0/84855525 shutdown_connections 2020-11-20T12:05:30.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.969+0000 7f66797dc700 1 -- 172.21.15.44:0/84855525 wait complete. 2020-11-20T12:05:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.970+0000 7f66797dc700 1 Processor -- start 2020-11-20T12:05:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.970+0000 7f66797dc700 1 -- start start 2020-11-20T12:05:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66797dc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66740aa020 0x7f66741335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66797dc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f66740aa400 0x7f6674065cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66797dc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667413ac60 0x7f667412ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66797dc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6674067ea0 con 0x7f667413ac60 2020-11-20T12:05:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66797dc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66740ab830 con 0x7f66740aa020 2020-11-20T12:05:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66797dc700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740abad0 con 0x7f66740aa400 2020-11-20T12:05:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f6672ffd700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66740aa020 msgr2=0x7f66741335a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f6672ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66740aa020 0x7f66741335a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667413ac60 0x7f667412ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66737fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667413ac60 0x7f667412ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37744/0 (socket says 172.21.15.44:37744) 2020-11-20T12:05:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66737fe700 1 -- 172.21.15.44:0/1565900385 learned_addr learned my addr 172.21.15.44:0/1565900385 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:30.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66727fc700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f66740aa400 0x7f6674065cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66737fe700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f66740aa400 msgr2=0x7f6674065cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66737fe700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f66740aa400 0x7f6674065cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.971+0000 7f66737fe700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66740aa020 msgr2=0x7f66741335a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:05:30.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f66737fe700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66740aa020 0x7f66741335a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:30.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f66737fe700 1 -- 172.21.15.44:0/1565900385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740661c0 con 0x7f667413ac60 2020-11-20T12:05:30.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f66737fe700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667413ac60 0x7f667412ef80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f666c00bae0 tx=0x7f666c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c003c10 con 0x7f667413ac60 2020-11-20T12:05:30.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c003d70 con 0x7f667413ac60 2020-11-20T12:05:30.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66740ab440 con 0x7f667413ac60 2020-11-20T12:05:30.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c006170 con 0x7f667413ac60 2020-11-20T12:05:30.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.972+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6674066c30 con 0x7f667413ac60 2020-11-20T12:05:30.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.973+0000 7f66797dc700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674149530 0x7f667412f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.973+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740ab970 con 0x7f6674149530 2020-11-20T12:05:30.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.973+0000 7f6672ffd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674149530 0x7f667412f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.973+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f666c006560 con 0x7f667413ac60 2020-11-20T12:05:30.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.973+0000 7f6672ffd700 1 -- 172.21.15.44:0/1565900385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66740a44a0 con 0x7f6674149530 2020-11-20T12:05:30.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6650056d20 0x7f66500591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f666c011020 con 0x7f667413ac60 2020-11-20T12:05:30.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f6672ffd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674149530 0x7f667412f470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6668004530 tx=0x7f6668007a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f66727fc700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6650056d20 0x7f66500591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66680072e0 con 0x7f6674149530 2020-11-20T12:05:30.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f66727fc700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6650056d20 0x7f66500591e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f666400b150 tx=0x7f6664015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f6668007440 con 0x7f6674149530 2020-11-20T12:05:30.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674149530 msgr2=0x7f667412f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674149530 0x7f667412f470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6668004530 tx=0x7f6668007a00).stop 2020-11-20T12:05:30.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674149530 msgr2=0x7f667412f470 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:30.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.974+0000 7f665affd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674149530 0x7f667412f470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.982+0000 7f66797dc700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674067330 0x7f6674131f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:30.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.982+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740abc10 con 0x7f6674067330 2020-11-20T12:05:30.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.982+0000 7f6672ffd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674067330 0x7f6674131f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:30.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f6672ffd700 1 -- 172.21.15.44:0/1565900385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6674061650 con 0x7f6674067330 2020-11-20T12:05:30.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f6672ffd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674067330 0x7f6674131f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6668001230 tx=0x7f6668013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:30.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668017070 con 0x7f6674067330 2020-11-20T12:05:30.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4594 (secure 0 0 0) 0x7f66680070d0 con 0x7f6674067330 2020-11-20T12:05:30.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674067330 msgr2=0x7f6674131f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f665affd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674067330 0x7f6674131f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6668001230 tx=0x7f6668013000).stop 2020-11-20T12:05:30.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f665affd700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674067330 msgr2=0x7f6674131f10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:05:30.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.983+0000 7f665affd700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6674067330 0x7f6674131f10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:30.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6650056d20 msgr2=0x7f66500591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6650056d20 0x7f66500591e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f666400b150 tx=0x7f6664015000).stop 2020-11-20T12:05:30.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667413ac60 msgr2=0x7f667412ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:30.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 --2- 172.21.15.44:0/1565900385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f667413ac60 0x7f667412ef80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f666c00bae0 tx=0x7f666c0090f0).stop 2020-11-20T12:05:30.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66737fe700 1 -- 172.21.15.44:0/1565900385 reap_dead start 2020-11-20T12:05:30.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66737fe700 1 -- 172.21.15.44:0/1565900385 reap_dead start 2020-11-20T12:05:30.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 shutdown_connections 2020-11-20T12:05:30.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 >> 172.21.15.44:0/1565900385 conn(0x7f6674062690 msgr2=0x7f6674060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:30.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 shutdown_connections 2020-11-20T12:05:30.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:05:30.984+0000 7f66797dc700 1 -- 172.21.15.44:0/1565900385 wait complete. 2020-11-20T12:05:30.995 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:05:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:05:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:05:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:05:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 20, 2020-11-20T12:05:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:05:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:05:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:05:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:05:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:05:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:05:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:05:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:05:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:05:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:05:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:05:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:05:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:05:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:05:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:05:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:05:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:05:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:05:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:05:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:05:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:05:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:05:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:05:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:05:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:05:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:05:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:05:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:05:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:05:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:05:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:05:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:05:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:05:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:05:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:05:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:05:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:05:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:05:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:05:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:05:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:05:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:05:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:05:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:05:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:05:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:05:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:05:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:05:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:05:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:05:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:05:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:05:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:05:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:05:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:05:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:05:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:05:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:05:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:05:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:05:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:05:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:05:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:05:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:05:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:05:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:05:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:05:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:05:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:05:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:05:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:05:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:05:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:05:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:05:31.027 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:05:31.028 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:05:35.781 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:05:35.799 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:05:41.881 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir foo 2020-11-20T12:05:41.897 INFO:tasks.workunit.client.0.smithi044.stderr:+ cp linux-2.6.33.tar.bz2 foo 2020-11-20T12:05:42.651 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir foo/.snap/barsnap 2020-11-20T12:05:42.677 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm foo/linux-2.6.33.tar.bz2 2020-11-20T12:05:42.690 INFO:tasks.workunit.client.0.smithi044.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2020-11-20T12:05:43.888 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 'passed: files are identical' 2020-11-20T12:05:43.888 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir foo/.snap/barsnap 2020-11-20T12:05:43.889 INFO:tasks.workunit.client.0.smithi044.stdout:passed: files are identical 2020-11-20T12:05:43.900 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:05:43.900 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:05:43.900 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:05:43.901 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:05:43.952 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-0.sh... 2020-11-20T12:05:43.953 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-0.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-0.sh 2020-11-20T12:05:43.976 INFO:tasks.workunit.client.0.smithi044.stderr:+ set -e 2020-11-20T12:05:43.977 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps false 2020-11-20T12:05:44.078 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.074+0000 7f40786fa700 1 Processor -- start 2020-11-20T12:05:44.078 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.074+0000 7f40786fa700 1 -- start start 2020-11-20T12:05:44.080 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f40786fa700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f40700609c0 0x7f40700ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:44.080 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f40786fa700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40700663b0 con 0x7f40700acd50 2020-11-20T12:05:44.080 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f40786fa700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4070066960 con 0x7f4070067a30 2020-11-20T12:05:44.080 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f40786fa700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4070066770 con 0x7f40700609c0 2020-11-20T12:05:44.081 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4076496700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f40700609c0 0x7f40700ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:44.081 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4076c97700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f4070067a30 legacy=0x7f407013b120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:44.081 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4075c95700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f40700acd50 0x7f4070067320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41508/0 (socket says 172.21.15.44:41508) 2020-11-20T12:05:44.081 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4075c95700 1 -- 172.21.15.44:0/4207989461 learned_addr learned my addr 172.21.15.44:0/4207989461 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:44.082 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4076496700 1 -- 172.21.15.44:0/4207989461 >> v1:172.21.15.79:6789/0 conn(0x7f4070067a30 legacy=0x7f407013b120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:44.082 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4076496700 1 -- 172.21.15.44:0/4207989461 >> v1:172.21.15.44:6789/0 conn(0x7f40700acd50 legacy=0x7f4070067320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:44.082 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4075494700 1 -- 172.21.15.44:0/4207989461 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693140804 0 0) 0x7f40700663b0 con 0x7f40700acd50 2020-11-20T12:05:44.082 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4076496700 1 -- 172.21.15.44:0/4207989461 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4070066bc0 con 0x7f40700609c0 2020-11-20T12:05:44.082 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4076496700 1 --2- 172.21.15.44:0/4207989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700609c0 0x7f40700ac860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f406000b9a0 tx=0x7f4060006df0).ready entity=mon.0 client_cookie=deb88faed2e38a88 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:44.083 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.075+0000 7f4075494700 1 -- 172.21.15.44:0/4207989461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40600097b0 con 0x7f40700609c0 2020-11-20T12:05:44.083 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f4075494700 1 -- 172.21.15.44:0/4207989461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4060009910 con 0x7f40700609c0 2020-11-20T12:05:44.083 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f4075494700 1 -- 172.21.15.44:0/4207989461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060009c90 con 0x7f40700609c0 2020-11-20T12:05:44.083 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f40786fa700 1 -- 172.21.15.44:0/4207989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700609c0 msgr2=0x7f40700ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:44.084 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f40786fa700 1 --2- 172.21.15.44:0/4207989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700609c0 0x7f40700ac860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f406000b9a0 tx=0x7f4060006df0).stop 2020-11-20T12:05:44.084 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f40786fa700 1 -- 172.21.15.44:0/4207989461 shutdown_connections 2020-11-20T12:05:44.084 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f40786fa700 1 --2- 172.21.15.44:0/4207989461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700609c0 0x7f40700ac860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:44.084 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f40786fa700 1 -- 172.21.15.44:0/4207989461 >> 172.21.15.44:0/4207989461 conn(0x7f4070064300 msgr2=0x7f40700600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:44.085 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f40786fa700 1 -- 172.21.15.44:0/4207989461 shutdown_connections 2020-11-20T12:05:44.085 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.076+0000 7f40786fa700 1 -- 172.21.15.44:0/4207989461 wait complete. 2020-11-20T12:05:44.088 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.077+0000 7f40786fa700 1 Processor -- start 2020-11-20T12:05:44.088 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.077+0000 7f40786fa700 1 -- start start 2020-11-20T12:05:44.088 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40700609c0 0x7f407013ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:44.089 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4070067a30 0x7f40701401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:44.089 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700acd50 0x7f4070142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:44.089 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40700ad1f0 con 0x7f40700acd50 2020-11-20T12:05:44.090 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4070140a50 con 0x7f40700609c0 2020-11-20T12:05:44.090 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40701407b0 con 0x7f4070067a30 2020-11-20T12:05:44.090 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076496700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40700609c0 msgr2=0x7f407013ef40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:44.090 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076496700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40700609c0 0x7f407013ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:44.090 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700acd50 0x7f4070142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:44.091 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700acd50 0x7f4070142080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37770/0 (socket says 172.21.15.44:37770) 2020-11-20T12:05:44.091 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 -- 172.21.15.44:0/2540412930 learned_addr learned my addr 172.21.15.44:0/2540412930 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:44.091 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4075c95700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4070067a30 0x7f40701401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:44.091 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 -- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4070067a30 msgr2=0x7f40701401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:44.091 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4070067a30 0x7f40701401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:44.092 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 -- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40700609c0 msgr2=0x7f407013ef40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:05:44.092 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40700609c0 0x7f407013ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:44.092 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 -- 172.21.15.44:0/2540412930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407013eaa0 con 0x7f40700acd50 2020-11-20T12:05:44.092 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f4076c97700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700acd50 0x7f4070142080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f406800bae0 tx=0x7f4068006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:44.093 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068003c10 con 0x7f40700acd50 2020-11-20T12:05:44.093 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068003d70 con 0x7f40700acd50 2020-11-20T12:05:44.093 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4070142570 con 0x7f40700acd50 2020-11-20T12:05:44.093 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068006170 con 0x7f40700acd50 2020-11-20T12:05:44.093 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.078+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407014d840 con 0x7f40700acd50 2020-11-20T12:05:44.094 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.079+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40700626f0 con 0x7f40700acd50 2020-11-20T12:05:44.094 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.080+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4068006560 con 0x7f40700acd50 2020-11-20T12:05:44.094 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.080+0000 7f405e7fc700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4048056cd0 0x7f4048059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:44.094 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.080+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4068011020 con 0x7f40700acd50 2020-11-20T12:05:44.095 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.080+0000 7f4076496700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4048056cd0 0x7f4048059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:44.095 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.080+0000 7f4076496700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4048056cd0 0x7f4048059190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f406000b150 tx=0x7f4060015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:44.095 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.084+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4068044b50 con 0x7f40700acd50 2020-11-20T12:05:44.321 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.317+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"} v 0) v1 -- 0x7f40700446d0 con 0x7f40700acd50 2020-11-20T12:05:44.693 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.689+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"}]=0 disabled new snapshots v10) v1 ==== 139+0+0 (secure 0 0 0) 0x7f4068048170 con 0x7f40700acd50 2020-11-20T12:05:44.869 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:44.865+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"} v 0) v1 -- 0x7f40700622b0 con 0x7f40700acd50 2020-11-20T12:05:45.695 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.691+0000 7f405e7fc700 1 -- 172.21.15.44:0/2540412930 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"}]=0 disabled new snapshots v11) v1 ==== 139+0+0 (secure 0 0 0) 0x7f4068048350 con 0x7f40700acd50 2020-11-20T12:05:45.698 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4048056cd0 msgr2=0x7f4048059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:45.698 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4048056cd0 0x7f4048059190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f406000b150 tx=0x7f4060015000).stop 2020-11-20T12:05:45.699 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700acd50 msgr2=0x7f4070142080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:45.699 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700acd50 0x7f4070142080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f406800bae0 tx=0x7f4068006f90).stop 2020-11-20T12:05:45.699 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 shutdown_connections 2020-11-20T12:05:45.699 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4048056cd0 0x7f4048059190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:45.702 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40700acd50 0x7f4070142080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:45.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f40700609c0 0x7f407013ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:45.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 --2- 172.21.15.44:0/2540412930 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4070067a30 0x7f40701401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:45.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 >> 172.21.15.44:0/2540412930 conn(0x7f4070064300 msgr2=0x7f40700600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:45.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 shutdown_connections 2020-11-20T12:05:45.704 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.694+0000 7f40786fa700 1 -- 172.21.15.44:0/2540412930 wait complete. 2020-11-20T12:05:45.704 INFO:tasks.workunit.client.0.smithi044.stderr:disabled new snapshots 2020-11-20T12:05:45.711 INFO:tasks.workunit.client.0.smithi044.stderr:+ expect_failure mkdir .snap/foo 2020-11-20T12:05:45.711 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/foo 2020-11-20T12:05:45.714 INFO:tasks.workunit.client.0.smithi044.stderr:mkdir: cannot create directory ‘.snap/foo’: Operation not permitted 2020-11-20T12:05:45.715 INFO:tasks.workunit.client.0.smithi044.stderr:+ return 0 2020-11-20T12:05:45.715 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:05:45.823 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.819+0000 7f810fd7f700 1 Processor -- start 2020-11-20T12:05:45.824 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.819+0000 7f810fd7f700 1 -- start start 2020-11-20T12:05:45.830 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810fd7f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81080ada20 0x7f81080ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:45.830 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810fd7f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f81081385c0 0x7f8108138a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:45.831 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810fd7f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81080a8af0 con 0x7f81080ab220 2020-11-20T12:05:45.831 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810fd7f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81080a8d50 con 0x7f81081385c0 2020-11-20T12:05:45.832 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810fd7f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81080a8ff0 con 0x7f81080ada20 2020-11-20T12:05:45.832 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810db1b700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f81080ab220 legacy=0x7f81080aa600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:45.832 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810e31c700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f81081385c0 msgr2=0x7f8108138a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:45.833 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810e31c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f81081385c0 0x7f8108138a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:45.833 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810d31a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81080ada20 0x7f81080ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:45.833 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810d31a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81080ada20 0x7f81080ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49212/0 (socket says 172.21.15.44:49212) 2020-11-20T12:05:45.834 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.820+0000 7f810d31a700 1 -- 172.21.15.44:0/948494687 learned_addr learned my addr 172.21.15.44:0/948494687 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:45.834 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810d31a700 1 -- 172.21.15.44:0/948494687 >> v2:172.21.15.79:3300/0 conn(0x7f81081385c0 msgr2=0x7f8108138a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:45.834 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810d31a700 1 --2- 172.21.15.44:0/948494687 >> v2:172.21.15.79:3300/0 conn(0x7f81081385c0 0x7f8108138a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:45.835 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810d31a700 1 -- 172.21.15.44:0/948494687 >> v1:172.21.15.79:6789/0 conn(0x7f81080ab220 legacy=0x7f81080aa600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:45.835 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810d31a700 1 -- 172.21.15.44:0/948494687 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81080ae370 con 0x7f81080ada20 2020-11-20T12:05:45.835 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810d31a700 1 --2- 172.21.15.44:0/948494687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81080ada20 0x7f81080ade80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8104006fd0 tx=0x7f8104006eb0).ready entity=mon.2 client_cookie=b76a38fcdfe0b976 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:45.835 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810cb19700 1 -- 172.21.15.44:0/948494687 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104009920 con 0x7f81080ada20 2020-11-20T12:05:45.836 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810cb19700 1 -- 172.21.15.44:0/948494687 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8104009a80 con 0x7f81080ada20 2020-11-20T12:05:45.836 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.821+0000 7f810cb19700 1 -- 172.21.15.44:0/948494687 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104009e00 con 0x7f81080ada20 2020-11-20T12:05:45.836 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 -- 172.21.15.44:0/948494687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81080ada20 msgr2=0x7f81080ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:45.836 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 --2- 172.21.15.44:0/948494687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81080ada20 0x7f81080ade80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8104006fd0 tx=0x7f8104006eb0).stop 2020-11-20T12:05:45.836 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 -- 172.21.15.44:0/948494687 shutdown_connections 2020-11-20T12:05:45.837 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 --2- 172.21.15.44:0/948494687 >> v2:172.21.15.79:3300/0 conn(0x7f81081385c0 0x7f8108138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:45.837 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 --2- 172.21.15.44:0/948494687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81080ada20 0x7f81080ade80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:45.837 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 -- 172.21.15.44:0/948494687 >> 172.21.15.44:0/948494687 conn(0x7f8108062060 msgr2=0x7f81080600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:45.838 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 -- 172.21.15.44:0/948494687 shutdown_connections 2020-11-20T12:05:45.838 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.822+0000 7f810fd7f700 1 -- 172.21.15.44:0/948494687 wait complete. 2020-11-20T12:05:45.839 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 Processor -- start 2020-11-20T12:05:45.839 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 -- start start 2020-11-20T12:05:45.839 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81080ab220 0x7f81080a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:45.840 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81080ada20 0x7f81080a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:45.840 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81081385c0 0x7f810813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:45.840 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81080a18a0 con 0x7f81080ada20 2020-11-20T12:05:45.841 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81080a1b40 con 0x7f81080ab220 2020-11-20T12:05:45.841 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810fd7f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81080a1de0 con 0x7f81081385c0 2020-11-20T12:05:45.841 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810db1b700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81080ab220 msgr2=0x7f81080a2280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:45.841 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810db1b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81080ab220 0x7f81080a2280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:45.842 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810d31a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81080ada20 0x7f81080a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:45.843 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810d31a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81080ada20 0x7f81080a4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37782/0 (socket says 172.21.15.44:37782) 2020-11-20T12:05:45.843 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810d31a700 1 -- 172.21.15.44:0/2131243710 learned_addr learned my addr 172.21.15.44:0/2131243710 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:45.843 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.823+0000 7f810e31c700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81081385c0 0x7f810813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:45.844 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810d31a700 1 -- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81081385c0 msgr2=0x7f810813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:45.844 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810d31a700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81081385c0 0x7f810813e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:45.845 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810d31a700 1 -- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81080ab220 msgr2=0x7f81080a2280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:05:45.846 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810d31a700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81080ab220 0x7f81080a2280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:45.846 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810d31a700 1 -- 172.21.15.44:0/2131243710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81081405f0 con 0x7f81080ada20 2020-11-20T12:05:45.847 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810d31a700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81080ada20 0x7f81080a4340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8108064080 tx=0x7f8104015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:45.847 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810401e460 con 0x7f81080ada20 2020-11-20T12:05:45.847 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810401e5e0 con 0x7f81080ada20 2020-11-20T12:05:45.848 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81081411b0 con 0x7f81080ada20 2020-11-20T12:05:45.848 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810401e9a0 con 0x7f81080ada20 2020-11-20T12:05:45.848 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.824+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81081416c0 con 0x7f81080ada20 2020-11-20T12:05:45.848 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.825+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81080aeac0 con 0x7f81080ada20 2020-11-20T12:05:45.849 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.825+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f810401a040 con 0x7f81080ada20 2020-11-20T12:05:45.849 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.826+0000 7f80f5ffb700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80e0056cd0 0x7f80e0059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:45.849 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.826+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8104071740 con 0x7f81080ada20 2020-11-20T12:05:45.849 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.826+0000 7f810db1b700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80e0056cd0 0x7f80e0059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:45.850 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.826+0000 7f810db1b700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80e0056cd0 0x7f80e0059190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f80f8003650 tx=0x7f80f8006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:45.851 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:45.829+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8104046ae0 con 0x7f81080ada20 2020-11-20T12:05:46.065 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:46.062+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f810805c300 con 0x7f81080ada20 2020-11-20T12:05:46.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:46.697+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v12) v1 ==== 167+0+0 (secure 0 0 0) 0x7f8104023020 con 0x7f81080ada20 2020-11-20T12:05:46.880 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:46.876+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f810805c590 con 0x7f81080ada20 2020-11-20T12:05:47.708 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.704+0000 7f80f5ffb700 1 -- 172.21.15.44:0/2131243710 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v13) v1 ==== 167+0+0 (secure 0 0 0) 0x7f810402c0d0 con 0x7f81080ada20 2020-11-20T12:05:47.711 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80e0056cd0 msgr2=0x7f80e0059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:47.711 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80e0056cd0 0x7f80e0059190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f80f8003650 tx=0x7f80f8006ae0).stop 2020-11-20T12:05:47.712 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81080ada20 msgr2=0x7f81080a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:47.712 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81080ada20 0x7f81080a4340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8108064080 tx=0x7f8104015000).stop 2020-11-20T12:05:47.712 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 shutdown_connections 2020-11-20T12:05:47.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f80e0056cd0 0x7f80e0059190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:47.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81081385c0 0x7f810813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:47.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81080ab220 0x7f81080a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:47.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 --2- 172.21.15.44:0/2131243710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81080ada20 0x7f81080a4340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:47.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 >> 172.21.15.44:0/2131243710 conn(0x7f8108062060 msgr2=0x7f8108137930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:47.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 shutdown_connections 2020-11-20T12:05:47.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.706+0000 7f810fd7f700 1 -- 172.21.15.44:0/2131243710 wait complete. 2020-11-20T12:05:47.716 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:05:47.723 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo asdf 2020-11-20T12:05:47.727 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/foo 2020-11-20T12:05:47.759 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep asdf .snap/foo/foo 2020-11-20T12:05:47.761 INFO:tasks.workunit.client.0.smithi044.stdout:asdf 2020-11-20T12:05:47.761 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir .snap/foo 2020-11-20T12:05:47.776 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo asdf 2020-11-20T12:05:47.777 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/bar 2020-11-20T12:05:47.788 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm bar 2020-11-20T12:05:47.792 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep asdf .snap/bar/bar 2020-11-20T12:05:47.801 INFO:tasks.workunit.client.0.smithi044.stdout:asdf 2020-11-20T12:05:47.801 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir .snap/bar 2020-11-20T12:05:47.810 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm foo 2020-11-20T12:05:47.812 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps false 2020-11-20T12:05:47.930 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.912+0000 7f1dbb643700 1 Processor -- start 2020-11-20T12:05:47.931 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.912+0000 7f1dbb643700 1 -- start start 2020-11-20T12:05:47.931 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1dbb643700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1db40a95e0 0x7f1db40a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.932 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1dbb643700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db40a59a0 con 0x7f1db40a8b40 2020-11-20T12:05:47.932 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1dbb643700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db40a51c0 con 0x7f1db40a34f0 2020-11-20T12:05:47.933 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1dbb643700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db40a5460 con 0x7f1db40a95e0 2020-11-20T12:05:47.933 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1db93df700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1db40a8b40 0x7f1db40a6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41538/0 (socket says 172.21.15.44:41538) 2020-11-20T12:05:47.934 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1db93df700 1 -- 172.21.15.44:0/1795656334 learned_addr learned my addr 172.21.15.44:0/1795656334 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:47.934 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1db9be0700 1 -- 172.21.15.44:0/1795656334 >> v1:172.21.15.79:6789/0 conn(0x7f1db40a34f0 legacy=0x7f1db40a3940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:05:47.935 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1db8bde700 1 -- 172.21.15.44:0/1795656334 >> v2:172.21.15.79:3300/0 conn(0x7f1db40a95e0 msgr2=0x7f1db40a9a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:47.935 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1db8bde700 1 --2- 172.21.15.44:0/1795656334 >> v2:172.21.15.79:3300/0 conn(0x7f1db40a95e0 0x7f1db40a9a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:47.935 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341857961 0 0) 0x7f1db40a59a0 con 0x7f1db40a8b40 2020-11-20T12:05:47.936 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d9c003240 con 0x7f1db40a8b40 2020-11-20T12:05:47.936 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2661280809 0 0) 0x7f1d9c003240 con 0x7f1db40a8b40 2020-11-20T12:05:47.937 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 >> v2:172.21.15.79:3300/0 conn(0x7f1db40a95e0 msgr2=0x7f1db40a9a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:47.937 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 --2- 172.21.15.44:0/1795656334 >> v2:172.21.15.79:3300/0 conn(0x7f1db40a95e0 0x7f1db40a9a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:47.937 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 >> v1:172.21.15.79:6789/0 conn(0x7f1db40a34f0 legacy=0x7f1db40a3940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:47.938 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40a5700 con 0x7f1db40a8b40 2020-11-20T12:05:47.938 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.913+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f1da4001660 con 0x7f1db40a8b40 2020-11-20T12:05:47.939 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 >> v1:172.21.15.44:6789/0 conn(0x7f1db40a8b40 legacy=0x7f1db40a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:47.939 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d9c003730 0x7f1d9c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.939 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1d9c004560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.940 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9c004b60 0x7f1d9c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.940 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40a5460 con 0x7f1d9c004090 2020-11-20T12:05:47.940 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c005160 con 0x7f1d9c004b60 2020-11-20T12:05:47.941 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c005320 con 0x7f1d9c003730 2020-11-20T12:05:47.941 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db8bde700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1d9c004560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:47.942 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db9be0700 1 -- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9c004b60 msgr2=0x7f1d9c00b730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:47.942 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db9be0700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9c004b60 0x7f1d9c00b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:47.942 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db93df700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d9c003730 0x7f1d9c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:47.943 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db8bde700 1 -- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d9c003730 msgr2=0x7f1d9c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:05:47.944 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db8bde700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1d9c003730 0x7f1d9c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:47.944 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db8bde700 1 -- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9c004b60 msgr2=0x7f1d9c00b730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:47.944 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db8bde700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9c004b60 0x7f1d9c00b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:47.944 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db9be0700 1 -- 172.21.15.44:0/1795656334 reap_dead start 2020-11-20T12:05:47.945 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db8bde700 1 -- 172.21.15.44:0/1795656334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c003240 con 0x7f1d9c004090 2020-11-20T12:05:47.945 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1db8bde700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1d9c004560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1db0006fd0 tx=0x7f1db0006e90).ready entity=mon.0 client_cookie=fd6079f31c4eb0ed server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:47.945 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00050d0 con 0x7f1d9c004090 2020-11-20T12:05:47.945 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db0005250 con 0x7f1d9c004090 2020-11-20T12:05:47.946 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.914+0000 7f1da3fff700 1 -- 172.21.15.44:0/1795656334 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00038e0 con 0x7f1d9c004090 2020-11-20T12:05:47.946 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.915+0000 7f1dbb643700 1 -- 172.21.15.44:0/1795656334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 msgr2=0x7f1d9c004560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:47.946 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.915+0000 7f1dbb643700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1d9c004560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1db0006fd0 tx=0x7f1db0006e90).stop 2020-11-20T12:05:47.946 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.915+0000 7f1dbb643700 1 -- 172.21.15.44:0/1795656334 shutdown_connections 2020-11-20T12:05:47.946 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.915+0000 7f1dbb643700 1 --2- 172.21.15.44:0/1795656334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1d9c004560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:47.947 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.915+0000 7f1dbb643700 1 -- 172.21.15.44:0/1795656334 >> 172.21.15.44:0/1795656334 conn(0x7f1db40600f0 msgr2=0x7f1db4060500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:47.947 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.915+0000 7f1dbb643700 1 -- 172.21.15.44:0/1795656334 shutdown_connections 2020-11-20T12:05:47.947 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.915+0000 7f1dbb643700 1 -- 172.21.15.44:0/1795656334 wait complete. 2020-11-20T12:05:47.947 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 Processor -- start 2020-11-20T12:05:47.948 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 -- start start 2020-11-20T12:05:47.948 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1db4138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.948 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db409e2b0 0x7f1db409fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.948 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db41359d0 0x7f1db4135e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.948 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db413b8f0 con 0x7f1d9c004090 2020-11-20T12:05:47.949 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40a7600 con 0x7f1db41359d0 2020-11-20T12:05:47.951 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.916+0000 7f1dbb643700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4067f70 con 0x7f1db409e2b0 2020-11-20T12:05:47.951 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1db4138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:47.951 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db9be0700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db41359d0 msgr2=0x7f1db4135e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:47.952 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1db4138d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37802/0 (socket says 172.21.15.44:37802) 2020-11-20T12:05:47.952 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 -- 172.21.15.44:0/2250599109 learned_addr learned my addr 172.21.15.44:0/2250599109 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:47.953 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db9be0700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db41359d0 0x7f1db4135e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:47.953 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db8bde700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db409e2b0 0x7f1db409fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:47.953 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 -- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db409e2b0 msgr2=0x7f1db409fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:47.954 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db409e2b0 0x7f1db409fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:47.954 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 -- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db41359d0 msgr2=0x7f1db4135e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:05:47.955 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db41359d0 0x7f1db4135e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:47.955 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 -- 172.21.15.44:0/2250599109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db409d490 con 0x7f1d9c004090 2020-11-20T12:05:47.961 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1db93df700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1db4138d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1da400ec30 tx=0x7f1da4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:47.962 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4007320 con 0x7f1d9c004090 2020-11-20T12:05:47.962 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da400c6d0 con 0x7f1d9c004090 2020-11-20T12:05:47.962 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db409e730 con 0x7f1d9c004090 2020-11-20T12:05:47.963 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da400ca70 con 0x7f1d9c004090 2020-11-20T12:05:47.963 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.917+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db409de10 con 0x7f1d9c004090 2020-11-20T12:05:47.963 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.918+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db413bc90 con 0x7f1d9c004090 2020-11-20T12:05:47.964 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.918+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1da40054c0 con 0x7f1d9c004090 2020-11-20T12:05:47.964 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.919+0000 7f1da17fa700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d8c056cd0 0x7f1d8c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:47.965 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.919+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1da406fc70 con 0x7f1d9c004090 2020-11-20T12:05:47.966 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.919+0000 7f1db8bde700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d8c056cd0 0x7f1d8c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:47.966 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.919+0000 7f1db8bde700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d8c056cd0 0x7f1d8c059190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f1db40a2b80 tx=0x7f1db0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:47.967 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:47.923+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1da4044f00 con 0x7f1d9c004090 2020-11-20T12:05:48.158 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:48.154+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"} v 0) v1 -- 0x7f1db405c590 con 0x7f1d9c004090 2020-11-20T12:05:48.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:48.709+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"}]=0 disabled new snapshots v14) v1 ==== 139+0+0 (secure 0 0 0) 0x7f1da4048520 con 0x7f1d9c004090 2020-11-20T12:05:48.892 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:48.888+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"} v 0) v1 -- 0x7f1db40a8f20 con 0x7f1d9c004090 2020-11-20T12:05:49.720 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.716+0000 7f1da17fa700 1 -- 172.21.15.44:0/2250599109 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false"}]=0 disabled new snapshots v15) v1 ==== 139+0+0 (secure 0 0 0) 0x7f1da4022020 con 0x7f1d9c004090 2020-11-20T12:05:49.724 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.720+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d8c056cd0 msgr2=0x7f1d8c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:49.724 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.720+0000 7f1dbb643700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d8c056cd0 0x7f1d8c059190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f1db40a2b80 tx=0x7f1db0015000).stop 2020-11-20T12:05:49.725 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.720+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 msgr2=0x7f1db4138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:49.725 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.720+0000 7f1dbb643700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1db4138d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1da400ec30 tx=0x7f1da4009040).stop 2020-11-20T12:05:49.726 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 shutdown_connections 2020-11-20T12:05:49.726 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d8c056cd0 0x7f1d8c059190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:49.726 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db41359d0 0x7f1db4135e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:49.727 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d9c004090 0x7f1db4138d60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:49.727 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 --2- 172.21.15.44:0/2250599109 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db409e2b0 0x7f1db409fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:49.727 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 >> 172.21.15.44:0/2250599109 conn(0x7f1db40600f0 msgr2=0x7f1db40ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:49.728 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 shutdown_connections 2020-11-20T12:05:49.728 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.721+0000 7f1dbb643700 1 -- 172.21.15.44:0/2250599109 wait complete. 2020-11-20T12:05:49.729 INFO:tasks.workunit.client.0.smithi044.stderr:disabled new snapshots 2020-11-20T12:05:49.738 INFO:tasks.workunit.client.0.smithi044.stderr:+ expect_failure mkdir .snap/baz 2020-11-20T12:05:49.738 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/baz 2020-11-20T12:05:49.743 INFO:tasks.workunit.client.0.smithi044.stderr:mkdir: cannot create directory ‘.snap/baz’: Operation not permitted 2020-11-20T12:05:49.743 INFO:tasks.workunit.client.0.smithi044.stderr:+ return 0 2020-11-20T12:05:49.743 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:05:49.744 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:05:49.744 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:05:49.744 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:05:49.823 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2020-11-20T12:05:49.823 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2020-11-20T12:05:49.891 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:05:49.995 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.991+0000 7ff6d6571700 1 Processor -- start 2020-11-20T12:05:49.995 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.991+0000 7ff6d6571700 1 -- start start 2020-11-20T12:05:49.999 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d6571700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff6d00a8b40 0x7ff6d013aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:49.999 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d6571700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff6d00a3930 0x7ff6d01387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:49.999 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d6571700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d0064ed0 con 0x7ff6d013b3b0 2020-11-20T12:05:49.999 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d6571700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d00a6320 con 0x7ff6d00a3930 2020-11-20T12:05:50.000 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d6571700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d00a5be0 con 0x7ff6d00a8b40 2020-11-20T12:05:50.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d4b0e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff6d00a3930 0x7ff6d01387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:50.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6cffff700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7ff6d00a8b40 msgr2=0x7ff6d013aec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:50.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6cf7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff6d013b3b0 0x7ff6d01380b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41564/0 (socket says 172.21.15.44:41564) 2020-11-20T12:05:50.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6cf7fe700 1 -- 172.21.15.44:0/1237297600 learned_addr learned my addr 172.21.15.44:0/1237297600 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:05:50.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6cffff700 1 --2- 172.21.15.44:0/1237297600 >> v2:172.21.15.79:3300/0 conn(0x7ff6d00a8b40 0x7ff6d013aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:50.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d4b0e700 1 -- 172.21.15.44:0/1237297600 >> v2:172.21.15.79:3300/0 conn(0x7ff6d00a8b40 msgr2=0x7ff6d013aec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:05:50.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d4b0e700 1 --2- 172.21.15.44:0/1237297600 >> v2:172.21.15.79:3300/0 conn(0x7ff6d00a8b40 0x7ff6d013aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:50.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d4b0e700 1 -- 172.21.15.44:0/1237297600 >> v1:172.21.15.44:6789/0 conn(0x7ff6d013b3b0 legacy=0x7ff6d01380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:50.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d4b0e700 1 -- 172.21.15.44:0/1237297600 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d00a5e80 con 0x7ff6d00a3930 2020-11-20T12:05:50.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6ceffd700 1 -- 172.21.15.44:0/1237297600 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209170118 0 0) 0x7ff6d0064ed0 con 0x7ff6d013b3b0 2020-11-20T12:05:50.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6d4b0e700 1 --2- 172.21.15.44:0/1237297600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a3930 0x7ff6d01387c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7ff6d00a74a0 tx=0x7ff6c80091d0).ready entity=mon.0 client_cookie=2aea794a63d8de5b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:50.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6ceffd700 1 -- 172.21.15.44:0/1237297600 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800bbc0 con 0x7ff6d00a3930 2020-11-20T12:05:50.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6ceffd700 1 -- 172.21.15.44:0/1237297600 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c800bd20 con 0x7ff6d00a3930 2020-11-20T12:05:50.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.992+0000 7ff6ceffd700 1 -- 172.21.15.44:0/1237297600 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8011610 con 0x7ff6d00a3930 2020-11-20T12:05:50.008 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 -- 172.21.15.44:0/1237297600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a3930 msgr2=0x7ff6d01387c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:50.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 --2- 172.21.15.44:0/1237297600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a3930 0x7ff6d01387c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7ff6d00a74a0 tx=0x7ff6c80091d0).stop 2020-11-20T12:05:50.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 -- 172.21.15.44:0/1237297600 shutdown_connections 2020-11-20T12:05:50.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 --2- 172.21.15.44:0/1237297600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a3930 0x7ff6d01387c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:50.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 --2- 172.21.15.44:0/1237297600 >> v2:172.21.15.79:3300/0 conn(0x7ff6d00a8b40 0x7ff6d013aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:50.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 -- 172.21.15.44:0/1237297600 >> 172.21.15.44:0/1237297600 conn(0x7ff6d00aced0 msgr2=0x7ff6d00ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:05:50.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 -- 172.21.15.44:0/1237297600 shutdown_connections 2020-11-20T12:05:50.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.993+0000 7ff6d6571700 1 -- 172.21.15.44:0/1237297600 wait complete. 2020-11-20T12:05:50.012 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.994+0000 7ff6d6571700 1 Processor -- start 2020-11-20T12:05:50.012 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.994+0000 7ff6d6571700 1 -- start start 2020-11-20T12:05:50.017 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d6571700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d00a3930 0x7ff6d00a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:50.018 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d6571700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a8b40 0x7ff6d00a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:50.018 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d6571700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6d013b3b0 0x7ff6d00a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:50.019 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d6571700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00a1d00 con 0x7ff6d00a8b40 2020-11-20T12:05:50.019 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d6571700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00a0160 con 0x7ff6d00a3930 2020-11-20T12:05:50.019 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d6571700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d00a0400 con 0x7ff6d013b3b0 2020-11-20T12:05:50.020 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a8b40 0x7ff6d00a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:50.020 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a8b40 0x7ff6d00a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37820/0 (socket says 172.21.15.44:37820) 2020-11-20T12:05:50.021 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cffff700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d00a3930 msgr2=0x7ff6d00a16d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:05:50.021 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 -- 172.21.15.44:0/3075895340 learned_addr learned my addr 172.21.15.44:0/3075895340 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:05:50.021 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cffff700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d00a3930 0x7ff6d00a16d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:05:50.022 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d4b0e700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6d013b3b0 0x7ff6d00a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:50.022 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 -- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6d013b3b0 msgr2=0x7ff6d00a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:05:50.023 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6d013b3b0 0x7ff6d00a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:05:50.023 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 -- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d00a3930 msgr2=0x7ff6d00a16d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:05:50.023 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d00a3930 0x7ff6d00a16d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:05:50.024 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 -- 172.21.15.44:0/3075895340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d01190a0 con 0x7ff6d00a8b40 2020-11-20T12:05:50.024 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6cf7fe700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a8b40 0x7ff6d00a7910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff6c400c760 tx=0x7ff6c4007cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:50.024 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4007e70 con 0x7ff6d00a8b40 2020-11-20T12:05:50.025 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c4007320 con 0x7ff6d00a8b40 2020-11-20T12:05:50.025 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4005800 con 0x7ff6d00a8b40 2020-11-20T12:05:50.025 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.995+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d00a7e00 con 0x7ff6d00a8b40 2020-11-20T12:05:50.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.996+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d0149b30 con 0x7ff6d00a8b40 2020-11-20T12:05:50.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.996+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d00a05c0 con 0x7ff6d00a8b40 2020-11-20T12:05:50.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.997+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff6c4010030 con 0x7ff6d00a8b40 2020-11-20T12:05:50.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.997+0000 7ff6affff700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff6a8056cd0 0x7ff6a8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:05:50.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.997+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6487+0+0 (secure 0 0 0) 0x7ff6c406fdd0 con 0x7ff6d00a8b40 2020-11-20T12:05:50.027 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.997+0000 7ff6cffff700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff6a8056cd0 0x7ff6a8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:05:50.027 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:49.998+0000 7ff6cffff700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff6a8056cd0 0x7ff6a8059190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff6c0003650 tx=0x7ff6c0006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:05:50.027 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:50.001+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff6c4045080 con 0x7ff6d00a8b40 2020-11-20T12:05:50.243 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:50.240+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6d005c590 con 0x7ff6d00a8b40 2020-11-20T12:05:50.725 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:50.721+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v16) v1 ==== 167+0+0 (secure 0 0 0) 0x7ff6c40486a0 con 0x7ff6d00a8b40 2020-11-20T12:05:50.905 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:05:50.902+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6d0149d80 con 0x7ff6d00a8b40 2020-11-20T12:05:51.029 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:05:51.030 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:05:51.030 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:05:51.031 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:05:51.034 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:05:51.035 INFO:tasks.mon_thrash:Sending STOP to mon c 2020-11-20T12:05:51.035 INFO:tasks.ceph.mon.c:Sent signal 19 2020-11-20T12:05:51.036 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2020-11-20T12:06:05.876 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:06:05.880 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:06:06.037 INFO:tasks.mon_thrash:Sending CONT to mon c 2020-11-20T12:06:06.038 INFO:tasks.ceph.mon.c:Sent signal 18 2020-11-20T12:06:06.039 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:06:06.039 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:06:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.175+0000 7f3a96310700 1 Processor -- start 2020-11-20T12:06:06.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.175+0000 7f3a96310700 1 -- start start 2020-11-20T12:06:06.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a96310700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a96310700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a96310700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a900a9770 con 0x7f3a900aa400 2020-11-20T12:06:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a96310700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a900a56c0 con 0x7f3a900aa020 2020-11-20T12:06:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a96310700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900a6da0 con 0x7f3a9013ac60 2020-11-20T12:06:06.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a948ad700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a948ad700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49268/0 (socket says 172.21.15.44:49268) 2020-11-20T12:06:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a8ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 learned_addr learned my addr 172.21.15.44:0/2476299089 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:06.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:06:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:06.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:06:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:06.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.176+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:06.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:06.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:06.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:06:06.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:06:06.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:06.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:06:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.377+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:06:06.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:06:06.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:06:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:06:06.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.778+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:06.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:06.779+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:06:07.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:07.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:07.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:06:07.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a8ffff700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:07.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:07.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:06:07.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:07.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:07.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:06:07.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:07.580+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:06:09.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 msgr2=0x7f3a900a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a9013ac60 0x7f3a900a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 msgr2=0x7f3a900a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.79:3300/0 conn(0x7f3a900aa020 0x7f3a900a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:09.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 -- 172.21.15.44:0/2476299089 >> v1:172.21.15.79:6789/0 conn(0x7f3a900aa400 legacy=0x7f3a900a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 -- 172.21.15.44:0/2476299089 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a900a7040 con 0x7f3a700055c0 2020-11-20T12:06:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8dffb700 1 -- 172.21.15.44:0/2476299089 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a900a56c0 con 0x7f3a70009270 2020-11-20T12:06:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:09.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:09.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.176+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:09.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.377+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:09.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.377+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:09.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.377+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:09.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.377+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:09.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.377+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:06:09.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.778+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:09.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.778+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.778+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.778+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:09.778+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:06:10.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.580+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.580+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:10.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.580+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:10.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.580+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:10.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.580+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:06:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.690+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369771560 0 0) 0x7f3a900a7040 con 0x7f3a700055c0 2020-11-20T12:06:10.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.690+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7c003240 con 0x7f3a700055c0 2020-11-20T12:06:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2966272118 0 0) 0x7f3a7c003240 con 0x7f3a700055c0 2020-11-20T12:06:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 msgr2=0x7f3a7000b6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 --2- 172.21.15.44:0/2476299089 >> v2:172.21.15.44:3300/0 conn(0x7f3a70009270 0x7f3a7000b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7000ce10 con 0x7f3a700055c0 2020-11-20T12:06:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f3a8000d1c0 con 0x7f3a700055c0 2020-11-20T12:06:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 >> v1:172.21.15.44:6789/0 conn(0x7f3a700055c0 legacy=0x7f3a70007a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a948ad700 1 -- 172.21.15.44:0/2476299089 reap_dead start 2020-11-20T12:06:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c003730 0x7f3a7c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:10.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c0044e0 0x7f3a7c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c004e50 0x7f3a7c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900a56c0 con 0x7f3a7c0044e0 2020-11-20T12:06:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c005360 con 0x7f3a7c003730 2020-11-20T12:06:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c00bc50 con 0x7f3a7c004e50 2020-11-20T12:06:10.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c0044e0 0x7f3a7c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a948ad700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c004e50 0x7f3a7c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.691+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c003730 0x7f3a7c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c004e50 msgr2=0x7f3a7c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c004e50 0x7f3a7c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c003730 msgr2=0x7f3a7c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c003730 0x7f3a7c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/2476299089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c003240 con 0x7f3a7c0044e0 2020-11-20T12:06:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c0044e0 0x7f3a7c004960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f3a84003760 tx=0x7f3a84009040).ready entity=mon.0 client_cookie=460a7434987dc31e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84016760 con 0x7f3a7c0044e0 2020-11-20T12:06:10.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840168c0 con 0x7f3a7c0044e0 2020-11-20T12:06:10.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.692+0000 7f3a8effd700 1 -- 172.21.15.44:0/2476299089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84016c60 con 0x7f3a7c0044e0 2020-11-20T12:06:10.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 -- 172.21.15.44:0/2476299089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c0044e0 msgr2=0x7f3a7c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:10.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c0044e0 0x7f3a7c004960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f3a84003760 tx=0x7f3a84009040).stop 2020-11-20T12:06:10.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 -- 172.21.15.44:0/2476299089 shutdown_connections 2020-11-20T12:06:10.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c004e50 0x7f3a7c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c0044e0 0x7f3a7c004960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 --2- 172.21.15.44:0/2476299089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c003730 0x7f3a7c003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 -- 172.21.15.44:0/2476299089 >> 172.21.15.44:0/2476299089 conn(0x7f3a90062690 msgr2=0x7f3a900a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 -- 172.21.15.44:0/2476299089 shutdown_connections 2020-11-20T12:06:10.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.693+0000 7f3a96310700 1 -- 172.21.15.44:0/2476299089 wait complete. 2020-11-20T12:06:10.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.694+0000 7f3a96310700 1 Processor -- start 2020-11-20T12:06:10.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a96310700 1 -- start start 2020-11-20T12:06:10.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a96310700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c003730 0x7f3a9009d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:10.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a96310700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 0x7f3a900ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:10.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a96310700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 0x7f3a90067050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a96310700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9009f6c0 con 0x7f3a7c003730 2020-11-20T12:06:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a96310700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9009f180 con 0x7f3a7c004e50 2020-11-20T12:06:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a96310700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9009f420 con 0x7f3a7c0044e0 2020-11-20T12:06:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c003730 0x7f3a9009d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a8f7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 0x7f3a900ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c003730 0x7f3a9009d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37862/0 (socket says 172.21.15.44:37862) 2020-11-20T12:06:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.695+0000 7f3a8ffff700 1 -- 172.21.15.44:0/117037104 learned_addr learned my addr 172.21.15.44:0/117037104 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a948ad700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 0x7f3a90067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 msgr2=0x7f3a900ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:06:10.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8f7fe700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 msgr2=0x7f3a900ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:06:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 0x7f3a900ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 0x7f3a900ad030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a948ad700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 msgr2=0x7f3a90067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:06:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a948ad700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 msgr2=0x7f3a90067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:06:10.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a948ad700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 0x7f3a90067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:10.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a948ad700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 0x7f3a90067050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:10.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8ffff700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 msgr2=0x7f3a900ad030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:10.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 0x7f3a900ad030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8ffff700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 msgr2=0x7f3a90067050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:10.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 0x7f3a90067050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8ffff700 1 -- 172.21.15.44:0/117037104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9009ee40 con 0x7f3a7c003730 2020-11-20T12:06:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a8ffff700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c003730 0x7f3a9009d630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f3a80016d50 tx=0x7f3a80008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a6ffff700 1 -- 172.21.15.44:0/117037104 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80014d50 con 0x7f3a7c003730 2020-11-20T12:06:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a6ffff700 1 -- 172.21.15.44:0/117037104 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8001b070 con 0x7f3a7c003730 2020-11-20T12:06:10.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a6ffff700 1 -- 172.21.15.44:0/117037104 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80017440 con 0x7f3a7c003730 2020-11-20T12:06:10.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a900ad520 con 0x7f3a7c003730 2020-11-20T12:06:10.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.696+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a90067780 con 0x7f3a7c003730 2020-11-20T12:06:10.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.698+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a900679d0 con 0x7f3a7c003730 2020-11-20T12:06:10.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.698+0000 7f3a6ffff700 1 -- 172.21.15.44:0/117037104 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3a80027070 con 0x7f3a7c003730 2020-11-20T12:06:10.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.698+0000 7f3a6ffff700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a68056d20 0x7f3a680591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:10.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.698+0000 7f3a6ffff700 1 -- 172.21.15.44:0/117037104 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f3a80070df0 con 0x7f3a7c003730 2020-11-20T12:06:10.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.698+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a68056d20 0x7f3a680591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:10.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.699+0000 7f3a8f7fe700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a68056d20 0x7f3a680591e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f3a90143050 tx=0x7f3a8401d000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:10.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.702+0000 7f3a6ffff700 1 -- 172.21.15.44:0/117037104 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3a800460a0 con 0x7f3a7c003730 2020-11-20T12:06:10.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.943+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3a90061650 con 0x7f3a7c003730 2020-11-20T12:06:10.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.944+0000 7f3a6ffff700 1 -- 172.21.15.44:0/117037104 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7f3a800496c0 con 0x7f3a7c003730 2020-11-20T12:06:10.948 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":22,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:06:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a68056d20 msgr2=0x7f3a680591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:10.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a68056d20 0x7f3a680591e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f3a90143050 tx=0x7f3a8401d000).stop 2020-11-20T12:06:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c003730 msgr2=0x7f3a9009d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c003730 0x7f3a9009d630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f3a80016d50 tx=0x7f3a80008040).stop 2020-11-20T12:06:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 shutdown_connections 2020-11-20T12:06:10.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a68056d20 0x7f3a680591e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a7c004e50 0x7f3a90067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a7c003730 0x7f3a9009d630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 --2- 172.21.15.44:0/117037104 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a7c0044e0 0x7f3a900ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.945+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 >> 172.21.15.44:0/117037104 conn(0x7f3a90062690 msgr2=0x7f3a900a3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:10.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.946+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 shutdown_connections 2020-11-20T12:06:10.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:10.946+0000 7f3a96310700 1 -- 172.21.15.44:0/117037104 wait complete. 2020-11-20T12:06:11.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.699+0000 7ff6affff700 1 -- 172.21.15.44:0/3075895340 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7ff6c4048880 con 0x7ff6d00a8b40 2020-11-20T12:06:11.705 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff6a8056cd0 msgr2=0x7ff6a8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:11.706 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff6a8056cd0 0x7ff6a8059190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff6c0003650 tx=0x7ff6c0006ae0).stop 2020-11-20T12:06:11.706 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a8b40 msgr2=0x7ff6d00a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:11.706 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a8b40 0x7ff6d00a7910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff6c400c760 tx=0x7ff6c4007cb0).stop 2020-11-20T12:06:11.706 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 shutdown_connections 2020-11-20T12:06:11.706 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff6a8056cd0 0x7ff6a8059190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:11.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6d013b3b0 0x7ff6d00a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:11.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6d00a3930 0x7ff6d00a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:11.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 --2- 172.21.15.44:0/3075895340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6d00a8b40 0x7ff6d00a7910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:11.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 >> 172.21.15.44:0/3075895340 conn(0x7ff6d00aced0 msgr2=0x7ff6d00ab4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:11.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 shutdown_connections 2020-11-20T12:06:11.708 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:11.701+0000 7ff6d6571700 1 -- 172.21.15.44:0/3075895340 wait complete. 2020-11-20T12:06:11.708 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:06:11.717 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 1 2020-11-20T12:06:11.719 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 2 2020-11-20T12:06:11.722 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 3 2020-11-20T12:06:11.725 INFO:tasks.workunit.client.0.smithi044.stderr:+ '[' -e file4 ']' 2020-11-20T12:06:11.725 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/snap1 2020-11-20T12:06:11.742 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 4 2020-11-20T12:06:11.744 INFO:tasks.workunit.client.0.smithi044.stderr:++ ls 2020-11-20T12:06:11.751 INFO:tasks.workunit.client.0.smithi044.stderr:+ now='file1 2020-11-20T12:06:11.752 INFO:tasks.workunit.client.0.smithi044.stderr:file2 2020-11-20T12:06:11.752 INFO:tasks.workunit.client.0.smithi044.stderr:file3 2020-11-20T12:06:11.752 INFO:tasks.workunit.client.0.smithi044.stderr:file4' 2020-11-20T12:06:11.753 INFO:tasks.workunit.client.0.smithi044.stderr:++ ls .snap/snap1 2020-11-20T12:06:11.757 INFO:tasks.workunit.client.0.smithi044.stderr:+ then='file1 2020-11-20T12:06:11.758 INFO:tasks.workunit.client.0.smithi044.stderr:file2 2020-11-20T12:06:11.758 INFO:tasks.workunit.client.0.smithi044.stderr:file3' 2020-11-20T12:06:11.758 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir .snap/snap1 2020-11-20T12:06:11.769 INFO:tasks.workunit.client.0.smithi044.stderr:+ '[' 'file1 2020-11-20T12:06:11.769 INFO:tasks.workunit.client.0.smithi044.stderr:file2 2020-11-20T12:06:11.770 INFO:tasks.workunit.client.0.smithi044.stderr:file3 2020-11-20T12:06:11.770 INFO:tasks.workunit.client.0.smithi044.stderr:file4' = 'file1 2020-11-20T12:06:11.770 INFO:tasks.workunit.client.0.smithi044.stderr:file2 2020-11-20T12:06:11.770 INFO:tasks.workunit.client.0.smithi044.stderr:file3' ']' 2020-11-20T12:06:11.770 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 1 2020-11-20T12:06:11.780 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 2 2020-11-20T12:06:11.792 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 3 2020-11-20T12:06:11.801 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/snap1 2020-11-20T12:06:11.813 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 4 2020-11-20T12:06:11.821 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir .snap/snap1 2020-11-20T12:06:11.836 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm file1 file2 file3 file4 2020-11-20T12:06:11.846 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:06:11.847 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:06:11.847 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:06:11.848 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:06:11.893 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2020-11-20T12:06:11.894 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2020-11-20T12:06:12.021 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.017+0000 7f1380864700 1 Processor -- start 2020-11-20T12:06:12.021 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.017+0000 7f1380864700 1 -- start start 2020-11-20T12:06:12.031 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f1380864700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a3930 0x7f137c1380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:12.032 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f1380864700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f137c0a3d90 0x7f137c1385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:12.032 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f1380864700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137c064ed0 con 0x7f137c0a8b40 2020-11-20T12:06:12.033 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f1380864700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0a6320 con 0x7f137c0a3d90 2020-11-20T12:06:12.033 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f1380864700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0a5be0 con 0x7f137c0a3930 2020-11-20T12:06:12.034 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f137ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f137c0a3d90 0x7f137c1385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:12.034 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a3930 0x7f137c1380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:12.034 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f137ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f137c0a3d90 0x7f137c1385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58156/0 (socket says 172.21.15.44:58156) 2020-11-20T12:06:12.035 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f137ad9d700 1 -- 172.21.15.44:0/2208554308 learned_addr learned my addr 172.21.15.44:0/2208554308 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:12.035 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f137ad9d700 1 -- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a3930 msgr2=0x7f137c1380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:12.035 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f137ad9d700 1 --2- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a3930 0x7f137c1380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:12.036 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f137ad9d700 1 -- 172.21.15.44:0/2208554308 >> v1:172.21.15.79:6789/0 conn(0x7f137c0a8b40 legacy=0x7f137c13aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:12.036 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.018+0000 7f137ad9d700 1 -- 172.21.15.44:0/2208554308 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0a5e80 con 0x7f137c0a3d90 2020-11-20T12:06:12.036 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1379d9b700 1 --2- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a3930 0x7f137c1380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:12.037 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f137ad9d700 1 --2- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3d90 0x7f137c1385a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1374009d50 tx=0x7f1374009290).ready entity=mon.1 client_cookie=bdf62bd3e198f753 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:12.037 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f137959a700 1 -- 172.21.15.44:0/2208554308 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137400bbc0 con 0x7f137c0a3d90 2020-11-20T12:06:12.037 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f137959a700 1 -- 172.21.15.44:0/2208554308 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137400bd20 con 0x7f137c0a3d90 2020-11-20T12:06:12.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f137959a700 1 -- 172.21.15.44:0/2208554308 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374011610 con 0x7f137c0a3d90 2020-11-20T12:06:12.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 -- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3d90 msgr2=0x7f137c1385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:12.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 --2- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3d90 0x7f137c1385a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1374009d50 tx=0x7f1374009290).stop 2020-11-20T12:06:12.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 -- 172.21.15.44:0/2208554308 shutdown_connections 2020-11-20T12:06:12.039 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 --2- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3d90 0x7f137c1385a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:12.039 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 --2- 172.21.15.44:0/2208554308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a3930 0x7f137c1380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:12.039 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 -- 172.21.15.44:0/2208554308 >> 172.21.15.44:0/2208554308 conn(0x7f137c0aced0 msgr2=0x7f137c0ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:12.039 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 -- 172.21.15.44:0/2208554308 shutdown_connections 2020-11-20T12:06:12.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.019+0000 7f1380864700 1 -- 172.21.15.44:0/2208554308 wait complete. 2020-11-20T12:06:12.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.020+0000 7f1380864700 1 Processor -- start 2020-11-20T12:06:12.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.020+0000 7f1380864700 1 -- start start 2020-11-20T12:06:12.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1380864700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3930 0x7f137c13a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:12.041 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1380864700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a3d90 0x7f137c12fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:12.041 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1380864700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a8b40 0x7f137c1301c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:12.041 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1380864700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c139750 con 0x7f137c0a3d90 2020-11-20T12:06:12.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1380864700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c1306b0 con 0x7f137c0a3930 2020-11-20T12:06:12.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1380864700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0acab0 con 0x7f137c0a8b40 2020-11-20T12:06:12.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a3d90 0x7f137c12fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:12.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a3d90 0x7f137c12fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37878/0 (socket says 172.21.15.44:37878) 2020-11-20T12:06:12.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1379d9b700 1 -- 172.21.15.44:0/2464213472 learned_addr learned my addr 172.21.15.44:0/2464213472 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:12.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f137ad9d700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a8b40 0x7f137c1301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:12.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f137a59c700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3930 0x7f137c13a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:12.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1379d9b700 1 -- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a8b40 msgr2=0x7f137c1301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:12.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1379d9b700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a8b40 0x7f137c1301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:12.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1379d9b700 1 -- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3930 msgr2=0x7f137c13a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:12.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.021+0000 7f1379d9b700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3930 0x7f137c13a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:12.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.022+0000 7f1379d9b700 1 -- 172.21.15.44:0/2464213472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0acce0 con 0x7f137c0a3d90 2020-11-20T12:06:12.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.022+0000 7f1379d9b700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a3d90 0x7f137c12fcd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f136c00ef80 tx=0x7f136c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:12.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.022+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00cb90 con 0x7f137c0a3d90 2020-11-20T12:06:12.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.022+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c00ccf0 con 0x7f137c0a3d90 2020-11-20T12:06:12.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.022+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c12f590 con 0x7f137c0a3d90 2020-11-20T12:06:12.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.022+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c012600 con 0x7f137c0a3d90 2020-11-20T12:06:12.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.022+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c0aace0 con 0x7f137c0a3d90 2020-11-20T12:06:12.046 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.023+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c05c590 con 0x7f137c0a3d90 2020-11-20T12:06:12.046 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.023+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f136c0129b0 con 0x7f137c0a3d90 2020-11-20T12:06:12.046 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.023+0000 7f136a7fc700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:12.046 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.023+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f136c070760 con 0x7f137c0a3d90 2020-11-20T12:06:12.046 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.023+0000 7f137a59c700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:12.047 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.024+0000 7f137a59c700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f1370003670 tx=0x7f1370007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:12.047 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.027+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f136c045a10 con 0x7f137c0a3d90 2020-11-20T12:06:12.265 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.262+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f137c0a8f20 con 0x7f137c0a3d90 2020-11-20T12:06:12.708 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.705+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v18) v1 ==== 167+0+0 (secure 0 0 0) 0x7f136c049030 con 0x7f137c0a3d90 2020-11-20T12:06:12.889 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:12.886+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f137c0a4280 con 0x7f137c0a3d90 2020-11-20T12:06:13.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.707+0000 7f136a7fc700 1 -- 172.21.15.44:0/2464213472 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v19) v1 ==== 167+0+0 (secure 0 0 0) 0x7f136c049210 con 0x7f137c0a3d90 2020-11-20T12:06:13.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 msgr2=0x7f13500591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:13.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f1370003670 tx=0x7f1370007b90).stop 2020-11-20T12:06:13.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a3d90 msgr2=0x7f137c12fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:13.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a3d90 0x7f137c12fcd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f136c00ef80 tx=0x7f136c006df0).stop 2020-11-20T12:06:13.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 shutdown_connections 2020-11-20T12:06:13.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:13.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a8b40 0x7f137c1301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:13.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0a3930 0x7f137c13a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:13.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 --2- 172.21.15.44:0/2464213472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a3d90 0x7f137c12fcd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:13.716 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 >> 172.21.15.44:0/2464213472 conn(0x7f137c0aced0 msgr2=0x7f137c0642c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:13.716 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 shutdown_connections 2020-11-20T12:06:13.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:06:13.709+0000 7f1380864700 1 -- 172.21.15.44:0/2464213472 wait complete. 2020-11-20T12:06:13.718 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:06:13.725 INFO:tasks.workunit.client.0.smithi044.stdout:Create dir 100 to 199 ... 2020-11-20T12:06:13.735 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 100 2020-11-20T12:06:13.766 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 101 2020-11-20T12:06:13.797 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 102 2020-11-20T12:06:13.826 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 103 2020-11-20T12:06:13.857 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 104 2020-11-20T12:06:13.887 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 105 2020-11-20T12:06:13.912 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 106 2020-11-20T12:06:13.939 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 107 2020-11-20T12:06:13.962 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:06:13.967 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 108 2020-11-20T12:06:13.999 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 109 2020-11-20T12:06:14.031 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 110 2020-11-20T12:06:14.062 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 111 2020-11-20T12:06:14.092 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 112 2020-11-20T12:06:14.120 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 113 2020-11-20T12:06:14.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.140+0000 7f4140d87700 1 Processor -- start 2020-11-20T12:06:14.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.140+0000 7f4140d87700 1 -- start start 2020-11-20T12:06:14.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4140d87700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f413c0aad00 0x7f413c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4140d87700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f413c0ab0e0 0x7f413c13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4140d87700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413c1385f0 con 0x7f413c0a86d0 2020-11-20T12:06:14.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4140d87700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c13e250 con 0x7f413c0ab0e0 2020-11-20T12:06:14.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4140d87700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c13e4f0 con 0x7f413c0aad00 2020-11-20T12:06:14.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4133fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f413c0ab0e0 0x7f413c13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f413a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f413c0aad00 0x7f413c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4133fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f413c0ab0e0 0x7f413c13ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37888/0 (socket says 172.21.15.44:37888) 2020-11-20T12:06:14.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f413a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f413c0aad00 0x7f413c0a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58168/0 (socket says 172.21.15.44:58168) 2020-11-20T12:06:14.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f4133fff700 1 -- 172.21.15.44:0/219385498 learned_addr learned my addr 172.21.15.44:0/219385498 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:14.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f413a59c700 1 -- 172.21.15.44:0/219385498 >> v2:172.21.15.44:3300/0 conn(0x7f413c0ab0e0 msgr2=0x7f413c13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:14.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f413a59c700 1 --2- 172.21.15.44:0/219385498 >> v2:172.21.15.44:3300/0 conn(0x7f413c0ab0e0 0x7f413c13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f413a59c700 1 -- 172.21.15.44:0/219385498 >> v1:172.21.15.79:6789/0 conn(0x7f413c0a86d0 legacy=0x7f413c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.141+0000 7f413a59c700 1 -- 172.21.15.44:0/219385498 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c0ac340 con 0x7f413c0aad00 2020-11-20T12:06:14.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.142+0000 7f413a59c700 1 --2- 172.21.15.44:0/219385498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c0a94d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f413c060f50 tx=0x7f412c006df0).ready entity=mon.1 client_cookie=41f32ef00151ab9f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.142+0000 7f4139d9b700 1 -- 172.21.15.44:0/219385498 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0097b0 con 0x7f413c0aad00 2020-11-20T12:06:14.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.142+0000 7f4139d9b700 1 -- 172.21.15.44:0/219385498 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c009910 con 0x7f413c0aad00 2020-11-20T12:06:14.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.142+0000 7f4139d9b700 1 -- 172.21.15.44:0/219385498 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c009c90 con 0x7f413c0aad00 2020-11-20T12:06:14.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.144+0000 7f4140d87700 1 -- 172.21.15.44:0/219385498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 msgr2=0x7f413c0a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.144+0000 7f4140d87700 1 --2- 172.21.15.44:0/219385498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c0a94d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f413c060f50 tx=0x7f412c006df0).stop 2020-11-20T12:06:14.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.145+0000 7f4140d87700 1 -- 172.21.15.44:0/219385498 shutdown_connections 2020-11-20T12:06:14.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.145+0000 7f4140d87700 1 --2- 172.21.15.44:0/219385498 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c0a94d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.145+0000 7f4140d87700 1 --2- 172.21.15.44:0/219385498 >> v2:172.21.15.44:3300/0 conn(0x7f413c0ab0e0 0x7f413c13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.145+0000 7f4140d87700 1 -- 172.21.15.44:0/219385498 >> 172.21.15.44:0/219385498 conn(0x7f413c060170 msgr2=0x7f413c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:14.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.146+0000 7f4140d87700 1 -- 172.21.15.44:0/219385498 shutdown_connections 2020-11-20T12:06:14.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.146+0000 7f4140d87700 1 -- 172.21.15.44:0/219385498 wait complete. 2020-11-20T12:06:14.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.148+0000 7f4140d87700 1 Processor -- start 2020-11-20T12:06:14.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.148+0000 7f4140d87700 1 -- start start 2020-11-20T12:06:14.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4140d87700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f413c0a86d0 0x7f413c0a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4140d87700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c09d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4140d87700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f413c0ab0e0 0x7f413c09e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4140d87700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c09df90 con 0x7f413c0ab0e0 2020-11-20T12:06:14.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4140d87700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c09d9d0 con 0x7f413c0aad00 2020-11-20T12:06:14.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4140d87700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c09e1c0 con 0x7f413c0a86d0 2020-11-20T12:06:14.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f413a59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f413c0a86d0 0x7f413c0a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4133fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c09d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4133fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c09d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58174/0 (socket says 172.21.15.44:58174) 2020-11-20T12:06:14.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f413a59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f413c0a86d0 0x7f413c0a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49328/0 (socket says 172.21.15.44:49328) 2020-11-20T12:06:14.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4133fff700 1 -- 172.21.15.44:0/689795011 learned_addr learned my addr 172.21.15.44:0/689795011 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:14.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f413a59c700 1 -- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 msgr2=0x7f413c09d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f413a59c700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c09d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f413a59c700 1 -- 172.21.15.44:0/689795011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f413c0ab0e0 msgr2=0x7f413c09e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f413a59c700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f413c0ab0e0 0x7f413c09e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:14.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f413a59c700 1 -- 172.21.15.44:0/689795011 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c09dc60 con 0x7f413c0a86d0 2020-11-20T12:06:14.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.149+0000 7f4133fff700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c09d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:14.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.150+0000 7f413a59c700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f413c0a86d0 0x7f413c0a7590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f413c0a9d40 tx=0x7f412c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.150+0000 7f41327fc700 1 -- 172.21.15.44:0/689795011 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c011dc0 con 0x7f413c0a86d0 2020-11-20T12:06:14.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.150+0000 7f41327fc700 1 -- 172.21.15.44:0/689795011 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c01e460 con 0x7f413c0a86d0 2020-11-20T12:06:14.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.150+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413c09eb50 con 0x7f413c0a86d0 2020-11-20T12:06:14.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.150+0000 7f41327fc700 1 -- 172.21.15.44:0/689795011 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c01e800 con 0x7f413c0a86d0 2020-11-20T12:06:14.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.150+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c0a05e0 con 0x7f413c0a86d0 2020-11-20T12:06:14.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.152+0000 7f41327fc700 1 -- 172.21.15.44:0/689795011 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f412c025030 con 0x7f413c0a86d0 2020-11-20T12:06:14.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.152+0000 7f41327fc700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4118056d70 0x7f4118059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.152+0000 7f41327fc700 1 -- 172.21.15.44:0/689795011 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f412c071830 con 0x7f413c0a86d0 2020-11-20T12:06:14.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.152+0000 7f4133fff700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4118056d70 0x7f4118059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.152+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413c069680 con 0x7f413c0a86d0 2020-11-20T12:06:14.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.153+0000 7f4133fff700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4118056d70 0x7f4118059230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f413c09fbb0 tx=0x7f4124008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.160 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 114 2020-11-20T12:06:14.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.156+0000 7f41327fc700 1 -- 172.21.15.44:0/689795011 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f412c023070 con 0x7f413c0a86d0 2020-11-20T12:06:14.184 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 115 2020-11-20T12:06:14.218 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 116 2020-11-20T12:06:14.250 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 117 2020-11-20T12:06:14.281 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 118 2020-11-20T12:06:14.310 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 119 2020-11-20T12:06:14.344 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 120 2020-11-20T12:06:14.367 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 121 2020-11-20T12:06:14.398 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 122 2020-11-20T12:06:14.426 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 123 2020-11-20T12:06:14.456 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 124 2020-11-20T12:06:14.483 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 125 2020-11-20T12:06:14.516 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 126 2020-11-20T12:06:14.544 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 127 2020-11-20T12:06:14.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.553+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f413c0446d0 con 0x7f413c0a86d0 2020-11-20T12:06:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.554+0000 7f41327fc700 1 -- 172.21.15.44:0/689795011 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f412c02d080 con 0x7f413c0a86d0 2020-11-20T12:06:14.558 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:06:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.556+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4118056d70 msgr2=0x7f4118059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.557+0000 7f4140d87700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4118056d70 0x7f4118059230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f413c09fbb0 tx=0x7f4124008040).stop 2020-11-20T12:06:14.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.557+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f413c0a86d0 msgr2=0x7f413c0a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.557+0000 7f4140d87700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f413c0a86d0 0x7f413c0a7590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f413c0a9d40 tx=0x7f412c015000).stop 2020-11-20T12:06:14.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 shutdown_connections 2020-11-20T12:06:14.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4118056d70 0x7f4118059230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f413c0ab0e0 0x7f413c09e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:14.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f413c0a86d0 0x7f413c0a7590 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 --2- 172.21.15.44:0/689795011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f413c0aad00 0x7f413c09d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 >> 172.21.15.44:0/689795011 conn(0x7f413c060170 msgr2=0x7f413c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:14.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 shutdown_connections 2020-11-20T12:06:14.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.558+0000 7f4140d87700 1 -- 172.21.15.44:0/689795011 wait complete. 2020-11-20T12:06:14.575 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 128 2020-11-20T12:06:14.578 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:06:14.578 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:06:14.602 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 129 2020-11-20T12:06:14.634 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 130 2020-11-20T12:06:14.667 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 131 2020-11-20T12:06:14.700 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 132 2020-11-20T12:06:14.734 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 133 2020-11-20T12:06:14.735 INFO:tasks.ceph.osd.4.smithi079.stderr:2020-11-20T12:06:14.731+0000 7fa86ae90700 -1 osd.4 pg_epoch: 28 pg[4.1s0( v 27'59 (0'0,27'59] local-lis/les=23/24 n=43 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) [4,5,6,7]p4(0) r=0 lpr=23 crt=27'59 lcod 27'58 mlcod 27'58 active+clean+snaptrim trimq=[a~2] ps=[2~8]] removing snap head 2020-11-20T12:06:14.770 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 134 2020-11-20T12:06:14.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.793+0000 7f17eec34700 1 Processor -- start 2020-11-20T12:06:14.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.793+0000 7f17eec34700 1 -- start start 2020-11-20T12:06:14.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17eec34700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f17e8068f60 0x7f17e813aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17eec34700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f17e813afe0 0x7f17e813e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17eec34700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e813b520 con 0x7f17e813e910 2020-11-20T12:06:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17eec34700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e80660c0 con 0x7f17e8068f60 2020-11-20T12:06:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17eec34700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e813b7c0 con 0x7f17e813afe0 2020-11-20T12:06:14.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17ec9d0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f17e8068f60 0x7f17e813aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17ed1d1700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f17e813e910 0x7f17e8060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44524/0 (socket says 172.21.15.44:44524) 2020-11-20T12:06:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17ed1d1700 1 -- 172.21.15.44:0/1308739494 learned_addr learned my addr 172.21.15.44:0/1308739494 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:06:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17e7fff700 1 --2- 172.21.15.44:0/1308739494 >> v2:172.21.15.79:3300/0 conn(0x7f17e813afe0 0x7f17e813e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17ec9d0700 1 -- 172.21.15.44:0/1308739494 >> v2:172.21.15.79:3300/0 conn(0x7f17e813afe0 msgr2=0x7f17e813e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17ec9d0700 1 --2- 172.21.15.44:0/1308739494 >> v2:172.21.15.79:3300/0 conn(0x7f17e813afe0 0x7f17e813e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17ec9d0700 1 -- 172.21.15.44:0/1308739494 >> v1:172.21.15.79:6789/0 conn(0x7f17e813e910 legacy=0x7f17e8060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17ec9d0700 1 -- 172.21.15.44:0/1308739494 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e813ed80 con 0x7f17e8068f60 2020-11-20T12:06:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.794+0000 7f17e77fe700 1 -- 172.21.15.44:0/1308739494 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139397421 0 0) 0x7f17e813b520 con 0x7f17e813e910 2020-11-20T12:06:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.795+0000 7f17ec9d0700 1 --2- 172.21.15.44:0/1308739494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 0x7f17e813aaf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f17d800b9a0 tx=0x7f17d8006df0).ready entity=mon.0 client_cookie=6053904e9b05246 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.796+0000 7f17e77fe700 1 -- 172.21.15.44:0/1308739494 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8009770 con 0x7f17e8068f60 2020-11-20T12:06:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.796+0000 7f17e77fe700 1 -- 172.21.15.44:0/1308739494 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d80098d0 con 0x7f17e8068f60 2020-11-20T12:06:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.796+0000 7f17e77fe700 1 -- 172.21.15.44:0/1308739494 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8009c50 con 0x7f17e8068f60 2020-11-20T12:06:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 -- 172.21.15.44:0/1308739494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 msgr2=0x7f17e813aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 --2- 172.21.15.44:0/1308739494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 0x7f17e813aaf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f17d800b9a0 tx=0x7f17d8006df0).stop 2020-11-20T12:06:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 -- 172.21.15.44:0/1308739494 shutdown_connections 2020-11-20T12:06:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 --2- 172.21.15.44:0/1308739494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 0x7f17e813aaf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 --2- 172.21.15.44:0/1308739494 >> v2:172.21.15.79:3300/0 conn(0x7f17e813afe0 0x7f17e813e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 -- 172.21.15.44:0/1308739494 >> 172.21.15.44:0/1308739494 conn(0x7f17e8062cf0 msgr2=0x7f17e80ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 -- 172.21.15.44:0/1308739494 shutdown_connections 2020-11-20T12:06:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.797+0000 7f17eec34700 1 -- 172.21.15.44:0/1308739494 wait complete. 2020-11-20T12:06:14.806 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 135 2020-11-20T12:06:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.800+0000 7f17eec34700 1 Processor -- start 2020-11-20T12:06:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.800+0000 7f17eec34700 1 -- start start 2020-11-20T12:06:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17eec34700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 0x7f17e8140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17eec34700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17e813afe0 0x7f17e8137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17eec34700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17e813e910 0x7f17e809f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17eec34700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e809e950 con 0x7f17e8068f60 2020-11-20T12:06:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17eec34700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e809ebf0 con 0x7f17e813e910 2020-11-20T12:06:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17eec34700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e809ee90 con 0x7f17e813afe0 2020-11-20T12:06:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17e813e910 0x7f17e809f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ec9d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 0x7f17e8140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17e7fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17e813afe0 0x7f17e8137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17e813e910 0x7f17e809f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58190/0 (socket says 172.21.15.44:58190) 2020-11-20T12:06:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 -- 172.21.15.44:0/1549573116 learned_addr learned my addr 172.21.15.44:0/1549573116 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17e813afe0 msgr2=0x7f17e8137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f17e813afe0 0x7f17e8137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 msgr2=0x7f17e8140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 0x7f17e8140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 -- 172.21.15.44:0/1549573116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8140a50 con 0x7f17e813e910 2020-11-20T12:06:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ec9d0700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e8068f60 0x7f17e8140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.801+0000 7f17ed1d1700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17e813e910 0x7f17e809f560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f17e000cae0 tx=0x7f17e0001320).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.802+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0005b80 con 0x7f17e813e910 2020-11-20T12:06:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.802+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e0005ce0 con 0x7f17e813e910 2020-11-20T12:06:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.802+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0007360 con 0x7f17e813e910 2020-11-20T12:06:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.802+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e809d5e0 con 0x7f17e813e910 2020-11-20T12:06:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.802+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e809e6e0 con 0x7f17e813e910 2020-11-20T12:06:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.802+0000 7f17eec34700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e81381c0 0x7f17e809fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.803+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e809f2c0 con 0x7f17e81381c0 2020-11-20T12:06:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.803+0000 7f17ec9d0700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e81381c0 0x7f17e809fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.803+0000 7f17ec9d0700 1 -- 172.21.15.44:0/1549573116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17e8063100 con 0x7f17e81381c0 2020-11-20T12:06:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.803+0000 7f17ec9d0700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e81381c0 0x7f17e809fa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f17d800b9a0 tx=0x7f17d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.803+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f17e0010030 con 0x7f17e813e910 2020-11-20T12:06:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17c0056da0 0x7f17c0059260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f17e006fe10 con 0x7f17e813e910 2020-11-20T12:06:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d801f720 con 0x7f17e81381c0 2020-11-20T12:06:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e7fff700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17c0056da0 0x7f17c0059260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f17d801f880 con 0x7f17e81381c0 2020-11-20T12:06:14.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e81381c0 msgr2=0x7f17e809fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e81381c0 0x7f17e809fa50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f17d800b9a0 tx=0x7f17d8015000).stop 2020-11-20T12:06:14.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e81381c0 msgr2=0x7f17e809fa50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:14.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.804+0000 7f17e4ff9700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e81381c0 0x7f17e809fa50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.808+0000 7f17e7fff700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17c0056da0 0x7f17c0059260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f17e813ab20 tx=0x7f17dc006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.817+0000 7f17eec34700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e80a1760 0x7f17e8139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:14.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.817+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8069340 con 0x7f17e80a1760 2020-11-20T12:06:14.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.817+0000 7f17ec9d0700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e80a1760 0x7f17e8139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:14.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.817+0000 7f17ec9d0700 1 -- 172.21.15.44:0/1549573116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f17e80446d0 con 0x7f17e80a1760 2020-11-20T12:06:14.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.818+0000 7f17ec9d0700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e80a1760 0x7f17e8139a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f17d8006e30 tx=0x7f17d8021070).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:14.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.818+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8028470 con 0x7f17e80a1760 2020-11-20T12:06:14.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.818+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f17d801f510 con 0x7f17e80a1760 2020-11-20T12:06:14.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.818+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e80a1760 msgr2=0x7f17e8139a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.818+0000 7f17e4ff9700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e80a1760 0x7f17e8139a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f17d8006e30 tx=0x7f17d8021070).stop 2020-11-20T12:06:14.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.818+0000 7f17e4ff9700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e80a1760 msgr2=0x7f17e8139a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:14.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.818+0000 7f17e4ff9700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e80a1760 0x7f17e8139a30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:14.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17c0056da0 msgr2=0x7f17c0059260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17c0056da0 0x7f17c0059260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f17e813ab20 tx=0x7f17dc006df0).stop 2020-11-20T12:06:14.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17e813e910 msgr2=0x7f17e809f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:14.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 --2- 172.21.15.44:0/1549573116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17e813e910 0x7f17e809f560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f17e000cae0 tx=0x7f17e0001320).stop 2020-11-20T12:06:14.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17ed1d1700 1 -- 172.21.15.44:0/1549573116 reap_dead start 2020-11-20T12:06:14.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17ed1d1700 1 -- 172.21.15.44:0/1549573116 reap_dead start 2020-11-20T12:06:14.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 shutdown_connections 2020-11-20T12:06:14.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 >> 172.21.15.44:0/1549573116 conn(0x7f17e8062cf0 msgr2=0x7f17e8067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:14.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 shutdown_connections 2020-11-20T12:06:14.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:14.819+0000 7f17eec34700 1 -- 172.21.15.44:0/1549573116 wait complete. 2020-11-20T12:06:14.828 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:06:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 26, 2020-11-20T12:06:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:06:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:06:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:06:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:06:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:06:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:06:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:06:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:06:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:14.865 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:14.867 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 136 2020-11-20T12:06:14.867 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 137 2020-11-20T12:06:14.868 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:06:14.888 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 138 2020-11-20T12:06:14.919 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 139 2020-11-20T12:06:14.946 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 140 2020-11-20T12:06:14.979 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 141 2020-11-20T12:06:15.009 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 142 2020-11-20T12:06:15.042 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 143 2020-11-20T12:06:15.069 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 144 2020-11-20T12:06:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.072+0000 7fd88be55700 1 Processor -- start 2020-11-20T12:06:15.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.072+0000 7fd88be55700 1 -- start start 2020-11-20T12:06:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd88be55700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd88413ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd88be55700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8840ae7d0 0x7fd8840ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd88be55700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8840a9800 con 0x7fd8840a9050 2020-11-20T12:06:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd88be55700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8841385a0 con 0x7fd8840ae7d0 2020-11-20T12:06:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd88be55700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840a9560 con 0x7fd8840ae3f0 2020-11-20T12:06:15.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd8893f0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8840ae7d0 0x7fd8840ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd8893f0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8840ae7d0 0x7fd8840ab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58200/0 (socket says 172.21.15.44:58200) 2020-11-20T12:06:15.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd8893f0700 1 -- 172.21.15.44:0/3837328085 learned_addr learned my addr 172.21.15.44:0/3837328085 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:15.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.073+0000 7fd889bf1700 1 --2- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd88413ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd8893f0700 1 -- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 msgr2=0x7fd88413ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:15.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd8893f0700 1 --2- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd88413ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd8893f0700 1 -- 172.21.15.44:0/3837328085 >> v1:172.21.15.79:6789/0 conn(0x7fd8840a9050 legacy=0x7fd88413b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:15.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd8893f0700 1 -- 172.21.15.44:0/3837328085 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840aad60 con 0x7fd8840ae7d0 2020-11-20T12:06:15.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd8893f0700 1 --2- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840ab200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd874006fd0 tx=0x7fd874006eb0).ready entity=mon.1 client_cookie=d9a70f57e727e72f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd888bef700 1 -- 172.21.15.44:0/3837328085 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8740097d0 con 0x7fd8840ae7d0 2020-11-20T12:06:15.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd888bef700 1 -- 172.21.15.44:0/3837328085 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd874009930 con 0x7fd8840ae7d0 2020-11-20T12:06:15.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.074+0000 7fd888bef700 1 -- 172.21.15.44:0/3837328085 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874009cb0 con 0x7fd8840ae7d0 2020-11-20T12:06:15.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.075+0000 7fd88be55700 1 -- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 msgr2=0x7fd8840ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.075+0000 7fd88be55700 1 --2- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840ab200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd874006fd0 tx=0x7fd874006eb0).stop 2020-11-20T12:06:15.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.076+0000 7fd88be55700 1 -- 172.21.15.44:0/3837328085 shutdown_connections 2020-11-20T12:06:15.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.076+0000 7fd88be55700 1 --2- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd88413ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.076+0000 7fd88be55700 1 --2- 172.21.15.44:0/3837328085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840ab200 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.076+0000 7fd88be55700 1 -- 172.21.15.44:0/3837328085 >> 172.21.15.44:0/3837328085 conn(0x7fd8840b1d10 msgr2=0x7fd8840adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:15.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.076+0000 7fd88be55700 1 -- 172.21.15.44:0/3837328085 shutdown_connections 2020-11-20T12:06:15.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.076+0000 7fd88be55700 1 -- 172.21.15.44:0/3837328085 wait complete. 2020-11-20T12:06:15.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.078+0000 7fd88be55700 1 Processor -- start 2020-11-20T12:06:15.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.078+0000 7fd88be55700 1 -- start start 2020-11-20T12:06:15.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88be55700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8840a9050 0x7fd8840a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88be55700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd8840a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88be55700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88be55700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88414e640 con 0x7fd8840a9050 2020-11-20T12:06:15.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88be55700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840a47a0 con 0x7fd8840ae7d0 2020-11-20T12:06:15.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88be55700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840a58b0 con 0x7fd8840ae3f0 2020-11-20T12:06:15.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd8893f0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd8840a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88a3f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd889bf1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8840a9050 0x7fd8840a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd8893f0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd8840a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49360/0 (socket says 172.21.15.44:49360) 2020-11-20T12:06:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd8893f0700 1 -- 172.21.15.44:0/4272702772 learned_addr learned my addr 172.21.15.44:0/4272702772 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:15.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88a3f2700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 msgr2=0x7fd8840a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88a3f2700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd8840a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88a3f2700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8840a9050 msgr2=0x7fd8840a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88a3f2700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8840a9050 0x7fd8840a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88a3f2700 1 -- 172.21.15.44:0/4272702772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840a5b10 con 0x7fd8840ae7d0 2020-11-20T12:06:15.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd8893f0700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd8840ae3f0 0x7fd8840a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd889bf1700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8840a9050 0x7fd8840a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:15.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.079+0000 7fd88a3f2700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840a5fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd87c00c860 tx=0x7fd87c007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.080+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00e560 con 0x7fd8840ae7d0 2020-11-20T12:06:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.080+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00e6c0 con 0x7fd8840ae7d0 2020-11-20T12:06:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.080+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00ea60 con 0x7fd8840ae7d0 2020-11-20T12:06:15.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.080+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88413ab20 con 0x7fd8840ae7d0 2020-11-20T12:06:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.080+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8840a66e0 con 0x7fd8840ae7d0 2020-11-20T12:06:15.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.081+0000 7fd88be55700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414d8c0 0x7fd88414dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.081+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88405c690 con 0x7fd88414d8c0 2020-11-20T12:06:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.081+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd87c010070 con 0x7fd8840ae7d0 2020-11-20T12:06:15.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.081+0000 7fd889bf1700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414d8c0 0x7fd88414dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd85c056d20 0x7fd85c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fd87c06fd80 con 0x7fd8840ae7d0 2020-11-20T12:06:15.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd889bf1700 1 -- 172.21.15.44:0/4272702772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8840a5ca0 con 0x7fd88414d8c0 2020-11-20T12:06:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd889bf1700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414d8c0 0x7fd88414dca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd88000f470 tx=0x7fd880006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd8893f0700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd85c056d20 0x7fd85c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8800135b0 con 0x7fd88414d8c0 2020-11-20T12:06:15.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fd880013710 con 0x7fd88414d8c0 2020-11-20T12:06:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414d8c0 msgr2=0x7fd88414dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414d8c0 0x7fd88414dca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd88000f470 tx=0x7fd880006df0).stop 2020-11-20T12:06:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414d8c0 msgr2=0x7fd88414dca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:15.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd871ffb700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414d8c0 0x7fd88414dca0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.082+0000 7fd8893f0700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd85c056d20 0x7fd85c0591e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fd88413afd0 tx=0x7fd874015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.095+0000 7fd88be55700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414e190 0x7fd88414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.095+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88413ad10 con 0x7fd88414e190 2020-11-20T12:06:15.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.095+0000 7fd889bf1700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414e190 0x7fd88414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.095+0000 7fd889bf1700 1 -- 172.21.15.44:0/4272702772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd884044f60 con 0x7fd88414e190 2020-11-20T12:06:15.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.095+0000 7fd889bf1700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414e190 0x7fd88414a130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fd88000f470 tx=0x7fd88001b000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.095+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880014dc0 con 0x7fd88414e190 2020-11-20T12:06:15.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.096+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4265 (secure 0 0 0) 0x7fd8800133a0 con 0x7fd88414e190 2020-11-20T12:06:15.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.096+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414e190 msgr2=0x7fd88414a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.096+0000 7fd871ffb700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414e190 0x7fd88414a130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fd88000f470 tx=0x7fd88001b000).stop 2020-11-20T12:06:15.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.096+0000 7fd871ffb700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414e190 msgr2=0x7fd88414a130 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:15.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.096+0000 7fd871ffb700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd88414e190 0x7fd88414a130 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0x7fd88000f470 tx=0x7fd88001b000).stop 2020-11-20T12:06:15.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.097+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd85c056d20 msgr2=0x7fd85c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.097+0000 7fd88be55700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd85c056d20 0x7fd85c0591e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fd88413afd0 tx=0x7fd874015000).stop 2020-11-20T12:06:15.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.097+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 msgr2=0x7fd8840a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.097+0000 7fd88be55700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840a5fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd87c00c860 tx=0x7fd87c007ef0).stop 2020-11-20T12:06:15.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.097+0000 7fd88a3f2700 1 -- 172.21.15.44:0/4272702772 reap_dead start 2020-11-20T12:06:15.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.098+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 shutdown_connections 2020-11-20T12:06:15.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.098+0000 7fd88be55700 1 --2- 172.21.15.44:0/4272702772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8840ae7d0 0x7fd8840a5fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.098+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 >> 172.21.15.44:0/4272702772 conn(0x7fd8840b1d10 msgr2=0x7fd88413b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:15.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.098+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 shutdown_connections 2020-11-20T12:06:15.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.098+0000 7fd88be55700 1 -- 172.21.15.44:0/4272702772 wait complete. 2020-11-20T12:06:15.116 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 145 2020-11-20T12:06:15.116 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:15.116 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:15.117 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:06:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 26, 2020-11-20T12:06:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:06:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:15.118 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 4, 2020-11-20T12:06:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:15.120 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:15.121 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:15.122 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:15.123 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:15.124 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:15.125 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.126 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:15.127 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:15.128 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:15.129 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:15.130 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.131 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:15.132 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:15.133 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:15.134 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.135 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:15.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:15.137 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.139 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:15.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.144 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:15.145 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:15.147 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:06:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.150 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:15.150 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:15.151 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:06:15.152 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 146 2020-11-20T12:06:15.155 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 147 2020-11-20T12:06:15.187 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 148 2020-11-20T12:06:15.216 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 149 2020-11-20T12:06:15.247 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 150 2020-11-20T12:06:15.278 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 151 2020-11-20T12:06:15.310 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 152 2020-11-20T12:06:15.338 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 153 2020-11-20T12:06:15.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.336+0000 7f281c674700 1 Processor -- start 2020-11-20T12:06:15.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.336+0000 7f281c674700 1 -- start start 2020-11-20T12:06:15.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281c674700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f281413a730 0x7f281413ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281c674700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f28141345a0 0x7f28140681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281c674700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28141349e0 con 0x7f2814067d80 2020-11-20T12:06:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281c674700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28140686d0 con 0x7f28141345a0 2020-11-20T12:06:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281c674700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28140658a0 con 0x7f281413a730 2020-11-20T12:06:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281ac11700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f28141345a0 0x7f28140681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f2819c0f700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2814067d80 0x7f2814133e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41682/0 (socket says 172.21.15.44:41682) 2020-11-20T12:06:15.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f2819c0f700 1 -- 172.21.15.44:0/3441298082 learned_addr learned my addr 172.21.15.44:0/3441298082 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:06:15.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281a410700 1 --2- 172.21.15.44:0/3441298082 >> v2:172.21.15.79:3300/0 conn(0x7f281413a730 0x7f281413ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281a410700 1 -- 172.21.15.44:0/3441298082 >> v2:172.21.15.44:3300/0 conn(0x7f28141345a0 msgr2=0x7f28140681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:15.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281a410700 1 --2- 172.21.15.44:0/3441298082 >> v2:172.21.15.44:3300/0 conn(0x7f28141345a0 0x7f28140681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281940e700 1 -- 172.21.15.44:0/3441298082 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750980092 0 0) 0x7f28141349e0 con 0x7f2814067d80 2020-11-20T12:06:15.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281a410700 1 -- 172.21.15.44:0/3441298082 >> v1:172.21.15.44:6789/0 conn(0x7f2814067d80 legacy=0x7f2814133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281a410700 1 -- 172.21.15.44:0/3441298082 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281413b390 con 0x7f281413a730 2020-11-20T12:06:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281a410700 1 --2- 172.21.15.44:0/3441298082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281413a730 0x7f281413ab10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f28140ae200 tx=0x7f28040090f0).ready entity=mon.1 client_cookie=b1e1551b39b2701a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.338+0000 7f281940e700 1 -- 172.21.15.44:0/3441298082 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280400be10 con 0x7f281413a730 2020-11-20T12:06:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.339+0000 7f281940e700 1 -- 172.21.15.44:0/3441298082 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28040183f0 con 0x7f281413a730 2020-11-20T12:06:15.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.339+0000 7f281940e700 1 -- 172.21.15.44:0/3441298082 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804018770 con 0x7f281413a730 2020-11-20T12:06:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 -- 172.21.15.44:0/3441298082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281413a730 msgr2=0x7f281413ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 --2- 172.21.15.44:0/3441298082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281413a730 0x7f281413ab10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f28140ae200 tx=0x7f28040090f0).stop 2020-11-20T12:06:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 -- 172.21.15.44:0/3441298082 shutdown_connections 2020-11-20T12:06:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 --2- 172.21.15.44:0/3441298082 >> v2:172.21.15.44:3300/0 conn(0x7f28141345a0 0x7f28140681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 --2- 172.21.15.44:0/3441298082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281413a730 0x7f281413ab10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 -- 172.21.15.44:0/3441298082 >> 172.21.15.44:0/3441298082 conn(0x7f2814062cf0 msgr2=0x7f28140ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:15.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 -- 172.21.15.44:0/3441298082 shutdown_connections 2020-11-20T12:06:15.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.340+0000 7f281c674700 1 -- 172.21.15.44:0/3441298082 wait complete. 2020-11-20T12:06:15.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.342+0000 7f281c674700 1 Processor -- start 2020-11-20T12:06:15.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.342+0000 7f281c674700 1 -- start start 2020-11-20T12:06:15.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281c674700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2814067d80 0x7f28140acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281c674700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28141345a0 0x7f28141353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281c674700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281413a730 0x7f28140a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281c674700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281409db40 con 0x7f28141345a0 2020-11-20T12:06:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281c674700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814134c10 con 0x7f2814067d80 2020-11-20T12:06:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281c674700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281409d8a0 con 0x7f281413a730 2020-11-20T12:06:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28141345a0 0x7f28141353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281a410700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2814067d80 0x7f28140acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28141345a0 0x7f28141353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37942/0 (socket says 172.21.15.44:37942) 2020-11-20T12:06:15.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 -- 172.21.15.44:0/1566637897 learned_addr learned my addr 172.21.15.44:0/1566637897 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:15.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281ac11700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281413a730 0x7f28140a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281413a730 msgr2=0x7f28140a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281413a730 0x7f28140a7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2814067d80 msgr2=0x7f28140acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2814067d80 0x7f28140acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 -- 172.21.15.44:0/1566637897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814134f50 con 0x7f28141345a0 2020-11-20T12:06:15.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281a410700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2814067d80 0x7f28140acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f2819c0f700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28141345a0 0x7f28141353f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f281000c720 tx=0x7f2810007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100072e0 con 0x7f28141345a0 2020-11-20T12:06:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2810007440 con 0x7f28141345a0 2020-11-20T12:06:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810005800 con 0x7f28141345a0 2020-11-20T12:06:15.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.343+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2814136050 con 0x7f28141345a0 2020-11-20T12:06:15.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.344+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28141398b0 con 0x7f28141345a0 2020-11-20T12:06:15.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.345+0000 7f281c674700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28141358e0 0x7f28140a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.345+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2814065fc0 con 0x7f28141358e0 2020-11-20T12:06:15.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.345+0000 7f281a410700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28141358e0 0x7f28140a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.345+0000 7f281a410700 1 -- 172.21.15.44:0/1566637897 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28140661c0 con 0x7f28141358e0 2020-11-20T12:06:15.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.345+0000 7f281a410700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28141358e0 0x7f28140a7500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2804009130 tx=0x7f2804018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.345+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804017cc0 con 0x7f28141358e0 2020-11-20T12:06:15.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.346+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f280401d070 con 0x7f28141358e0 2020-11-20T12:06:15.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.346+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28141358e0 msgr2=0x7f28140a7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.346+0000 7f28027fc700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28141358e0 0x7f28140a7500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2804009130 tx=0x7f2804018000).stop 2020-11-20T12:06:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.346+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28141358e0 msgr2=0x7f28140a7500 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.346+0000 7f28027fc700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28141358e0 0x7f28140a7500 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0x7f2804009130 tx=0x7f2804018000).stop 2020-11-20T12:06:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.346+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2810010070 con 0x7f28141345a0 2020-11-20T12:06:15.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.347+0000 7f28027fc700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27ec057410 0x7f27ec0598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.347+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f281006f790 con 0x7f28141345a0 2020-11-20T12:06:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.349+0000 7f281a410700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27ec057410 0x7f27ec0598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.349+0000 7f281a410700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27ec057410 0x7f27ec0598d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f2814066150 tx=0x7f2804023000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.361+0000 7f281c674700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281409d470 0x7f28141490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.361+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28141350e0 con 0x7f281409d470 2020-11-20T12:06:15.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.361+0000 7f281ac11700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281409d470 0x7f28141490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.361+0000 7f281ac11700 1 -- 172.21.15.44:0/1566637897 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2814063100 con 0x7f281409d470 2020-11-20T12:06:15.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.361+0000 7f281ac11700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281409d470 0x7f28141490a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f280c011ef0 tx=0x7f280c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c00fe90 con 0x7f281409d470 2020-11-20T12:06:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4605 (secure 0 0 0) 0x7f280c014440 con 0x7f281409d470 2020-11-20T12:06:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281409d470 msgr2=0x7f28141490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f28027fc700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281409d470 0x7f28141490a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f280c011ef0 tx=0x7f280c0091d0).stop 2020-11-20T12:06:15.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f28027fc700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281409d470 msgr2=0x7f28141490a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f28027fc700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281409d470 0x7f28141490a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27ec057410 msgr2=0x7f27ec0598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f281c674700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27ec057410 0x7f27ec0598d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f2814066150 tx=0x7f2804023000).stop 2020-11-20T12:06:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28141345a0 msgr2=0x7f28141353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f281c674700 1 --2- 172.21.15.44:0/1566637897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28141345a0 0x7f28141353f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f281000c720 tx=0x7f2810007ac0).stop 2020-11-20T12:06:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.362+0000 7f281ac11700 1 -- 172.21.15.44:0/1566637897 reap_dead start 2020-11-20T12:06:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.363+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 shutdown_connections 2020-11-20T12:06:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.363+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 >> 172.21.15.44:0/1566637897 conn(0x7f2814062cf0 msgr2=0x7f28140a9490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:15.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.363+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 shutdown_connections 2020-11-20T12:06:15.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.364+0000 7f281c674700 1 -- 172.21.15.44:0/1566637897 wait complete. 2020-11-20T12:06:15.371 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:06:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 26, 2020-11-20T12:06:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:06:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 4, 2020-11-20T12:06:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:06:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:06:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:06:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:06:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:15.406 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:15.407 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 154 2020-11-20T12:06:15.408 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 155 2020-11-20T12:06:15.408 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:06:15.408 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:06:15.425 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 156 2020-11-20T12:06:15.454 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 157 2020-11-20T12:06:15.485 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 158 2020-11-20T12:06:15.516 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 159 2020-11-20T12:06:15.546 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 160 2020-11-20T12:06:15.574 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 161 2020-11-20T12:06:15.608 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 162 2020-11-20T12:06:15.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.612+0000 7f8a35d0c700 1 Processor -- start 2020-11-20T12:06:15.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.612+0000 7f8a35d0c700 1 -- start start 2020-11-20T12:06:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.613+0000 7f8a35d0c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a3013c0d0 0x7f8a300a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.613+0000 7f8a35d0c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a300a8cb0 con 0x7f8a300a72d0 2020-11-20T12:06:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.613+0000 7f8a35d0c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a30067fe0 con 0x7f8a300abc70 2020-11-20T12:06:15.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.613+0000 7f8a35d0c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300ac550 con 0x7f8a3013c0d0 2020-11-20T12:06:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.613+0000 7f8a2effd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8a300a72d0 0x7f8a300678d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41700/0 (socket says 172.21.15.44:41700) 2020-11-20T12:06:15.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.613+0000 7f8a2effd700 1 -- 172.21.15.44:0/1365609250 learned_addr learned my addr 172.21.15.44:0/1365609250 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:06:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.613+0000 7f8a2ffff700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a3013c0d0 0x7f8a300a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266012975 0 0) 0x7f8a30067fe0 con 0x7f8a300abc70 2020-11-20T12:06:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a1c003240 con 0x7f8a300abc70 2020-11-20T12:06:15.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487109400 0 0) 0x7f8a300a8cb0 con 0x7f8a300a72d0 2020-11-20T12:06:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a30067fe0 con 0x7f8a300a72d0 2020-11-20T12:06:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3837968841 0 0) 0x7f8a1c003240 con 0x7f8a300abc70 2020-11-20T12:06:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a3013c0d0 msgr2=0x7f8a300a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a3013c0d0 0x7f8a300a87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 >> v1:172.21.15.44:6789/0 conn(0x7f8a300a72d0 legacy=0x7f8a300678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3013c650 con 0x7f8a300abc70 2020-11-20T12:06:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f8a24001520 con 0x7f8a300abc70 2020-11-20T12:06:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 >> v1:172.21.15.79:6789/0 conn(0x7f8a300abc70 legacy=0x7f8a300671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a1c004920 0x7f8a1c004d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a1c003480 0x7f8a1c003900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a1c003df0 0x7f8a1c009720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0042d0 con 0x7f8a1c003df0 2020-11-20T12:06:15.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c009c10 con 0x7f8a1c003480 2020-11-20T12:06:15.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c009d80 con 0x7f8a1c004920 2020-11-20T12:06:15.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2effd700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a1c003480 0x7f8a1c003900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2ffff700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a1c003df0 0x7f8a1c009720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.614+0000 7f8a2f7fe700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a1c004920 0x7f8a1c004d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.615+0000 7f8a2f7fe700 1 -- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a1c003480 msgr2=0x7f8a1c003900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:15.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.615+0000 7f8a2f7fe700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a1c003480 0x7f8a1c003900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.615+0000 7f8a2f7fe700 1 -- 172.21.15.44:0/1365609250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a1c003df0 msgr2=0x7f8a1c009720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.615+0000 7f8a2f7fe700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a1c003df0 0x7f8a1c009720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.615+0000 7f8a2f7fe700 1 -- 172.21.15.44:0/1365609250 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c003240 con 0x7f8a1c004920 2020-11-20T12:06:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.615+0000 7f8a2ffff700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a1c003df0 0x7f8a1c009720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:15.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.616+0000 7f8a2f7fe700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a1c004920 0x7f8a1c004d20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8a2400c0e0 tx=0x7f8a24007640).ready entity=mon.2 client_cookie=31a4c29329322db6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.616+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2400cd30 con 0x7f8a1c004920 2020-11-20T12:06:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.616+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a24012400 con 0x7f8a1c004920 2020-11-20T12:06:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.616+0000 7f8a2e7fc700 1 -- 172.21.15.44:0/1365609250 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240127c0 con 0x7f8a1c004920 2020-11-20T12:06:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.617+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a1c004920 msgr2=0x7f8a1c004d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.617+0000 7f8a35d0c700 1 --2- 172.21.15.44:0/1365609250 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a1c004920 0x7f8a1c004d20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8a2400c0e0 tx=0x7f8a24007640).stop 2020-11-20T12:06:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.617+0000 7f8a2ffff700 1 -- 172.21.15.44:0/1365609250 reap_dead start 2020-11-20T12:06:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.617+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1365609250 shutdown_connections 2020-11-20T12:06:15.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.617+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1365609250 >> 172.21.15.44:0/1365609250 conn(0x7f8a30060170 msgr2=0x7f8a30060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.617+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1365609250 shutdown_connections 2020-11-20T12:06:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.618+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1365609250 wait complete. 2020-11-20T12:06:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.619+0000 7f8a35d0c700 1 Processor -- start 2020-11-20T12:06:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.620+0000 7f8a35d0c700 1 -- start start 2020-11-20T12:06:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.620+0000 7f8a35d0c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a3009de30 0x7f8a3009f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a35d0c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a300a5d10 0x7f8a30136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a35d0c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a301372b0 0x7f8a301496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a35d0c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3009ecc0 con 0x7f8a3009de30 2020-11-20T12:06:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a35d0c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3009d980 con 0x7f8a301372b0 2020-11-20T12:06:15.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a35d0c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3009ffa0 con 0x7f8a300a5d10 2020-11-20T12:06:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a3009de30 0x7f8a3009f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a301372b0 0x7f8a301496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2effd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a300a5d10 0x7f8a30136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a301372b0 0x7f8a301496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58250/0 (socket says 172.21.15.44:58250) 2020-11-20T12:06:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2ffff700 1 -- 172.21.15.44:0/1729015854 learned_addr learned my addr 172.21.15.44:0/1729015854 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2effd700 1 -- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a301372b0 msgr2=0x7f8a301496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2effd700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a301372b0 0x7f8a301496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2effd700 1 -- 172.21.15.44:0/1729015854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a3009de30 msgr2=0x7f8a3009f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:15.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2effd700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a3009de30 0x7f8a3009f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2effd700 1 -- 172.21.15.44:0/1729015854 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3009fd00 con 0x7f8a300a5d10 2020-11-20T12:06:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2ffff700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a301372b0 0x7f8a301496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a2effd700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a300a5d10 0x7f8a30136dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8a20001950 tx=0x7f8a20007c80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a0f7fe700 1 -- 172.21.15.44:0/1729015854 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20001d20 con 0x7f8a300a5d10 2020-11-20T12:06:15.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a0f7fe700 1 -- 172.21.15.44:0/1729015854 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2000f8a0 con 0x7f8a300a5d10 2020-11-20T12:06:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a300a5870 con 0x7f8a300a5d10 2020-11-20T12:06:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.621+0000 7f8a0f7fe700 1 -- 172.21.15.44:0/1729015854 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000fc40 con 0x7f8a300a5d10 2020-11-20T12:06:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.622+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a300a4ba0 con 0x7f8a300a5d10 2020-11-20T12:06:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.623+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a300a8520 con 0x7f8a300a5d10 2020-11-20T12:06:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.623+0000 7f8a0f7fe700 1 -- 172.21.15.44:0/1729015854 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8a20015410 con 0x7f8a300a5d10 2020-11-20T12:06:15.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.623+0000 7f8a0f7fe700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8a08056d20 0x7f8a080591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.624+0000 7f8a0f7fe700 1 -- 172.21.15.44:0/1729015854 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f8a20070540 con 0x7f8a300a5d10 2020-11-20T12:06:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.624+0000 7f8a2f7fe700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8a08056d20 0x7f8a080591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:15.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.624+0000 7f8a2f7fe700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8a08056d20 0x7f8a080591e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8a300ac520 tx=0x7f8a24017000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:15.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:15.628+0000 7f8a0f7fe700 1 -- 172.21.15.44:0/1729015854 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8a20045680 con 0x7f8a300a5d10 2020-11-20T12:06:15.638 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 163 2020-11-20T12:06:15.665 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 164 2020-11-20T12:06:15.701 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 165 2020-11-20T12:06:15.735 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 166 2020-11-20T12:06:15.764 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 167 2020-11-20T12:06:15.789 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 168 2020-11-20T12:06:15.820 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 169 2020-11-20T12:06:15.850 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 170 2020-11-20T12:06:15.879 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 171 2020-11-20T12:06:15.908 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 172 2020-11-20T12:06:15.940 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 173 2020-11-20T12:06:15.967 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 174 2020-11-20T12:06:16.000 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 175 2020-11-20T12:06:16.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.026+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f8a3009dac0 con 0x7f8a300a5d10 2020-11-20T12:06:16.030 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 176 2020-11-20T12:06:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.028+0000 7f8a0f7fe700 1 -- 172.21.15.44:0/1729015854 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f8a20048ca0 con 0x7f8a300a5d10 2020-11-20T12:06:16.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8a08056d20 msgr2=0x7f8a080591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:16.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8a08056d20 0x7f8a080591e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8a300ac520 tx=0x7f8a24017000).stop 2020-11-20T12:06:16.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a300a5d10 msgr2=0x7f8a30136dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:16.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a300a5d10 0x7f8a30136dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8a20001950 tx=0x7f8a20007c80).stop 2020-11-20T12:06:16.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 shutdown_connections 2020-11-20T12:06:16.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8a08056d20 0x7f8a080591e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a301372b0 0x7f8a301496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a3009de30 0x7f8a3009f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 --2- 172.21.15.44:0/1729015854 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a300a5d10 0x7f8a30136dc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:16.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 >> 172.21.15.44:0/1729015854 conn(0x7f8a30060170 msgr2=0x7f8a300a7730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 shutdown_connections 2020-11-20T12:06:16.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:16.030+0000 7f8a35d0c700 1 -- 172.21.15.44:0/1729015854 wait complete. 2020-11-20T12:06:16.051 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:06:16.061 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 177 2020-11-20T12:06:16.088 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 178 2020-11-20T12:06:16.116 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 179 2020-11-20T12:06:16.144 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 180 2020-11-20T12:06:16.171 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 181 2020-11-20T12:06:16.195 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 182 2020-11-20T12:06:16.508 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 183 2020-11-20T12:06:16.536 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 184 2020-11-20T12:06:16.565 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 185 2020-11-20T12:06:16.594 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 186 2020-11-20T12:06:16.620 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 187 2020-11-20T12:06:16.649 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 188 2020-11-20T12:06:16.677 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 189 2020-11-20T12:06:16.715 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 190 2020-11-20T12:06:16.743 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 191 2020-11-20T12:06:16.773 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 192 2020-11-20T12:06:16.803 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 193 2020-11-20T12:06:16.829 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 194 2020-11-20T12:06:16.854 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 195 2020-11-20T12:06:16.878 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 196 2020-11-20T12:06:16.905 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 197 2020-11-20T12:06:16.934 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 198 2020-11-20T12:06:16.959 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 199 2020-11-20T12:06:16.986 INFO:tasks.workunit.client.0.smithi044.stdout:Take first snapshot .snap/test1 2020-11-20T12:06:17.144 INFO:tasks.workunit.client.0.smithi044.stdout:Create dir 200 to 299 ... 2020-11-20T12:06:17.145 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 200 2020-11-20T12:06:17.598 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 201 2020-11-20T12:06:17.623 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 202 2020-11-20T12:06:17.649 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 203 2020-11-20T12:06:17.677 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 204 2020-11-20T12:06:17.703 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 205 2020-11-20T12:06:17.735 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 206 2020-11-20T12:06:17.760 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 207 2020-11-20T12:06:17.786 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 208 2020-11-20T12:06:17.807 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 209 2020-11-20T12:06:17.833 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 210 2020-11-20T12:06:17.859 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 211 2020-11-20T12:06:17.883 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 212 2020-11-20T12:06:17.909 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 213 2020-11-20T12:06:17.934 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 214 2020-11-20T12:06:17.960 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 215 2020-11-20T12:06:17.984 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 216 2020-11-20T12:06:18.011 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 217 2020-11-20T12:06:18.037 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 218 2020-11-20T12:06:18.063 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 219 2020-11-20T12:06:18.088 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 220 2020-11-20T12:06:18.115 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 221 2020-11-20T12:06:18.141 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 222 2020-11-20T12:06:18.167 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 223 2020-11-20T12:06:18.191 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 224 2020-11-20T12:06:18.217 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 225 2020-11-20T12:06:18.245 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 226 2020-11-20T12:06:18.270 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 227 2020-11-20T12:06:18.297 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 228 2020-11-20T12:06:18.325 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 229 2020-11-20T12:06:18.349 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 230 2020-11-20T12:06:18.375 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 231 2020-11-20T12:06:18.400 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 232 2020-11-20T12:06:18.426 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 233 2020-11-20T12:06:18.452 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 234 2020-11-20T12:06:18.477 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 235 2020-11-20T12:06:18.505 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 236 2020-11-20T12:06:18.531 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 237 2020-11-20T12:06:18.557 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 238 2020-11-20T12:06:18.582 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 239 2020-11-20T12:06:18.606 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 240 2020-11-20T12:06:18.631 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 241 2020-11-20T12:06:18.654 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 242 2020-11-20T12:06:18.679 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 243 2020-11-20T12:06:18.706 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 244 2020-11-20T12:06:18.740 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 245 2020-11-20T12:06:18.782 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 246 2020-11-20T12:06:18.832 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 247 2020-11-20T12:06:18.883 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 248 2020-11-20T12:06:18.936 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 249 2020-11-20T12:06:18.982 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 250 2020-11-20T12:06:19.030 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 251 2020-11-20T12:06:19.167 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 252 2020-11-20T12:06:19.194 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 253 2020-11-20T12:06:19.220 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 254 2020-11-20T12:06:19.243 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 255 2020-11-20T12:06:19.268 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 256 2020-11-20T12:06:19.291 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 257 2020-11-20T12:06:19.317 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 258 2020-11-20T12:06:19.348 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 259 2020-11-20T12:06:19.367 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 260 2020-11-20T12:06:19.392 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 261 2020-11-20T12:06:19.421 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 262 2020-11-20T12:06:19.445 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 263 2020-11-20T12:06:19.470 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 264 2020-11-20T12:06:19.496 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 265 2020-11-20T12:06:19.521 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 266 2020-11-20T12:06:19.548 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 267 2020-11-20T12:06:19.574 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 268 2020-11-20T12:06:19.600 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 269 2020-11-20T12:06:19.627 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 270 2020-11-20T12:06:19.654 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 271 2020-11-20T12:06:19.678 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 272 2020-11-20T12:06:19.706 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 273 2020-11-20T12:06:19.896 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 274 2020-11-20T12:06:19.923 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 275 2020-11-20T12:06:19.949 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 276 2020-11-20T12:06:19.977 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 277 2020-11-20T12:06:20.000 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 278 2020-11-20T12:06:20.026 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 279 2020-11-20T12:06:20.052 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 280 2020-11-20T12:06:20.079 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 281 2020-11-20T12:06:20.101 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 282 2020-11-20T12:06:20.127 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 283 2020-11-20T12:06:20.154 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 284 2020-11-20T12:06:20.179 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 285 2020-11-20T12:06:20.205 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 286 2020-11-20T12:06:20.227 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 287 2020-11-20T12:06:20.250 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 288 2020-11-20T12:06:20.278 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 289 2020-11-20T12:06:20.307 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 290 2020-11-20T12:06:20.348 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 291 2020-11-20T12:06:20.389 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 292 2020-11-20T12:06:20.430 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 293 2020-11-20T12:06:20.536 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 294 2020-11-20T12:06:20.583 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 295 2020-11-20T12:06:20.626 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 296 2020-11-20T12:06:20.657 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 297 2020-11-20T12:06:20.686 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 298 2020-11-20T12:06:20.713 INFO:tasks.workunit.client.0.smithi044.stdout: create dir 299 2020-11-20T12:06:20.738 INFO:tasks.workunit.client.0.smithi044.stdout:Create a snapshot in every first level dir ... 2020-11-20T12:06:20.985 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/.snap/snap-subdir-test 2020-11-20T12:06:21.092 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/30 file after the snapshot 2020-11-20T12:06:21.093 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/31 file after the snapshot 2020-11-20T12:06:21.095 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/32 file after the snapshot 2020-11-20T12:06:21.097 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/33 file after the snapshot 2020-11-20T12:06:21.099 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/34 file after the snapshot 2020-11-20T12:06:21.103 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/35 file after the snapshot 2020-11-20T12:06:21.104 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/36 file after the snapshot 2020-11-20T12:06:21.106 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/37 file after the snapshot 2020-11-20T12:06:21.108 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/38 file after the snapshot 2020-11-20T12:06:21.110 INFO:tasks.workunit.client.0.smithi044.stdout: create 100/39 file after the snapshot 2020-11-20T12:06:21.113 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/.snap/snap-subdir-test 2020-11-20T12:06:21.150 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/30 file after the snapshot 2020-11-20T12:06:21.152 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/31 file after the snapshot 2020-11-20T12:06:21.154 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/32 file after the snapshot 2020-11-20T12:06:21.157 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/33 file after the snapshot 2020-11-20T12:06:21.159 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/34 file after the snapshot 2020-11-20T12:06:21.161 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/35 file after the snapshot 2020-11-20T12:06:21.163 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/36 file after the snapshot 2020-11-20T12:06:21.166 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/37 file after the snapshot 2020-11-20T12:06:21.169 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/38 file after the snapshot 2020-11-20T12:06:21.172 INFO:tasks.workunit.client.0.smithi044.stdout: create 101/39 file after the snapshot 2020-11-20T12:06:21.175 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/.snap/snap-subdir-test 2020-11-20T12:06:21.207 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/30 file after the snapshot 2020-11-20T12:06:21.208 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/31 file after the snapshot 2020-11-20T12:06:21.208 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/32 file after the snapshot 2020-11-20T12:06:21.209 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/33 file after the snapshot 2020-11-20T12:06:21.211 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/34 file after the snapshot 2020-11-20T12:06:21.213 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/35 file after the snapshot 2020-11-20T12:06:21.215 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/36 file after the snapshot 2020-11-20T12:06:21.217 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/37 file after the snapshot 2020-11-20T12:06:21.219 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/38 file after the snapshot 2020-11-20T12:06:21.221 INFO:tasks.workunit.client.0.smithi044.stdout: create 102/39 file after the snapshot 2020-11-20T12:06:21.223 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/.snap/snap-subdir-test 2020-11-20T12:06:21.249 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/30 file after the snapshot 2020-11-20T12:06:21.251 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/31 file after the snapshot 2020-11-20T12:06:21.252 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/32 file after the snapshot 2020-11-20T12:06:21.255 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/33 file after the snapshot 2020-11-20T12:06:21.257 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/34 file after the snapshot 2020-11-20T12:06:21.260 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/35 file after the snapshot 2020-11-20T12:06:21.264 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/36 file after the snapshot 2020-11-20T12:06:21.266 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/37 file after the snapshot 2020-11-20T12:06:21.269 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/38 file after the snapshot 2020-11-20T12:06:21.272 INFO:tasks.workunit.client.0.smithi044.stdout: create 103/39 file after the snapshot 2020-11-20T12:06:21.274 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/.snap/snap-subdir-test 2020-11-20T12:06:21.301 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/30 file after the snapshot 2020-11-20T12:06:21.303 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/31 file after the snapshot 2020-11-20T12:06:21.305 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/32 file after the snapshot 2020-11-20T12:06:21.307 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/33 file after the snapshot 2020-11-20T12:06:21.310 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/34 file after the snapshot 2020-11-20T12:06:21.312 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/35 file after the snapshot 2020-11-20T12:06:21.314 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/36 file after the snapshot 2020-11-20T12:06:21.316 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/37 file after the snapshot 2020-11-20T12:06:21.319 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/38 file after the snapshot 2020-11-20T12:06:21.322 INFO:tasks.workunit.client.0.smithi044.stdout: create 104/39 file after the snapshot 2020-11-20T12:06:21.324 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/.snap/snap-subdir-test 2020-11-20T12:06:21.349 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/30 file after the snapshot 2020-11-20T12:06:21.351 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/31 file after the snapshot 2020-11-20T12:06:21.353 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/32 file after the snapshot 2020-11-20T12:06:21.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/33 file after the snapshot 2020-11-20T12:06:21.359 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/34 file after the snapshot 2020-11-20T12:06:21.362 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/35 file after the snapshot 2020-11-20T12:06:21.366 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/36 file after the snapshot 2020-11-20T12:06:21.368 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/37 file after the snapshot 2020-11-20T12:06:21.370 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/38 file after the snapshot 2020-11-20T12:06:21.373 INFO:tasks.workunit.client.0.smithi044.stdout: create 105/39 file after the snapshot 2020-11-20T12:06:21.376 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/.snap/snap-subdir-test 2020-11-20T12:06:21.402 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/30 file after the snapshot 2020-11-20T12:06:21.403 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/31 file after the snapshot 2020-11-20T12:06:21.404 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/32 file after the snapshot 2020-11-20T12:06:21.407 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/33 file after the snapshot 2020-11-20T12:06:21.409 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/34 file after the snapshot 2020-11-20T12:06:21.412 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/35 file after the snapshot 2020-11-20T12:06:21.415 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/36 file after the snapshot 2020-11-20T12:06:21.418 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/37 file after the snapshot 2020-11-20T12:06:21.419 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/38 file after the snapshot 2020-11-20T12:06:21.422 INFO:tasks.workunit.client.0.smithi044.stdout: create 106/39 file after the snapshot 2020-11-20T12:06:21.424 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/.snap/snap-subdir-test 2020-11-20T12:06:21.450 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/30 file after the snapshot 2020-11-20T12:06:21.452 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/31 file after the snapshot 2020-11-20T12:06:21.454 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/32 file after the snapshot 2020-11-20T12:06:21.457 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/33 file after the snapshot 2020-11-20T12:06:21.459 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/34 file after the snapshot 2020-11-20T12:06:21.461 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/35 file after the snapshot 2020-11-20T12:06:21.464 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/36 file after the snapshot 2020-11-20T12:06:21.466 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/37 file after the snapshot 2020-11-20T12:06:21.468 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/38 file after the snapshot 2020-11-20T12:06:21.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 107/39 file after the snapshot 2020-11-20T12:06:21.473 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/.snap/snap-subdir-test 2020-11-20T12:06:21.499 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/30 file after the snapshot 2020-11-20T12:06:21.501 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/31 file after the snapshot 2020-11-20T12:06:21.503 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/32 file after the snapshot 2020-11-20T12:06:21.505 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/33 file after the snapshot 2020-11-20T12:06:21.508 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/34 file after the snapshot 2020-11-20T12:06:21.511 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/35 file after the snapshot 2020-11-20T12:06:21.513 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/36 file after the snapshot 2020-11-20T12:06:21.515 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/37 file after the snapshot 2020-11-20T12:06:21.518 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/38 file after the snapshot 2020-11-20T12:06:21.520 INFO:tasks.workunit.client.0.smithi044.stdout: create 108/39 file after the snapshot 2020-11-20T12:06:21.523 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/.snap/snap-subdir-test 2020-11-20T12:06:21.549 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/30 file after the snapshot 2020-11-20T12:06:21.553 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/31 file after the snapshot 2020-11-20T12:06:21.554 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/32 file after the snapshot 2020-11-20T12:06:21.556 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/33 file after the snapshot 2020-11-20T12:06:21.558 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/34 file after the snapshot 2020-11-20T12:06:21.560 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/35 file after the snapshot 2020-11-20T12:06:21.562 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/36 file after the snapshot 2020-11-20T12:06:21.564 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/37 file after the snapshot 2020-11-20T12:06:21.566 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/38 file after the snapshot 2020-11-20T12:06:21.568 INFO:tasks.workunit.client.0.smithi044.stdout: create 109/39 file after the snapshot 2020-11-20T12:06:21.571 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/.snap/snap-subdir-test 2020-11-20T12:06:21.606 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/30 file after the snapshot 2020-11-20T12:06:21.607 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/31 file after the snapshot 2020-11-20T12:06:21.609 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/32 file after the snapshot 2020-11-20T12:06:21.612 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/33 file after the snapshot 2020-11-20T12:06:21.614 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/34 file after the snapshot 2020-11-20T12:06:21.616 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/35 file after the snapshot 2020-11-20T12:06:21.618 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/36 file after the snapshot 2020-11-20T12:06:21.621 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/37 file after the snapshot 2020-11-20T12:06:21.623 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/38 file after the snapshot 2020-11-20T12:06:21.626 INFO:tasks.workunit.client.0.smithi044.stdout: create 110/39 file after the snapshot 2020-11-20T12:06:21.628 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/.snap/snap-subdir-test 2020-11-20T12:06:21.660 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/30 file after the snapshot 2020-11-20T12:06:21.662 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/31 file after the snapshot 2020-11-20T12:06:21.663 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/32 file after the snapshot 2020-11-20T12:06:21.665 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/33 file after the snapshot 2020-11-20T12:06:21.668 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/34 file after the snapshot 2020-11-20T12:06:21.670 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/35 file after the snapshot 2020-11-20T12:06:21.673 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/36 file after the snapshot 2020-11-20T12:06:21.677 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/37 file after the snapshot 2020-11-20T12:06:21.680 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/38 file after the snapshot 2020-11-20T12:06:21.683 INFO:tasks.workunit.client.0.smithi044.stdout: create 111/39 file after the snapshot 2020-11-20T12:06:21.686 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/.snap/snap-subdir-test 2020-11-20T12:06:21.712 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/30 file after the snapshot 2020-11-20T12:06:21.713 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/31 file after the snapshot 2020-11-20T12:06:21.715 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/32 file after the snapshot 2020-11-20T12:06:21.718 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/33 file after the snapshot 2020-11-20T12:06:21.720 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/34 file after the snapshot 2020-11-20T12:06:21.723 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/35 file after the snapshot 2020-11-20T12:06:21.724 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/36 file after the snapshot 2020-11-20T12:06:21.726 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/37 file after the snapshot 2020-11-20T12:06:21.727 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/38 file after the snapshot 2020-11-20T12:06:21.730 INFO:tasks.workunit.client.0.smithi044.stdout: create 112/39 file after the snapshot 2020-11-20T12:06:21.732 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/.snap/snap-subdir-test 2020-11-20T12:06:21.759 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/30 file after the snapshot 2020-11-20T12:06:21.760 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/31 file after the snapshot 2020-11-20T12:06:21.762 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/32 file after the snapshot 2020-11-20T12:06:21.765 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/33 file after the snapshot 2020-11-20T12:06:21.767 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/34 file after the snapshot 2020-11-20T12:06:21.769 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/35 file after the snapshot 2020-11-20T12:06:21.796 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/36 file after the snapshot 2020-11-20T12:06:21.904 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/37 file after the snapshot 2020-11-20T12:06:21.906 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/38 file after the snapshot 2020-11-20T12:06:21.909 INFO:tasks.workunit.client.0.smithi044.stdout: create 113/39 file after the snapshot 2020-11-20T12:06:21.911 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/.snap/snap-subdir-test 2020-11-20T12:06:22.006 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/30 file after the snapshot 2020-11-20T12:06:22.008 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/31 file after the snapshot 2020-11-20T12:06:22.010 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/32 file after the snapshot 2020-11-20T12:06:22.012 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/33 file after the snapshot 2020-11-20T12:06:22.014 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/34 file after the snapshot 2020-11-20T12:06:22.017 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/35 file after the snapshot 2020-11-20T12:06:22.018 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/36 file after the snapshot 2020-11-20T12:06:22.020 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/37 file after the snapshot 2020-11-20T12:06:22.024 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/38 file after the snapshot 2020-11-20T12:06:22.026 INFO:tasks.workunit.client.0.smithi044.stdout: create 114/39 file after the snapshot 2020-11-20T12:06:22.029 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/.snap/snap-subdir-test 2020-11-20T12:06:22.055 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/30 file after the snapshot 2020-11-20T12:06:22.057 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/31 file after the snapshot 2020-11-20T12:06:22.060 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/32 file after the snapshot 2020-11-20T12:06:22.062 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/33 file after the snapshot 2020-11-20T12:06:22.065 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/34 file after the snapshot 2020-11-20T12:06:22.067 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/35 file after the snapshot 2020-11-20T12:06:22.069 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/36 file after the snapshot 2020-11-20T12:06:22.074 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/37 file after the snapshot 2020-11-20T12:06:22.076 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/38 file after the snapshot 2020-11-20T12:06:22.078 INFO:tasks.workunit.client.0.smithi044.stdout: create 115/39 file after the snapshot 2020-11-20T12:06:22.081 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/.snap/snap-subdir-test 2020-11-20T12:06:22.105 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/30 file after the snapshot 2020-11-20T12:06:22.107 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/31 file after the snapshot 2020-11-20T12:06:22.109 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/32 file after the snapshot 2020-11-20T12:06:22.112 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/33 file after the snapshot 2020-11-20T12:06:22.114 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/34 file after the snapshot 2020-11-20T12:06:22.117 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/35 file after the snapshot 2020-11-20T12:06:22.120 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/36 file after the snapshot 2020-11-20T12:06:22.123 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/37 file after the snapshot 2020-11-20T12:06:22.125 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/38 file after the snapshot 2020-11-20T12:06:22.127 INFO:tasks.workunit.client.0.smithi044.stdout: create 116/39 file after the snapshot 2020-11-20T12:06:22.131 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/.snap/snap-subdir-test 2020-11-20T12:06:22.156 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/30 file after the snapshot 2020-11-20T12:06:22.158 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/31 file after the snapshot 2020-11-20T12:06:22.160 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/32 file after the snapshot 2020-11-20T12:06:22.163 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/33 file after the snapshot 2020-11-20T12:06:22.165 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/34 file after the snapshot 2020-11-20T12:06:22.167 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/35 file after the snapshot 2020-11-20T12:06:22.169 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/36 file after the snapshot 2020-11-20T12:06:22.172 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/37 file after the snapshot 2020-11-20T12:06:22.175 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/38 file after the snapshot 2020-11-20T12:06:22.177 INFO:tasks.workunit.client.0.smithi044.stdout: create 117/39 file after the snapshot 2020-11-20T12:06:22.179 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/.snap/snap-subdir-test 2020-11-20T12:06:22.205 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/30 file after the snapshot 2020-11-20T12:06:22.206 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/31 file after the snapshot 2020-11-20T12:06:22.208 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/32 file after the snapshot 2020-11-20T12:06:22.211 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/33 file after the snapshot 2020-11-20T12:06:22.213 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/34 file after the snapshot 2020-11-20T12:06:22.215 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/35 file after the snapshot 2020-11-20T12:06:22.218 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/36 file after the snapshot 2020-11-20T12:06:22.220 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/37 file after the snapshot 2020-11-20T12:06:22.222 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/38 file after the snapshot 2020-11-20T12:06:22.225 INFO:tasks.workunit.client.0.smithi044.stdout: create 118/39 file after the snapshot 2020-11-20T12:06:22.227 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/.snap/snap-subdir-test 2020-11-20T12:06:22.251 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/30 file after the snapshot 2020-11-20T12:06:22.253 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/31 file after the snapshot 2020-11-20T12:06:22.255 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/32 file after the snapshot 2020-11-20T12:06:22.257 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/33 file after the snapshot 2020-11-20T12:06:22.261 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/34 file after the snapshot 2020-11-20T12:06:22.263 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/35 file after the snapshot 2020-11-20T12:06:22.267 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/36 file after the snapshot 2020-11-20T12:06:22.269 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/37 file after the snapshot 2020-11-20T12:06:22.271 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/38 file after the snapshot 2020-11-20T12:06:22.273 INFO:tasks.workunit.client.0.smithi044.stdout: create 119/39 file after the snapshot 2020-11-20T12:06:22.276 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/.snap/snap-subdir-test 2020-11-20T12:06:22.300 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/30 file after the snapshot 2020-11-20T12:06:22.302 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/31 file after the snapshot 2020-11-20T12:06:22.304 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/32 file after the snapshot 2020-11-20T12:06:22.306 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/33 file after the snapshot 2020-11-20T12:06:22.309 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/34 file after the snapshot 2020-11-20T12:06:22.312 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/35 file after the snapshot 2020-11-20T12:06:22.315 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/36 file after the snapshot 2020-11-20T12:06:22.317 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/37 file after the snapshot 2020-11-20T12:06:22.320 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/38 file after the snapshot 2020-11-20T12:06:22.323 INFO:tasks.workunit.client.0.smithi044.stdout: create 120/39 file after the snapshot 2020-11-20T12:06:22.324 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/.snap/snap-subdir-test 2020-11-20T12:06:22.350 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/30 file after the snapshot 2020-11-20T12:06:22.351 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/31 file after the snapshot 2020-11-20T12:06:22.353 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/32 file after the snapshot 2020-11-20T12:06:22.356 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/33 file after the snapshot 2020-11-20T12:06:22.358 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/34 file after the snapshot 2020-11-20T12:06:22.360 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/35 file after the snapshot 2020-11-20T12:06:22.363 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/36 file after the snapshot 2020-11-20T12:06:22.365 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/37 file after the snapshot 2020-11-20T12:06:22.367 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/38 file after the snapshot 2020-11-20T12:06:22.369 INFO:tasks.workunit.client.0.smithi044.stdout: create 121/39 file after the snapshot 2020-11-20T12:06:22.371 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/.snap/snap-subdir-test 2020-11-20T12:06:22.397 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/30 file after the snapshot 2020-11-20T12:06:22.399 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/31 file after the snapshot 2020-11-20T12:06:22.401 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/32 file after the snapshot 2020-11-20T12:06:22.403 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/33 file after the snapshot 2020-11-20T12:06:22.405 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/34 file after the snapshot 2020-11-20T12:06:22.408 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/35 file after the snapshot 2020-11-20T12:06:22.410 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/36 file after the snapshot 2020-11-20T12:06:22.412 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/37 file after the snapshot 2020-11-20T12:06:22.414 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/38 file after the snapshot 2020-11-20T12:06:22.417 INFO:tasks.workunit.client.0.smithi044.stdout: create 122/39 file after the snapshot 2020-11-20T12:06:22.419 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/.snap/snap-subdir-test 2020-11-20T12:06:22.453 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/30 file after the snapshot 2020-11-20T12:06:22.454 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/31 file after the snapshot 2020-11-20T12:06:22.456 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/32 file after the snapshot 2020-11-20T12:06:22.458 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/33 file after the snapshot 2020-11-20T12:06:22.461 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/34 file after the snapshot 2020-11-20T12:06:22.463 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/35 file after the snapshot 2020-11-20T12:06:22.466 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/36 file after the snapshot 2020-11-20T12:06:22.468 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/37 file after the snapshot 2020-11-20T12:06:22.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/38 file after the snapshot 2020-11-20T12:06:22.473 INFO:tasks.workunit.client.0.smithi044.stdout: create 123/39 file after the snapshot 2020-11-20T12:06:22.477 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/.snap/snap-subdir-test 2020-11-20T12:06:22.505 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/30 file after the snapshot 2020-11-20T12:06:22.507 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/31 file after the snapshot 2020-11-20T12:06:22.509 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/32 file after the snapshot 2020-11-20T12:06:22.511 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/33 file after the snapshot 2020-11-20T12:06:22.513 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/34 file after the snapshot 2020-11-20T12:06:22.517 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/35 file after the snapshot 2020-11-20T12:06:22.519 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/36 file after the snapshot 2020-11-20T12:06:22.522 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/37 file after the snapshot 2020-11-20T12:06:22.525 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/38 file after the snapshot 2020-11-20T12:06:22.527 INFO:tasks.workunit.client.0.smithi044.stdout: create 124/39 file after the snapshot 2020-11-20T12:06:22.529 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/.snap/snap-subdir-test 2020-11-20T12:06:22.554 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/30 file after the snapshot 2020-11-20T12:06:22.555 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/31 file after the snapshot 2020-11-20T12:06:22.557 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/32 file after the snapshot 2020-11-20T12:06:22.559 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/33 file after the snapshot 2020-11-20T12:06:22.561 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/34 file after the snapshot 2020-11-20T12:06:22.565 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/35 file after the snapshot 2020-11-20T12:06:22.566 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/36 file after the snapshot 2020-11-20T12:06:22.569 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/37 file after the snapshot 2020-11-20T12:06:22.572 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/38 file after the snapshot 2020-11-20T12:06:22.574 INFO:tasks.workunit.client.0.smithi044.stdout: create 125/39 file after the snapshot 2020-11-20T12:06:22.576 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/.snap/snap-subdir-test 2020-11-20T12:06:22.601 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/30 file after the snapshot 2020-11-20T12:06:22.603 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/31 file after the snapshot 2020-11-20T12:06:22.605 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/32 file after the snapshot 2020-11-20T12:06:22.608 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/33 file after the snapshot 2020-11-20T12:06:22.611 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/34 file after the snapshot 2020-11-20T12:06:22.613 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/35 file after the snapshot 2020-11-20T12:06:22.616 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/36 file after the snapshot 2020-11-20T12:06:22.618 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/37 file after the snapshot 2020-11-20T12:06:22.621 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/38 file after the snapshot 2020-11-20T12:06:22.624 INFO:tasks.workunit.client.0.smithi044.stdout: create 126/39 file after the snapshot 2020-11-20T12:06:22.626 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/.snap/snap-subdir-test 2020-11-20T12:06:22.660 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/30 file after the snapshot 2020-11-20T12:06:22.661 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/31 file after the snapshot 2020-11-20T12:06:22.663 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/32 file after the snapshot 2020-11-20T12:06:22.666 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/33 file after the snapshot 2020-11-20T12:06:22.668 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/34 file after the snapshot 2020-11-20T12:06:22.670 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/35 file after the snapshot 2020-11-20T12:06:22.673 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/36 file after the snapshot 2020-11-20T12:06:22.675 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/37 file after the snapshot 2020-11-20T12:06:22.677 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/38 file after the snapshot 2020-11-20T12:06:22.680 INFO:tasks.workunit.client.0.smithi044.stdout: create 127/39 file after the snapshot 2020-11-20T12:06:22.681 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/.snap/snap-subdir-test 2020-11-20T12:06:22.705 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/30 file after the snapshot 2020-11-20T12:06:22.706 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/31 file after the snapshot 2020-11-20T12:06:22.708 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/32 file after the snapshot 2020-11-20T12:06:22.711 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/33 file after the snapshot 2020-11-20T12:06:22.713 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/34 file after the snapshot 2020-11-20T12:06:22.716 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/35 file after the snapshot 2020-11-20T12:06:22.718 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/36 file after the snapshot 2020-11-20T12:06:22.721 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/37 file after the snapshot 2020-11-20T12:06:22.723 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/38 file after the snapshot 2020-11-20T12:06:22.725 INFO:tasks.workunit.client.0.smithi044.stdout: create 128/39 file after the snapshot 2020-11-20T12:06:22.728 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/.snap/snap-subdir-test 2020-11-20T12:06:22.752 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/30 file after the snapshot 2020-11-20T12:06:22.754 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/31 file after the snapshot 2020-11-20T12:06:22.755 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/32 file after the snapshot 2020-11-20T12:06:22.759 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/33 file after the snapshot 2020-11-20T12:06:22.762 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/34 file after the snapshot 2020-11-20T12:06:22.765 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/35 file after the snapshot 2020-11-20T12:06:22.767 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/36 file after the snapshot 2020-11-20T12:06:22.771 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/37 file after the snapshot 2020-11-20T12:06:22.772 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/38 file after the snapshot 2020-11-20T12:06:22.774 INFO:tasks.workunit.client.0.smithi044.stdout: create 129/39 file after the snapshot 2020-11-20T12:06:22.778 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/.snap/snap-subdir-test 2020-11-20T12:06:22.817 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/30 file after the snapshot 2020-11-20T12:06:22.882 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/31 file after the snapshot 2020-11-20T12:06:22.883 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/32 file after the snapshot 2020-11-20T12:06:22.886 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/33 file after the snapshot 2020-11-20T12:06:22.888 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/34 file after the snapshot 2020-11-20T12:06:22.890 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/35 file after the snapshot 2020-11-20T12:06:22.893 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/36 file after the snapshot 2020-11-20T12:06:22.895 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/37 file after the snapshot 2020-11-20T12:06:22.898 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/38 file after the snapshot 2020-11-20T12:06:22.900 INFO:tasks.workunit.client.0.smithi044.stdout: create 130/39 file after the snapshot 2020-11-20T12:06:22.903 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/.snap/snap-subdir-test 2020-11-20T12:06:22.964 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/30 file after the snapshot 2020-11-20T12:06:22.966 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/31 file after the snapshot 2020-11-20T12:06:22.967 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/32 file after the snapshot 2020-11-20T12:06:22.969 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/33 file after the snapshot 2020-11-20T12:06:22.971 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/34 file after the snapshot 2020-11-20T12:06:22.974 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/35 file after the snapshot 2020-11-20T12:06:22.976 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/36 file after the snapshot 2020-11-20T12:06:22.978 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/37 file after the snapshot 2020-11-20T12:06:22.980 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/38 file after the snapshot 2020-11-20T12:06:22.982 INFO:tasks.workunit.client.0.smithi044.stdout: create 131/39 file after the snapshot 2020-11-20T12:06:22.984 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/.snap/snap-subdir-test 2020-11-20T12:06:23.008 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/30 file after the snapshot 2020-11-20T12:06:23.010 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/31 file after the snapshot 2020-11-20T12:06:23.012 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/32 file after the snapshot 2020-11-20T12:06:23.014 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/33 file after the snapshot 2020-11-20T12:06:23.018 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/34 file after the snapshot 2020-11-20T12:06:23.020 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/35 file after the snapshot 2020-11-20T12:06:23.022 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/36 file after the snapshot 2020-11-20T12:06:23.024 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/37 file after the snapshot 2020-11-20T12:06:23.027 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/38 file after the snapshot 2020-11-20T12:06:23.030 INFO:tasks.workunit.client.0.smithi044.stdout: create 132/39 file after the snapshot 2020-11-20T12:06:23.033 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/.snap/snap-subdir-test 2020-11-20T12:06:23.057 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/30 file after the snapshot 2020-11-20T12:06:23.059 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/31 file after the snapshot 2020-11-20T12:06:23.061 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/32 file after the snapshot 2020-11-20T12:06:23.063 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/33 file after the snapshot 2020-11-20T12:06:23.064 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/34 file after the snapshot 2020-11-20T12:06:23.066 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/35 file after the snapshot 2020-11-20T12:06:23.069 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/36 file after the snapshot 2020-11-20T12:06:23.071 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/37 file after the snapshot 2020-11-20T12:06:23.073 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/38 file after the snapshot 2020-11-20T12:06:23.076 INFO:tasks.workunit.client.0.smithi044.stdout: create 133/39 file after the snapshot 2020-11-20T12:06:23.078 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/.snap/snap-subdir-test 2020-11-20T12:06:23.103 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/30 file after the snapshot 2020-11-20T12:06:23.104 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/31 file after the snapshot 2020-11-20T12:06:23.107 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/32 file after the snapshot 2020-11-20T12:06:23.110 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/33 file after the snapshot 2020-11-20T12:06:23.113 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/34 file after the snapshot 2020-11-20T12:06:23.115 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/35 file after the snapshot 2020-11-20T12:06:23.117 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/36 file after the snapshot 2020-11-20T12:06:23.120 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/37 file after the snapshot 2020-11-20T12:06:23.122 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/38 file after the snapshot 2020-11-20T12:06:23.124 INFO:tasks.workunit.client.0.smithi044.stdout: create 134/39 file after the snapshot 2020-11-20T12:06:23.127 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/.snap/snap-subdir-test 2020-11-20T12:06:23.152 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/30 file after the snapshot 2020-11-20T12:06:23.153 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/31 file after the snapshot 2020-11-20T12:06:23.155 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/32 file after the snapshot 2020-11-20T12:06:23.158 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/33 file after the snapshot 2020-11-20T12:06:23.160 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/34 file after the snapshot 2020-11-20T12:06:23.163 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/35 file after the snapshot 2020-11-20T12:06:23.165 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/36 file after the snapshot 2020-11-20T12:06:23.168 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/37 file after the snapshot 2020-11-20T12:06:23.171 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/38 file after the snapshot 2020-11-20T12:06:23.173 INFO:tasks.workunit.client.0.smithi044.stdout: create 135/39 file after the snapshot 2020-11-20T12:06:23.175 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/.snap/snap-subdir-test 2020-11-20T12:06:23.200 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/30 file after the snapshot 2020-11-20T12:06:23.201 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/31 file after the snapshot 2020-11-20T12:06:23.203 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/32 file after the snapshot 2020-11-20T12:06:23.206 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/33 file after the snapshot 2020-11-20T12:06:23.208 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/34 file after the snapshot 2020-11-20T12:06:23.210 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/35 file after the snapshot 2020-11-20T12:06:23.212 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/36 file after the snapshot 2020-11-20T12:06:23.214 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/37 file after the snapshot 2020-11-20T12:06:23.217 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/38 file after the snapshot 2020-11-20T12:06:23.219 INFO:tasks.workunit.client.0.smithi044.stdout: create 136/39 file after the snapshot 2020-11-20T12:06:23.221 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/.snap/snap-subdir-test 2020-11-20T12:06:23.245 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/30 file after the snapshot 2020-11-20T12:06:23.247 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/31 file after the snapshot 2020-11-20T12:06:23.248 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/32 file after the snapshot 2020-11-20T12:06:23.250 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/33 file after the snapshot 2020-11-20T12:06:23.253 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/34 file after the snapshot 2020-11-20T12:06:23.256 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/35 file after the snapshot 2020-11-20T12:06:23.259 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/36 file after the snapshot 2020-11-20T12:06:23.261 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/37 file after the snapshot 2020-11-20T12:06:23.264 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/38 file after the snapshot 2020-11-20T12:06:23.267 INFO:tasks.workunit.client.0.smithi044.stdout: create 137/39 file after the snapshot 2020-11-20T12:06:23.270 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/.snap/snap-subdir-test 2020-11-20T12:06:23.302 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/30 file after the snapshot 2020-11-20T12:06:23.304 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/31 file after the snapshot 2020-11-20T12:06:23.306 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/32 file after the snapshot 2020-11-20T12:06:23.308 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/33 file after the snapshot 2020-11-20T12:06:23.311 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/34 file after the snapshot 2020-11-20T12:06:23.313 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/35 file after the snapshot 2020-11-20T12:06:23.317 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/36 file after the snapshot 2020-11-20T12:06:23.320 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/37 file after the snapshot 2020-11-20T12:06:23.323 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/38 file after the snapshot 2020-11-20T12:06:23.325 INFO:tasks.workunit.client.0.smithi044.stdout: create 138/39 file after the snapshot 2020-11-20T12:06:23.328 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/.snap/snap-subdir-test 2020-11-20T12:06:23.352 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/30 file after the snapshot 2020-11-20T12:06:23.354 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/31 file after the snapshot 2020-11-20T12:06:23.356 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/32 file after the snapshot 2020-11-20T12:06:23.359 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/33 file after the snapshot 2020-11-20T12:06:23.361 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/34 file after the snapshot 2020-11-20T12:06:23.363 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/35 file after the snapshot 2020-11-20T12:06:23.366 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/36 file after the snapshot 2020-11-20T12:06:23.368 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/37 file after the snapshot 2020-11-20T12:06:23.370 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/38 file after the snapshot 2020-11-20T12:06:23.373 INFO:tasks.workunit.client.0.smithi044.stdout: create 139/39 file after the snapshot 2020-11-20T12:06:23.376 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/.snap/snap-subdir-test 2020-11-20T12:06:23.400 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/30 file after the snapshot 2020-11-20T12:06:23.403 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/31 file after the snapshot 2020-11-20T12:06:23.405 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/32 file after the snapshot 2020-11-20T12:06:23.409 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/33 file after the snapshot 2020-11-20T12:06:23.411 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/34 file after the snapshot 2020-11-20T12:06:23.414 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/35 file after the snapshot 2020-11-20T12:06:23.416 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/36 file after the snapshot 2020-11-20T12:06:23.418 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/37 file after the snapshot 2020-11-20T12:06:23.422 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/38 file after the snapshot 2020-11-20T12:06:23.424 INFO:tasks.workunit.client.0.smithi044.stdout: create 140/39 file after the snapshot 2020-11-20T12:06:23.427 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/.snap/snap-subdir-test 2020-11-20T12:06:23.450 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/30 file after the snapshot 2020-11-20T12:06:23.452 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/31 file after the snapshot 2020-11-20T12:06:23.454 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/32 file after the snapshot 2020-11-20T12:06:23.456 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/33 file after the snapshot 2020-11-20T12:06:23.459 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/34 file after the snapshot 2020-11-20T12:06:23.461 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/35 file after the snapshot 2020-11-20T12:06:23.463 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/36 file after the snapshot 2020-11-20T12:06:23.465 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/37 file after the snapshot 2020-11-20T12:06:23.469 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/38 file after the snapshot 2020-11-20T12:06:23.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 141/39 file after the snapshot 2020-11-20T12:06:23.474 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/.snap/snap-subdir-test 2020-11-20T12:06:23.497 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/30 file after the snapshot 2020-11-20T12:06:23.498 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/31 file after the snapshot 2020-11-20T12:06:23.501 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/32 file after the snapshot 2020-11-20T12:06:23.503 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/33 file after the snapshot 2020-11-20T12:06:23.505 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/34 file after the snapshot 2020-11-20T12:06:23.507 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/35 file after the snapshot 2020-11-20T12:06:23.510 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/36 file after the snapshot 2020-11-20T12:06:23.513 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/37 file after the snapshot 2020-11-20T12:06:23.515 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/38 file after the snapshot 2020-11-20T12:06:23.516 INFO:tasks.workunit.client.0.smithi044.stdout: create 142/39 file after the snapshot 2020-11-20T12:06:23.518 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/.snap/snap-subdir-test 2020-11-20T12:06:23.543 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/30 file after the snapshot 2020-11-20T12:06:23.545 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/31 file after the snapshot 2020-11-20T12:06:23.547 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/32 file after the snapshot 2020-11-20T12:06:23.549 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/33 file after the snapshot 2020-11-20T12:06:23.551 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/34 file after the snapshot 2020-11-20T12:06:23.554 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/35 file after the snapshot 2020-11-20T12:06:23.556 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/36 file after the snapshot 2020-11-20T12:06:23.559 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/37 file after the snapshot 2020-11-20T12:06:23.562 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/38 file after the snapshot 2020-11-20T12:06:23.565 INFO:tasks.workunit.client.0.smithi044.stdout: create 143/39 file after the snapshot 2020-11-20T12:06:23.567 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/.snap/snap-subdir-test 2020-11-20T12:06:23.591 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/30 file after the snapshot 2020-11-20T12:06:23.592 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/31 file after the snapshot 2020-11-20T12:06:23.594 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/32 file after the snapshot 2020-11-20T12:06:23.595 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/33 file after the snapshot 2020-11-20T12:06:23.597 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/34 file after the snapshot 2020-11-20T12:06:23.599 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/35 file after the snapshot 2020-11-20T12:06:23.603 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/36 file after the snapshot 2020-11-20T12:06:23.606 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/37 file after the snapshot 2020-11-20T12:06:23.608 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/38 file after the snapshot 2020-11-20T12:06:23.610 INFO:tasks.workunit.client.0.smithi044.stdout: create 144/39 file after the snapshot 2020-11-20T12:06:23.612 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/.snap/snap-subdir-test 2020-11-20T12:06:23.637 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/30 file after the snapshot 2020-11-20T12:06:23.639 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/31 file after the snapshot 2020-11-20T12:06:23.641 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/32 file after the snapshot 2020-11-20T12:06:23.643 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/33 file after the snapshot 2020-11-20T12:06:23.645 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/34 file after the snapshot 2020-11-20T12:06:23.647 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/35 file after the snapshot 2020-11-20T12:06:23.650 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/36 file after the snapshot 2020-11-20T12:06:23.652 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/37 file after the snapshot 2020-11-20T12:06:23.655 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/38 file after the snapshot 2020-11-20T12:06:23.657 INFO:tasks.workunit.client.0.smithi044.stdout: create 145/39 file after the snapshot 2020-11-20T12:06:23.660 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/.snap/snap-subdir-test 2020-11-20T12:06:23.683 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/30 file after the snapshot 2020-11-20T12:06:23.685 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/31 file after the snapshot 2020-11-20T12:06:23.687 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/32 file after the snapshot 2020-11-20T12:06:23.689 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/33 file after the snapshot 2020-11-20T12:06:23.691 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/34 file after the snapshot 2020-11-20T12:06:23.693 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/35 file after the snapshot 2020-11-20T12:06:23.695 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/36 file after the snapshot 2020-11-20T12:06:23.697 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/37 file after the snapshot 2020-11-20T12:06:23.699 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/38 file after the snapshot 2020-11-20T12:06:23.702 INFO:tasks.workunit.client.0.smithi044.stdout: create 146/39 file after the snapshot 2020-11-20T12:06:23.705 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/.snap/snap-subdir-test 2020-11-20T12:06:23.729 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/30 file after the snapshot 2020-11-20T12:06:23.731 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/31 file after the snapshot 2020-11-20T12:06:23.732 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/32 file after the snapshot 2020-11-20T12:06:23.737 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/33 file after the snapshot 2020-11-20T12:06:23.740 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/34 file after the snapshot 2020-11-20T12:06:23.748 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/35 file after the snapshot 2020-11-20T12:06:23.751 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/36 file after the snapshot 2020-11-20T12:06:23.756 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/37 file after the snapshot 2020-11-20T12:06:23.759 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/38 file after the snapshot 2020-11-20T12:06:23.761 INFO:tasks.workunit.client.0.smithi044.stdout: create 147/39 file after the snapshot 2020-11-20T12:06:23.764 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/.snap/snap-subdir-test 2020-11-20T12:06:23.785 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/30 file after the snapshot 2020-11-20T12:06:23.787 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/31 file after the snapshot 2020-11-20T12:06:23.788 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/32 file after the snapshot 2020-11-20T12:06:23.791 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/33 file after the snapshot 2020-11-20T12:06:23.794 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/34 file after the snapshot 2020-11-20T12:06:23.796 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/35 file after the snapshot 2020-11-20T12:06:23.798 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/36 file after the snapshot 2020-11-20T12:06:23.800 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/37 file after the snapshot 2020-11-20T12:06:23.802 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/38 file after the snapshot 2020-11-20T12:06:23.805 INFO:tasks.workunit.client.0.smithi044.stdout: create 148/39 file after the snapshot 2020-11-20T12:06:23.807 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/.snap/snap-subdir-test 2020-11-20T12:06:23.841 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/30 file after the snapshot 2020-11-20T12:06:23.996 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/31 file after the snapshot 2020-11-20T12:06:23.998 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/32 file after the snapshot 2020-11-20T12:06:24.000 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/33 file after the snapshot 2020-11-20T12:06:24.003 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/34 file after the snapshot 2020-11-20T12:06:24.006 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/35 file after the snapshot 2020-11-20T12:06:24.008 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/36 file after the snapshot 2020-11-20T12:06:24.011 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/37 file after the snapshot 2020-11-20T12:06:24.014 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/38 file after the snapshot 2020-11-20T12:06:24.017 INFO:tasks.workunit.client.0.smithi044.stdout: create 149/39 file after the snapshot 2020-11-20T12:06:24.019 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/.snap/snap-subdir-test 2020-11-20T12:06:24.123 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/30 file after the snapshot 2020-11-20T12:06:24.124 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/31 file after the snapshot 2020-11-20T12:06:24.126 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/32 file after the snapshot 2020-11-20T12:06:24.129 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/33 file after the snapshot 2020-11-20T12:06:24.131 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/34 file after the snapshot 2020-11-20T12:06:24.134 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/35 file after the snapshot 2020-11-20T12:06:24.137 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/36 file after the snapshot 2020-11-20T12:06:24.139 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/37 file after the snapshot 2020-11-20T12:06:24.141 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/38 file after the snapshot 2020-11-20T12:06:24.143 INFO:tasks.workunit.client.0.smithi044.stdout: create 150/39 file after the snapshot 2020-11-20T12:06:24.145 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/.snap/snap-subdir-test 2020-11-20T12:06:24.169 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/30 file after the snapshot 2020-11-20T12:06:24.171 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/31 file after the snapshot 2020-11-20T12:06:24.172 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/32 file after the snapshot 2020-11-20T12:06:24.174 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/33 file after the snapshot 2020-11-20T12:06:24.177 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/34 file after the snapshot 2020-11-20T12:06:24.179 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/35 file after the snapshot 2020-11-20T12:06:24.182 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/36 file after the snapshot 2020-11-20T12:06:24.186 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/37 file after the snapshot 2020-11-20T12:06:24.188 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/38 file after the snapshot 2020-11-20T12:06:24.190 INFO:tasks.workunit.client.0.smithi044.stdout: create 151/39 file after the snapshot 2020-11-20T12:06:24.192 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/.snap/snap-subdir-test 2020-11-20T12:06:24.216 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/30 file after the snapshot 2020-11-20T12:06:24.217 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/31 file after the snapshot 2020-11-20T12:06:24.219 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/32 file after the snapshot 2020-11-20T12:06:24.221 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/33 file after the snapshot 2020-11-20T12:06:24.224 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/34 file after the snapshot 2020-11-20T12:06:24.227 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/35 file after the snapshot 2020-11-20T12:06:24.230 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/36 file after the snapshot 2020-11-20T12:06:24.232 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/37 file after the snapshot 2020-11-20T12:06:24.233 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/38 file after the snapshot 2020-11-20T12:06:24.236 INFO:tasks.workunit.client.0.smithi044.stdout: create 152/39 file after the snapshot 2020-11-20T12:06:24.239 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/.snap/snap-subdir-test 2020-11-20T12:06:24.271 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/30 file after the snapshot 2020-11-20T12:06:24.272 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/31 file after the snapshot 2020-11-20T12:06:24.274 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/32 file after the snapshot 2020-11-20T12:06:24.276 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/33 file after the snapshot 2020-11-20T12:06:24.278 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/34 file after the snapshot 2020-11-20T12:06:24.280 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/35 file after the snapshot 2020-11-20T12:06:24.282 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/36 file after the snapshot 2020-11-20T12:06:24.285 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/37 file after the snapshot 2020-11-20T12:06:24.287 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/38 file after the snapshot 2020-11-20T12:06:24.290 INFO:tasks.workunit.client.0.smithi044.stdout: create 153/39 file after the snapshot 2020-11-20T12:06:24.293 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/.snap/snap-subdir-test 2020-11-20T12:06:24.318 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/30 file after the snapshot 2020-11-20T12:06:24.319 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/31 file after the snapshot 2020-11-20T12:06:24.321 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/32 file after the snapshot 2020-11-20T12:06:24.325 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/33 file after the snapshot 2020-11-20T12:06:24.327 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/34 file after the snapshot 2020-11-20T12:06:24.329 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/35 file after the snapshot 2020-11-20T12:06:24.331 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/36 file after the snapshot 2020-11-20T12:06:24.335 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/37 file after the snapshot 2020-11-20T12:06:24.337 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/38 file after the snapshot 2020-11-20T12:06:24.340 INFO:tasks.workunit.client.0.smithi044.stdout: create 154/39 file after the snapshot 2020-11-20T12:06:24.343 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/.snap/snap-subdir-test 2020-11-20T12:06:24.374 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/30 file after the snapshot 2020-11-20T12:06:24.376 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/31 file after the snapshot 2020-11-20T12:06:24.378 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/32 file after the snapshot 2020-11-20T12:06:24.380 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/33 file after the snapshot 2020-11-20T12:06:24.383 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/34 file after the snapshot 2020-11-20T12:06:24.385 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/35 file after the snapshot 2020-11-20T12:06:24.388 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/36 file after the snapshot 2020-11-20T12:06:24.390 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/37 file after the snapshot 2020-11-20T12:06:24.394 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/38 file after the snapshot 2020-11-20T12:06:24.396 INFO:tasks.workunit.client.0.smithi044.stdout: create 155/39 file after the snapshot 2020-11-20T12:06:24.399 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/.snap/snap-subdir-test 2020-11-20T12:06:24.422 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/30 file after the snapshot 2020-11-20T12:06:24.424 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/31 file after the snapshot 2020-11-20T12:06:24.426 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/32 file after the snapshot 2020-11-20T12:06:24.428 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/33 file after the snapshot 2020-11-20T12:06:24.431 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/34 file after the snapshot 2020-11-20T12:06:24.433 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/35 file after the snapshot 2020-11-20T12:06:24.435 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/36 file after the snapshot 2020-11-20T12:06:24.437 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/37 file after the snapshot 2020-11-20T12:06:24.440 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/38 file after the snapshot 2020-11-20T12:06:24.442 INFO:tasks.workunit.client.0.smithi044.stdout: create 156/39 file after the snapshot 2020-11-20T12:06:24.444 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/.snap/snap-subdir-test 2020-11-20T12:06:24.488 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/30 file after the snapshot 2020-11-20T12:06:24.490 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/31 file after the snapshot 2020-11-20T12:06:24.492 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/32 file after the snapshot 2020-11-20T12:06:24.494 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/33 file after the snapshot 2020-11-20T12:06:24.496 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/34 file after the snapshot 2020-11-20T12:06:24.497 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/35 file after the snapshot 2020-11-20T12:06:24.499 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/36 file after the snapshot 2020-11-20T12:06:24.500 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/37 file after the snapshot 2020-11-20T12:06:24.502 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/38 file after the snapshot 2020-11-20T12:06:24.503 INFO:tasks.workunit.client.0.smithi044.stdout: create 157/39 file after the snapshot 2020-11-20T12:06:24.505 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/.snap/snap-subdir-test 2020-11-20T12:06:24.530 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/30 file after the snapshot 2020-11-20T12:06:24.533 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/31 file after the snapshot 2020-11-20T12:06:24.534 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/32 file after the snapshot 2020-11-20T12:06:24.536 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/33 file after the snapshot 2020-11-20T12:06:24.537 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/34 file after the snapshot 2020-11-20T12:06:24.539 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/35 file after the snapshot 2020-11-20T12:06:24.542 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/36 file after the snapshot 2020-11-20T12:06:24.545 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/37 file after the snapshot 2020-11-20T12:06:24.547 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/38 file after the snapshot 2020-11-20T12:06:24.549 INFO:tasks.workunit.client.0.smithi044.stdout: create 158/39 file after the snapshot 2020-11-20T12:06:24.553 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/.snap/snap-subdir-test 2020-11-20T12:06:24.577 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/30 file after the snapshot 2020-11-20T12:06:24.578 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/31 file after the snapshot 2020-11-20T12:06:24.581 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/32 file after the snapshot 2020-11-20T12:06:24.583 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/33 file after the snapshot 2020-11-20T12:06:24.588 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/34 file after the snapshot 2020-11-20T12:06:24.588 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/35 file after the snapshot 2020-11-20T12:06:24.590 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/36 file after the snapshot 2020-11-20T12:06:24.592 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/37 file after the snapshot 2020-11-20T12:06:24.594 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/38 file after the snapshot 2020-11-20T12:06:24.597 INFO:tasks.workunit.client.0.smithi044.stdout: create 159/39 file after the snapshot 2020-11-20T12:06:24.599 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/.snap/snap-subdir-test 2020-11-20T12:06:24.623 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/30 file after the snapshot 2020-11-20T12:06:24.625 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/31 file after the snapshot 2020-11-20T12:06:24.627 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/32 file after the snapshot 2020-11-20T12:06:24.629 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/33 file after the snapshot 2020-11-20T12:06:24.631 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/34 file after the snapshot 2020-11-20T12:06:24.632 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/35 file after the snapshot 2020-11-20T12:06:24.635 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/36 file after the snapshot 2020-11-20T12:06:24.637 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/37 file after the snapshot 2020-11-20T12:06:24.640 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/38 file after the snapshot 2020-11-20T12:06:24.644 INFO:tasks.workunit.client.0.smithi044.stdout: create 160/39 file after the snapshot 2020-11-20T12:06:24.646 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/.snap/snap-subdir-test 2020-11-20T12:06:24.667 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/30 file after the snapshot 2020-11-20T12:06:24.669 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/31 file after the snapshot 2020-11-20T12:06:24.671 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/32 file after the snapshot 2020-11-20T12:06:24.674 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/33 file after the snapshot 2020-11-20T12:06:24.675 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/34 file after the snapshot 2020-11-20T12:06:24.677 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/35 file after the snapshot 2020-11-20T12:06:24.679 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/36 file after the snapshot 2020-11-20T12:06:24.682 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/37 file after the snapshot 2020-11-20T12:06:24.685 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/38 file after the snapshot 2020-11-20T12:06:24.688 INFO:tasks.workunit.client.0.smithi044.stdout: create 161/39 file after the snapshot 2020-11-20T12:06:24.690 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/.snap/snap-subdir-test 2020-11-20T12:06:24.712 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/30 file after the snapshot 2020-11-20T12:06:24.714 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/31 file after the snapshot 2020-11-20T12:06:24.715 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/32 file after the snapshot 2020-11-20T12:06:24.717 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/33 file after the snapshot 2020-11-20T12:06:24.720 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/34 file after the snapshot 2020-11-20T12:06:24.722 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/35 file after the snapshot 2020-11-20T12:06:24.725 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/36 file after the snapshot 2020-11-20T12:06:24.728 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/37 file after the snapshot 2020-11-20T12:06:24.730 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/38 file after the snapshot 2020-11-20T12:06:24.732 INFO:tasks.workunit.client.0.smithi044.stdout: create 162/39 file after the snapshot 2020-11-20T12:06:24.734 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/.snap/snap-subdir-test 2020-11-20T12:06:24.756 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/30 file after the snapshot 2020-11-20T12:06:24.758 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/31 file after the snapshot 2020-11-20T12:06:24.761 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/32 file after the snapshot 2020-11-20T12:06:24.764 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/33 file after the snapshot 2020-11-20T12:06:24.766 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/34 file after the snapshot 2020-11-20T12:06:24.769 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/35 file after the snapshot 2020-11-20T12:06:24.771 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/36 file after the snapshot 2020-11-20T12:06:24.773 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/37 file after the snapshot 2020-11-20T12:06:24.777 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/38 file after the snapshot 2020-11-20T12:06:24.779 INFO:tasks.workunit.client.0.smithi044.stdout: create 163/39 file after the snapshot 2020-11-20T12:06:24.782 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/.snap/snap-subdir-test 2020-11-20T12:06:24.806 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/30 file after the snapshot 2020-11-20T12:06:24.808 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/31 file after the snapshot 2020-11-20T12:06:24.809 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/32 file after the snapshot 2020-11-20T12:06:24.811 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/33 file after the snapshot 2020-11-20T12:06:24.813 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/34 file after the snapshot 2020-11-20T12:06:24.817 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/35 file after the snapshot 2020-11-20T12:06:24.819 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/36 file after the snapshot 2020-11-20T12:06:24.821 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/37 file after the snapshot 2020-11-20T12:06:24.824 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/38 file after the snapshot 2020-11-20T12:06:24.826 INFO:tasks.workunit.client.0.smithi044.stdout: create 164/39 file after the snapshot 2020-11-20T12:06:24.827 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/.snap/snap-subdir-test 2020-11-20T12:06:24.855 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/30 file after the snapshot 2020-11-20T12:06:24.984 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/31 file after the snapshot 2020-11-20T12:06:24.986 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/32 file after the snapshot 2020-11-20T12:06:24.989 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/33 file after the snapshot 2020-11-20T12:06:24.992 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/34 file after the snapshot 2020-11-20T12:06:24.994 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/35 file after the snapshot 2020-11-20T12:06:24.996 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/36 file after the snapshot 2020-11-20T12:06:25.000 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/37 file after the snapshot 2020-11-20T12:06:25.002 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/38 file after the snapshot 2020-11-20T12:06:25.004 INFO:tasks.workunit.client.0.smithi044.stdout: create 165/39 file after the snapshot 2020-11-20T12:06:25.007 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/.snap/snap-subdir-test 2020-11-20T12:06:25.088 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/30 file after the snapshot 2020-11-20T12:06:25.090 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/31 file after the snapshot 2020-11-20T12:06:25.092 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/32 file after the snapshot 2020-11-20T12:06:25.095 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/33 file after the snapshot 2020-11-20T12:06:25.097 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/34 file after the snapshot 2020-11-20T12:06:25.100 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/35 file after the snapshot 2020-11-20T12:06:25.103 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/36 file after the snapshot 2020-11-20T12:06:25.106 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/37 file after the snapshot 2020-11-20T12:06:25.108 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/38 file after the snapshot 2020-11-20T12:06:25.111 INFO:tasks.workunit.client.0.smithi044.stdout: create 166/39 file after the snapshot 2020-11-20T12:06:25.114 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/.snap/snap-subdir-test 2020-11-20T12:06:25.145 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/30 file after the snapshot 2020-11-20T12:06:25.146 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/31 file after the snapshot 2020-11-20T12:06:25.148 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/32 file after the snapshot 2020-11-20T12:06:25.152 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/33 file after the snapshot 2020-11-20T12:06:25.155 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/34 file after the snapshot 2020-11-20T12:06:25.157 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/35 file after the snapshot 2020-11-20T12:06:25.159 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/36 file after the snapshot 2020-11-20T12:06:25.161 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/37 file after the snapshot 2020-11-20T12:06:25.164 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/38 file after the snapshot 2020-11-20T12:06:25.167 INFO:tasks.workunit.client.0.smithi044.stdout: create 167/39 file after the snapshot 2020-11-20T12:06:25.169 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/.snap/snap-subdir-test 2020-11-20T12:06:25.191 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/30 file after the snapshot 2020-11-20T12:06:25.192 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/31 file after the snapshot 2020-11-20T12:06:25.194 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/32 file after the snapshot 2020-11-20T12:06:25.196 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/33 file after the snapshot 2020-11-20T12:06:25.199 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/34 file after the snapshot 2020-11-20T12:06:25.202 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/35 file after the snapshot 2020-11-20T12:06:25.205 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/36 file after the snapshot 2020-11-20T12:06:25.207 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/37 file after the snapshot 2020-11-20T12:06:25.210 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/38 file after the snapshot 2020-11-20T12:06:25.211 INFO:tasks.workunit.client.0.smithi044.stdout: create 168/39 file after the snapshot 2020-11-20T12:06:25.214 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/.snap/snap-subdir-test 2020-11-20T12:06:25.237 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/30 file after the snapshot 2020-11-20T12:06:25.238 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/31 file after the snapshot 2020-11-20T12:06:25.241 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/32 file after the snapshot 2020-11-20T12:06:25.243 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/33 file after the snapshot 2020-11-20T12:06:25.247 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/34 file after the snapshot 2020-11-20T12:06:25.248 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/35 file after the snapshot 2020-11-20T12:06:25.250 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/36 file after the snapshot 2020-11-20T12:06:25.253 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/37 file after the snapshot 2020-11-20T12:06:25.255 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/38 file after the snapshot 2020-11-20T12:06:25.258 INFO:tasks.workunit.client.0.smithi044.stdout: create 169/39 file after the snapshot 2020-11-20T12:06:25.261 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/.snap/snap-subdir-test 2020-11-20T12:06:25.285 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/30 file after the snapshot 2020-11-20T12:06:25.286 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/31 file after the snapshot 2020-11-20T12:06:25.289 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/32 file after the snapshot 2020-11-20T12:06:25.291 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/33 file after the snapshot 2020-11-20T12:06:25.294 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/34 file after the snapshot 2020-11-20T12:06:25.296 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/35 file after the snapshot 2020-11-20T12:06:25.299 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/36 file after the snapshot 2020-11-20T12:06:25.301 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/37 file after the snapshot 2020-11-20T12:06:25.303 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/38 file after the snapshot 2020-11-20T12:06:25.306 INFO:tasks.workunit.client.0.smithi044.stdout: create 170/39 file after the snapshot 2020-11-20T12:06:25.309 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/.snap/snap-subdir-test 2020-11-20T12:06:25.332 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/30 file after the snapshot 2020-11-20T12:06:25.334 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/31 file after the snapshot 2020-11-20T12:06:25.336 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/32 file after the snapshot 2020-11-20T12:06:25.338 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/33 file after the snapshot 2020-11-20T12:06:25.340 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/34 file after the snapshot 2020-11-20T12:06:25.343 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/35 file after the snapshot 2020-11-20T12:06:25.345 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/36 file after the snapshot 2020-11-20T12:06:25.348 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/37 file after the snapshot 2020-11-20T12:06:25.350 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/38 file after the snapshot 2020-11-20T12:06:25.352 INFO:tasks.workunit.client.0.smithi044.stdout: create 171/39 file after the snapshot 2020-11-20T12:06:25.355 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/.snap/snap-subdir-test 2020-11-20T12:06:25.377 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/30 file after the snapshot 2020-11-20T12:06:25.379 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/31 file after the snapshot 2020-11-20T12:06:25.381 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/32 file after the snapshot 2020-11-20T12:06:25.383 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/33 file after the snapshot 2020-11-20T12:06:25.385 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/34 file after the snapshot 2020-11-20T12:06:25.387 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/35 file after the snapshot 2020-11-20T12:06:25.390 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/36 file after the snapshot 2020-11-20T12:06:25.392 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/37 file after the snapshot 2020-11-20T12:06:25.395 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/38 file after the snapshot 2020-11-20T12:06:25.397 INFO:tasks.workunit.client.0.smithi044.stdout: create 172/39 file after the snapshot 2020-11-20T12:06:25.400 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/.snap/snap-subdir-test 2020-11-20T12:06:25.423 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/30 file after the snapshot 2020-11-20T12:06:25.425 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/31 file after the snapshot 2020-11-20T12:06:25.426 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/32 file after the snapshot 2020-11-20T12:06:25.428 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/33 file after the snapshot 2020-11-20T12:06:25.430 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/34 file after the snapshot 2020-11-20T12:06:25.434 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/35 file after the snapshot 2020-11-20T12:06:25.436 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/36 file after the snapshot 2020-11-20T12:06:25.438 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/37 file after the snapshot 2020-11-20T12:06:25.441 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/38 file after the snapshot 2020-11-20T12:06:25.444 INFO:tasks.workunit.client.0.smithi044.stdout: create 173/39 file after the snapshot 2020-11-20T12:06:25.446 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/.snap/snap-subdir-test 2020-11-20T12:06:25.469 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/30 file after the snapshot 2020-11-20T12:06:25.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/31 file after the snapshot 2020-11-20T12:06:25.473 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/32 file after the snapshot 2020-11-20T12:06:25.475 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/33 file after the snapshot 2020-11-20T12:06:25.479 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/34 file after the snapshot 2020-11-20T12:06:25.481 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/35 file after the snapshot 2020-11-20T12:06:25.483 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/36 file after the snapshot 2020-11-20T12:06:25.486 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/37 file after the snapshot 2020-11-20T12:06:25.488 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/38 file after the snapshot 2020-11-20T12:06:25.491 INFO:tasks.workunit.client.0.smithi044.stdout: create 174/39 file after the snapshot 2020-11-20T12:06:25.494 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/.snap/snap-subdir-test 2020-11-20T12:06:25.517 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/30 file after the snapshot 2020-11-20T12:06:25.519 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/31 file after the snapshot 2020-11-20T12:06:25.520 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/32 file after the snapshot 2020-11-20T12:06:25.522 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/33 file after the snapshot 2020-11-20T12:06:25.524 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/34 file after the snapshot 2020-11-20T12:06:25.526 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/35 file after the snapshot 2020-11-20T12:06:25.528 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/36 file after the snapshot 2020-11-20T12:06:25.530 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/37 file after the snapshot 2020-11-20T12:06:25.532 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/38 file after the snapshot 2020-11-20T12:06:25.535 INFO:tasks.workunit.client.0.smithi044.stdout: create 175/39 file after the snapshot 2020-11-20T12:06:25.537 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/.snap/snap-subdir-test 2020-11-20T12:06:25.558 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/30 file after the snapshot 2020-11-20T12:06:25.560 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/31 file after the snapshot 2020-11-20T12:06:25.562 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/32 file after the snapshot 2020-11-20T12:06:25.564 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/33 file after the snapshot 2020-11-20T12:06:25.567 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/34 file after the snapshot 2020-11-20T12:06:25.569 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/35 file after the snapshot 2020-11-20T12:06:25.572 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/36 file after the snapshot 2020-11-20T12:06:25.574 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/37 file after the snapshot 2020-11-20T12:06:25.576 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/38 file after the snapshot 2020-11-20T12:06:25.580 INFO:tasks.workunit.client.0.smithi044.stdout: create 176/39 file after the snapshot 2020-11-20T12:06:25.583 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/.snap/snap-subdir-test 2020-11-20T12:06:25.606 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/30 file after the snapshot 2020-11-20T12:06:25.608 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/31 file after the snapshot 2020-11-20T12:06:25.609 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/32 file after the snapshot 2020-11-20T12:06:25.611 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/33 file after the snapshot 2020-11-20T12:06:25.614 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/34 file after the snapshot 2020-11-20T12:06:25.616 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/35 file after the snapshot 2020-11-20T12:06:25.619 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/36 file after the snapshot 2020-11-20T12:06:25.621 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/37 file after the snapshot 2020-11-20T12:06:25.624 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/38 file after the snapshot 2020-11-20T12:06:25.627 INFO:tasks.workunit.client.0.smithi044.stdout: create 177/39 file after the snapshot 2020-11-20T12:06:25.629 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/.snap/snap-subdir-test 2020-11-20T12:06:25.652 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/30 file after the snapshot 2020-11-20T12:06:25.654 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/31 file after the snapshot 2020-11-20T12:06:25.656 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/32 file after the snapshot 2020-11-20T12:06:25.659 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/33 file after the snapshot 2020-11-20T12:06:25.661 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/34 file after the snapshot 2020-11-20T12:06:25.664 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/35 file after the snapshot 2020-11-20T12:06:25.667 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/36 file after the snapshot 2020-11-20T12:06:25.669 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/37 file after the snapshot 2020-11-20T12:06:25.670 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/38 file after the snapshot 2020-11-20T12:06:25.672 INFO:tasks.workunit.client.0.smithi044.stdout: create 178/39 file after the snapshot 2020-11-20T12:06:25.675 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/.snap/snap-subdir-test 2020-11-20T12:06:25.697 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/30 file after the snapshot 2020-11-20T12:06:25.699 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/31 file after the snapshot 2020-11-20T12:06:25.700 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/32 file after the snapshot 2020-11-20T12:06:25.704 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/33 file after the snapshot 2020-11-20T12:06:25.707 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/34 file after the snapshot 2020-11-20T12:06:25.709 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/35 file after the snapshot 2020-11-20T12:06:25.711 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/36 file after the snapshot 2020-11-20T12:06:25.713 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/37 file after the snapshot 2020-11-20T12:06:25.715 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/38 file after the snapshot 2020-11-20T12:06:25.718 INFO:tasks.workunit.client.0.smithi044.stdout: create 179/39 file after the snapshot 2020-11-20T12:06:25.719 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/.snap/snap-subdir-test 2020-11-20T12:06:25.741 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/30 file after the snapshot 2020-11-20T12:06:25.743 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/31 file after the snapshot 2020-11-20T12:06:25.745 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/32 file after the snapshot 2020-11-20T12:06:25.749 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/33 file after the snapshot 2020-11-20T12:06:25.751 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/34 file after the snapshot 2020-11-20T12:06:25.753 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/35 file after the snapshot 2020-11-20T12:06:25.755 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/36 file after the snapshot 2020-11-20T12:06:25.757 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/37 file after the snapshot 2020-11-20T12:06:25.760 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/38 file after the snapshot 2020-11-20T12:06:25.762 INFO:tasks.workunit.client.0.smithi044.stdout: create 180/39 file after the snapshot 2020-11-20T12:06:25.765 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/.snap/snap-subdir-test 2020-11-20T12:06:25.787 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/30 file after the snapshot 2020-11-20T12:06:25.789 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/31 file after the snapshot 2020-11-20T12:06:25.791 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/32 file after the snapshot 2020-11-20T12:06:25.794 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/33 file after the snapshot 2020-11-20T12:06:25.797 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/34 file after the snapshot 2020-11-20T12:06:25.800 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/35 file after the snapshot 2020-11-20T12:06:25.803 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/36 file after the snapshot 2020-11-20T12:06:25.804 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/37 file after the snapshot 2020-11-20T12:06:25.808 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/38 file after the snapshot 2020-11-20T12:06:25.810 INFO:tasks.workunit.client.0.smithi044.stdout: create 181/39 file after the snapshot 2020-11-20T12:06:25.812 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/.snap/snap-subdir-test 2020-11-20T12:06:25.835 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/30 file after the snapshot 2020-11-20T12:06:25.837 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/31 file after the snapshot 2020-11-20T12:06:25.838 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/32 file after the snapshot 2020-11-20T12:06:25.841 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/33 file after the snapshot 2020-11-20T12:06:25.844 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/34 file after the snapshot 2020-11-20T12:06:25.850 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/35 file after the snapshot 2020-11-20T12:06:25.850 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/36 file after the snapshot 2020-11-20T12:06:25.851 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/37 file after the snapshot 2020-11-20T12:06:25.867 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/38 file after the snapshot 2020-11-20T12:06:25.990 INFO:tasks.workunit.client.0.smithi044.stdout: create 182/39 file after the snapshot 2020-11-20T12:06:25.991 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/.snap/snap-subdir-test 2020-11-20T12:06:26.052 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:06:26.053 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:06:26.077 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/30 file after the snapshot 2020-11-20T12:06:26.080 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/31 file after the snapshot 2020-11-20T12:06:26.082 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/32 file after the snapshot 2020-11-20T12:06:26.084 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/33 file after the snapshot 2020-11-20T12:06:26.087 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/34 file after the snapshot 2020-11-20T12:06:26.091 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/35 file after the snapshot 2020-11-20T12:06:26.094 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/36 file after the snapshot 2020-11-20T12:06:26.096 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/37 file after the snapshot 2020-11-20T12:06:26.099 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/38 file after the snapshot 2020-11-20T12:06:26.102 INFO:tasks.workunit.client.0.smithi044.stdout: create 183/39 file after the snapshot 2020-11-20T12:06:26.105 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/.snap/snap-subdir-test 2020-11-20T12:06:26.124 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/30 file after the snapshot 2020-11-20T12:06:26.126 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/31 file after the snapshot 2020-11-20T12:06:26.129 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/32 file after the snapshot 2020-11-20T12:06:26.133 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/33 file after the snapshot 2020-11-20T12:06:26.136 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/34 file after the snapshot 2020-11-20T12:06:26.138 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/35 file after the snapshot 2020-11-20T12:06:26.140 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/36 file after the snapshot 2020-11-20T12:06:26.143 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/37 file after the snapshot 2020-11-20T12:06:26.146 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/38 file after the snapshot 2020-11-20T12:06:26.148 INFO:tasks.workunit.client.0.smithi044.stdout: create 184/39 file after the snapshot 2020-11-20T12:06:26.152 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/.snap/snap-subdir-test 2020-11-20T12:06:26.171 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/30 file after the snapshot 2020-11-20T12:06:26.173 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/31 file after the snapshot 2020-11-20T12:06:26.175 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/32 file after the snapshot 2020-11-20T12:06:26.178 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/33 file after the snapshot 2020-11-20T12:06:26.180 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/34 file after the snapshot 2020-11-20T12:06:26.183 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/35 file after the snapshot 2020-11-20T12:06:26.186 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/36 file after the snapshot 2020-11-20T12:06:26.188 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/37 file after the snapshot 2020-11-20T12:06:26.191 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/38 file after the snapshot 2020-11-20T12:06:26.193 INFO:tasks.workunit.client.0.smithi044.stdout: create 185/39 file after the snapshot 2020-11-20T12:06:26.196 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/.snap/snap-subdir-test 2020-11-20T12:06:26.218 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/30 file after the snapshot 2020-11-20T12:06:26.220 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/31 file after the snapshot 2020-11-20T12:06:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.217+0000 7fd82b306700 1 Processor -- start 2020-11-20T12:06:26.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.217+0000 7fd82b306700 1 -- start start 2020-11-20T12:06:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd82b306700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd824064190 0x7fd8240aa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd82b306700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd82413c570 0x7fd82413ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd82b306700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82413bed0 con 0x7fd824064570 2020-11-20T12:06:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd82b306700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8240aa900 con 0x7fd824064190 2020-11-20T12:06:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd82b306700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8240a6a60 con 0x7fd82413c570 2020-11-20T12:06:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd8290a2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd824064190 0x7fd8240aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd8298a3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd82413c570 0x7fd82413ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd8298a3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd82413c570 0x7fd82413ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49414/0 (socket says 172.21.15.44:49414) 2020-11-20T12:06:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd8290a2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd824064190 0x7fd8240aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58254/0 (socket says 172.21.15.44:58254) 2020-11-20T12:06:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd8298a3700 1 -- 172.21.15.44:0/1482740624 learned_addr learned my addr 172.21.15.44:0/1482740624 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.218+0000 7fd8288a1700 1 --1- 172.21.15.44:0/1482740624 >> v1:172.21.15.79:6789/0 conn(0x7fd824064570 0x7fd824064e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44598/0 (socket says 172.21.15.44:44598) 2020-11-20T12:06:26.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd8298a3700 1 -- 172.21.15.44:0/1482740624 >> v2:172.21.15.79:3300/0 conn(0x7fd824064190 msgr2=0x7fd8240aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd8298a3700 1 --2- 172.21.15.44:0/1482740624 >> v2:172.21.15.79:3300/0 conn(0x7fd824064190 0x7fd8240aa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd8298a3700 1 -- 172.21.15.44:0/1482740624 >> v1:172.21.15.79:6789/0 conn(0x7fd824064570 legacy=0x7fd824064e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd8298a3700 1 -- 172.21.15.44:0/1482740624 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240a6d00 con 0x7fd82413c570 2020-11-20T12:06:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd8298a3700 1 --2- 172.21.15.44:0/1482740624 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd82413c570 0x7fd82413ca00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd81c009d50 tx=0x7fd81c009290).ready entity=mon.2 client_cookie=80b5a4a227c2cc65 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd813fff700 1 -- 172.21.15.44:0/1482740624 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00bc50 con 0x7fd82413c570 2020-11-20T12:06:26.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd813fff700 1 -- 172.21.15.44:0/1482740624 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c00bdb0 con 0x7fd82413c570 2020-11-20T12:06:26.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.219+0000 7fd813fff700 1 -- 172.21.15.44:0/1482740624 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c011610 con 0x7fd82413c570 2020-11-20T12:06:26.227 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/32 file after the snapshot 2020-11-20T12:06:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 -- 172.21.15.44:0/1482740624 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd82413c570 msgr2=0x7fd82413ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 --2- 172.21.15.44:0/1482740624 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd82413c570 0x7fd82413ca00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fd81c009d50 tx=0x7fd81c009290).stop 2020-11-20T12:06:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 -- 172.21.15.44:0/1482740624 shutdown_connections 2020-11-20T12:06:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 --2- 172.21.15.44:0/1482740624 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd82413c570 0x7fd82413ca00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 --2- 172.21.15.44:0/1482740624 >> v2:172.21.15.79:3300/0 conn(0x7fd824064190 0x7fd8240aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 -- 172.21.15.44:0/1482740624 >> 172.21.15.44:0/1482740624 conn(0x7fd8240af040 msgr2=0x7fd8240af450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 -- 172.21.15.44:0/1482740624 shutdown_connections 2020-11-20T12:06:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.220+0000 7fd82b306700 1 -- 172.21.15.44:0/1482740624 wait complete. 2020-11-20T12:06:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.222+0000 7fd82b306700 1 Processor -- start 2020-11-20T12:06:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.222+0000 7fd82b306700 1 -- start start 2020-11-20T12:06:26.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd82b306700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd824064190 0x7fd8240a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd82b306700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd824064570 0x7fd8240a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd82b306700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd82413c570 0x7fd8240a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd82b306700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8240a2a30 con 0x7fd82413c570 2020-11-20T12:06:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd82b306700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8240a4210 con 0x7fd824064190 2020-11-20T12:06:26.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd82b306700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8240a2c60 con 0x7fd824064570 2020-11-20T12:06:26.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd824064190 0x7fd8240a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd824064190 0x7fd8240a8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58260/0 (socket says 172.21.15.44:58260) 2020-11-20T12:06:26.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 -- 172.21.15.44:0/3521760156 learned_addr learned my addr 172.21.15.44:0/3521760156 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:26.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 -- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd824064570 msgr2=0x7fd8240a3100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2020-11-20T12:06:26.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd824064570 0x7fd8240a3100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 -- 172.21.15.44:0/3521760156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd82413c570 msgr2=0x7fd8240a39e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd82413c570 0x7fd8240a39e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.223+0000 7fd8290a2700 1 -- 172.21.15.44:0/3521760156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240a35f0 con 0x7fd824064190 2020-11-20T12:06:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.224+0000 7fd8290a2700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd824064190 0x7fd8240a8c30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd82000bf70 tx=0x7fd820006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.224+0000 7fd8117fa700 1 -- 172.21.15.44:0/3521760156 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000c9b0 con 0x7fd824064190 2020-11-20T12:06:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.224+0000 7fd8117fa700 1 -- 172.21.15.44:0/3521760156 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82000cb10 con 0x7fd824064190 2020-11-20T12:06:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.224+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8240a3ed0 con 0x7fd824064190 2020-11-20T12:06:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.224+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd824138bf0 con 0x7fd824064190 2020-11-20T12:06:26.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.224+0000 7fd8117fa700 1 -- 172.21.15.44:0/3521760156 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000ce90 con 0x7fd824064190 2020-11-20T12:06:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.225+0000 7fd8117fa700 1 -- 172.21.15.44:0/3521760156 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd820023610 con 0x7fd824064190 2020-11-20T12:06:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.225+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82405c2a0 con 0x7fd824064190 2020-11-20T12:06:26.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.226+0000 7fd8117fa700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd7fc056d70 0x7fd7fc059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.236 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/33 file after the snapshot 2020-11-20T12:06:26.236 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/34 file after the snapshot 2020-11-20T12:06:26.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.226+0000 7fd8117fa700 1 -- 172.21.15.44:0/3521760156 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd8200815a0 con 0x7fd824064190 2020-11-20T12:06:26.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.226+0000 7fd8288a1700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd7fc056d70 0x7fd7fc059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.227+0000 7fd8288a1700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd7fc056d70 0x7fd7fc059230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fd8240a5220 tx=0x7fd814007780).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.230+0000 7fd8117fa700 1 -- 172.21.15.44:0/3521760156 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd8200568f0 con 0x7fd824064190 2020-11-20T12:06:26.238 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/35 file after the snapshot 2020-11-20T12:06:26.239 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/36 file after the snapshot 2020-11-20T12:06:26.239 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/37 file after the snapshot 2020-11-20T12:06:26.239 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/38 file after the snapshot 2020-11-20T12:06:26.242 INFO:tasks.workunit.client.0.smithi044.stdout: create 186/39 file after the snapshot 2020-11-20T12:06:26.244 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/.snap/snap-subdir-test 2020-11-20T12:06:26.273 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/30 file after the snapshot 2020-11-20T12:06:26.275 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/31 file after the snapshot 2020-11-20T12:06:26.278 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/32 file after the snapshot 2020-11-20T12:06:26.281 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/33 file after the snapshot 2020-11-20T12:06:26.283 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/34 file after the snapshot 2020-11-20T12:06:26.287 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/35 file after the snapshot 2020-11-20T12:06:26.291 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/36 file after the snapshot 2020-11-20T12:06:26.294 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/37 file after the snapshot 2020-11-20T12:06:26.296 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/38 file after the snapshot 2020-11-20T12:06:26.298 INFO:tasks.workunit.client.0.smithi044.stdout: create 187/39 file after the snapshot 2020-11-20T12:06:26.300 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/.snap/snap-subdir-test 2020-11-20T12:06:26.322 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/30 file after the snapshot 2020-11-20T12:06:26.323 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/31 file after the snapshot 2020-11-20T12:06:26.325 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/32 file after the snapshot 2020-11-20T12:06:26.330 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/33 file after the snapshot 2020-11-20T12:06:26.332 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/34 file after the snapshot 2020-11-20T12:06:26.335 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/35 file after the snapshot 2020-11-20T12:06:26.337 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/36 file after the snapshot 2020-11-20T12:06:26.340 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/37 file after the snapshot 2020-11-20T12:06:26.343 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/38 file after the snapshot 2020-11-20T12:06:26.345 INFO:tasks.workunit.client.0.smithi044.stdout: create 188/39 file after the snapshot 2020-11-20T12:06:26.350 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/.snap/snap-subdir-test 2020-11-20T12:06:26.370 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/30 file after the snapshot 2020-11-20T12:06:26.372 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/31 file after the snapshot 2020-11-20T12:06:26.374 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/32 file after the snapshot 2020-11-20T12:06:26.376 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/33 file after the snapshot 2020-11-20T12:06:26.379 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/34 file after the snapshot 2020-11-20T12:06:26.382 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/35 file after the snapshot 2020-11-20T12:06:26.384 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/36 file after the snapshot 2020-11-20T12:06:26.386 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/37 file after the snapshot 2020-11-20T12:06:26.388 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/38 file after the snapshot 2020-11-20T12:06:26.391 INFO:tasks.workunit.client.0.smithi044.stdout: create 189/39 file after the snapshot 2020-11-20T12:06:26.394 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/.snap/snap-subdir-test 2020-11-20T12:06:26.418 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/30 file after the snapshot 2020-11-20T12:06:26.420 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/31 file after the snapshot 2020-11-20T12:06:26.422 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/32 file after the snapshot 2020-11-20T12:06:26.426 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/33 file after the snapshot 2020-11-20T12:06:26.428 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/34 file after the snapshot 2020-11-20T12:06:26.430 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/35 file after the snapshot 2020-11-20T12:06:26.433 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/36 file after the snapshot 2020-11-20T12:06:26.436 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/37 file after the snapshot 2020-11-20T12:06:26.438 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/38 file after the snapshot 2020-11-20T12:06:26.441 INFO:tasks.workunit.client.0.smithi044.stdout: create 190/39 file after the snapshot 2020-11-20T12:06:26.443 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/.snap/snap-subdir-test 2020-11-20T12:06:26.467 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/30 file after the snapshot 2020-11-20T12:06:26.469 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/31 file after the snapshot 2020-11-20T12:06:26.472 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/32 file after the snapshot 2020-11-20T12:06:26.475 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/33 file after the snapshot 2020-11-20T12:06:26.479 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/34 file after the snapshot 2020-11-20T12:06:26.481 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/35 file after the snapshot 2020-11-20T12:06:26.483 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/36 file after the snapshot 2020-11-20T12:06:26.487 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/37 file after the snapshot 2020-11-20T12:06:26.489 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/38 file after the snapshot 2020-11-20T12:06:26.491 INFO:tasks.workunit.client.0.smithi044.stdout: create 191/39 file after the snapshot 2020-11-20T12:06:26.494 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/.snap/snap-subdir-test 2020-11-20T12:06:26.519 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/30 file after the snapshot 2020-11-20T12:06:26.520 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/31 file after the snapshot 2020-11-20T12:06:26.522 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/32 file after the snapshot 2020-11-20T12:06:26.525 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/33 file after the snapshot 2020-11-20T12:06:26.528 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/34 file after the snapshot 2020-11-20T12:06:26.531 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/35 file after the snapshot 2020-11-20T12:06:26.533 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/36 file after the snapshot 2020-11-20T12:06:26.536 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/37 file after the snapshot 2020-11-20T12:06:26.540 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/38 file after the snapshot 2020-11-20T12:06:26.542 INFO:tasks.workunit.client.0.smithi044.stdout: create 192/39 file after the snapshot 2020-11-20T12:06:26.545 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/.snap/snap-subdir-test 2020-11-20T12:06:26.566 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/30 file after the snapshot 2020-11-20T12:06:26.568 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/31 file after the snapshot 2020-11-20T12:06:26.570 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/32 file after the snapshot 2020-11-20T12:06:26.573 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/33 file after the snapshot 2020-11-20T12:06:26.576 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/34 file after the snapshot 2020-11-20T12:06:26.578 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/35 file after the snapshot 2020-11-20T12:06:26.580 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/36 file after the snapshot 2020-11-20T12:06:26.583 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/37 file after the snapshot 2020-11-20T12:06:26.585 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/38 file after the snapshot 2020-11-20T12:06:26.587 INFO:tasks.workunit.client.0.smithi044.stdout: create 193/39 file after the snapshot 2020-11-20T12:06:26.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.585+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd8240446d0 con 0x7fd824064190 2020-11-20T12:06:26.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.585+0000 7fd8117fa700 1 -- 172.21.15.44:0/3521760156 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fd820059f10 con 0x7fd824064190 2020-11-20T12:06:26.590 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:06:26.591 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/.snap/snap-subdir-test 2020-11-20T12:06:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd7fc056d70 msgr2=0x7fd7fc059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd7fc056d70 0x7fd7fc059230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fd8240a5220 tx=0x7fd814007780).stop 2020-11-20T12:06:26.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd824064190 msgr2=0x7fd8240a8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd824064190 0x7fd8240a8c30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd82000bf70 tx=0x7fd820006df0).stop 2020-11-20T12:06:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 shutdown_connections 2020-11-20T12:06:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd7fc056d70 0x7fd7fc059230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd82413c570 0x7fd8240a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:26.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd824064190 0x7fd8240a8c30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 --2- 172.21.15.44:0/3521760156 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd824064570 0x7fd8240a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 >> 172.21.15.44:0/3521760156 conn(0x7fd8240af040 msgr2=0x7fd824066e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 shutdown_connections 2020-11-20T12:06:26.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.589+0000 7fd82b306700 1 -- 172.21.15.44:0/3521760156 wait complete. 2020-11-20T12:06:26.606 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:06:26.607 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:06:26.607 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:06:26.615 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/30 file after the snapshot 2020-11-20T12:06:26.617 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/31 file after the snapshot 2020-11-20T12:06:26.619 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/32 file after the snapshot 2020-11-20T12:06:26.621 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/33 file after the snapshot 2020-11-20T12:06:26.625 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/34 file after the snapshot 2020-11-20T12:06:26.627 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/35 file after the snapshot 2020-11-20T12:06:26.629 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/36 file after the snapshot 2020-11-20T12:06:26.633 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/37 file after the snapshot 2020-11-20T12:06:26.635 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/38 file after the snapshot 2020-11-20T12:06:26.638 INFO:tasks.workunit.client.0.smithi044.stdout: create 194/39 file after the snapshot 2020-11-20T12:06:26.641 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/.snap/snap-subdir-test 2020-11-20T12:06:26.666 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/30 file after the snapshot 2020-11-20T12:06:26.668 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/31 file after the snapshot 2020-11-20T12:06:26.671 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/32 file after the snapshot 2020-11-20T12:06:26.673 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/33 file after the snapshot 2020-11-20T12:06:26.675 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/34 file after the snapshot 2020-11-20T12:06:26.679 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/35 file after the snapshot 2020-11-20T12:06:26.681 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/36 file after the snapshot 2020-11-20T12:06:26.684 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/37 file after the snapshot 2020-11-20T12:06:26.687 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/38 file after the snapshot 2020-11-20T12:06:26.690 INFO:tasks.workunit.client.0.smithi044.stdout: create 195/39 file after the snapshot 2020-11-20T12:06:26.692 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/.snap/snap-subdir-test 2020-11-20T12:06:26.719 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/30 file after the snapshot 2020-11-20T12:06:26.720 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/31 file after the snapshot 2020-11-20T12:06:26.723 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/32 file after the snapshot 2020-11-20T12:06:26.726 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/33 file after the snapshot 2020-11-20T12:06:26.729 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/34 file after the snapshot 2020-11-20T12:06:26.732 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/35 file after the snapshot 2020-11-20T12:06:26.736 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/36 file after the snapshot 2020-11-20T12:06:26.738 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/37 file after the snapshot 2020-11-20T12:06:26.740 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/38 file after the snapshot 2020-11-20T12:06:26.743 INFO:tasks.workunit.client.0.smithi044.stdout: create 196/39 file after the snapshot 2020-11-20T12:06:26.746 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/.snap/snap-subdir-test 2020-11-20T12:06:26.770 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/30 file after the snapshot 2020-11-20T12:06:26.772 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/31 file after the snapshot 2020-11-20T12:06:26.774 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/32 file after the snapshot 2020-11-20T12:06:26.777 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/33 file after the snapshot 2020-11-20T12:06:26.779 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/34 file after the snapshot 2020-11-20T12:06:26.783 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/35 file after the snapshot 2020-11-20T12:06:26.786 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/36 file after the snapshot 2020-11-20T12:06:26.788 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/37 file after the snapshot 2020-11-20T12:06:26.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.787+0000 7f1c9d728700 1 Processor -- start 2020-11-20T12:06:26.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.787+0000 7f1c9d728700 1 -- start start 2020-11-20T12:06:26.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c9d728700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1c980aa1e0 0x7f1c981372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c9d728700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c980a7280 con 0x7f1c980a4800 2020-11-20T12:06:26.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c9d728700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c980a84c0 con 0x7f1c980a64c0 2020-11-20T12:06:26.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c9d728700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c980a7090 con 0x7f1c980aa1e0 2020-11-20T12:06:26.794 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/38 file after the snapshot 2020-11-20T12:06:26.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c96ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1c980aa1e0 0x7f1c981372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c96ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1c980aa1e0 0x7f1c981372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37984/0 (socket says 172.21.15.44:37984) 2020-11-20T12:06:26.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c977fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f1c980a64c0 0x7f1c98141c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44610/0 (socket says 172.21.15.44:44610) 2020-11-20T12:06:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.788+0000 7f1c96ffd700 1 -- 172.21.15.44:0/699178943 learned_addr learned my addr 172.21.15.44:0/699178943 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c95ffb700 1 -- 172.21.15.44:0/699178943 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900793381 0 0) 0x7f1c980a84c0 con 0x7f1c980a64c0 2020-11-20T12:06:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c95ffb700 1 -- 172.21.15.44:0/699178943 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c84003240 con 0x7f1c980a64c0 2020-11-20T12:06:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c96ffd700 1 -- 172.21.15.44:0/699178943 >> v1:172.21.15.79:6789/0 conn(0x7f1c980a64c0 legacy=0x7f1c98141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c96ffd700 1 -- 172.21.15.44:0/699178943 >> v1:172.21.15.44:6789/0 conn(0x7f1c980a4800 legacy=0x7f1c980a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c96ffd700 1 -- 172.21.15.44:0/699178943 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980a8760 con 0x7f1c980aa1e0 2020-11-20T12:06:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c96ffd700 1 --2- 172.21.15.44:0/699178943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980aa1e0 0x7f1c981372e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f1c88006fd0 tx=0x7f1c88006f40).ready entity=mon.0 client_cookie=819734495578c892 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c95ffb700 1 -- 172.21.15.44:0/699178943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88017380 con 0x7f1c980aa1e0 2020-11-20T12:06:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c95ffb700 1 -- 172.21.15.44:0/699178943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c880174e0 con 0x7f1c980aa1e0 2020-11-20T12:06:26.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.789+0000 7f1c95ffb700 1 -- 172.21.15.44:0/699178943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88017860 con 0x7f1c980aa1e0 2020-11-20T12:06:26.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.790+0000 7f1c9d728700 1 -- 172.21.15.44:0/699178943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980aa1e0 msgr2=0x7f1c981372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.790+0000 7f1c9d728700 1 --2- 172.21.15.44:0/699178943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980aa1e0 0x7f1c981372e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f1c88006fd0 tx=0x7f1c88006f40).stop 2020-11-20T12:06:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.790+0000 7f1c9d728700 1 -- 172.21.15.44:0/699178943 shutdown_connections 2020-11-20T12:06:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.790+0000 7f1c9d728700 1 --2- 172.21.15.44:0/699178943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980aa1e0 0x7f1c981372e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.790+0000 7f1c9d728700 1 -- 172.21.15.44:0/699178943 >> 172.21.15.44:0/699178943 conn(0x7f1c98062690 msgr2=0x7f1c980a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.791+0000 7f1c9d728700 1 -- 172.21.15.44:0/699178943 shutdown_connections 2020-11-20T12:06:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.791+0000 7f1c9d728700 1 -- 172.21.15.44:0/699178943 wait complete. 2020-11-20T12:06:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 Processor -- start 2020-11-20T12:06:26.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 -- start start 2020-11-20T12:06:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980a4800 0x7f1c98137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c980a64c0 0x7f1c9813a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c980aa1e0 0x7f1c981361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9813a0c0 con 0x7f1c980a4800 2020-11-20T12:06:26.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980a8cb0 con 0x7f1c980a64c0 2020-11-20T12:06:26.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.793+0000 7f1c9d728700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98067610 con 0x7f1c980aa1e0 2020-11-20T12:06:26.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c980aa1e0 0x7f1c981361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c980aa1e0 0x7f1c981361f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49428/0 (socket says 172.21.15.44:49428) 2020-11-20T12:06:26.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 -- 172.21.15.44:0/1593201913 learned_addr learned my addr 172.21.15.44:0/1593201913 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c967fc700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c980a64c0 0x7f1c9813a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c96ffd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980a4800 0x7f1c98137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c980a64c0 msgr2=0x7f1c9813a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c980a64c0 0x7f1c9813a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980a4800 msgr2=0x7f1c98137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c980a4800 0x7f1c98137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:26.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 -- 172.21.15.44:0/1593201913 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9813a2b0 con 0x7f1c980aa1e0 2020-11-20T12:06:26.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c977fe700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c980aa1e0 0x7f1c981361f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1c9000cac0 tx=0x7f1c90007ec0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90005ba0 con 0x7f1c980aa1e0 2020-11-20T12:06:26.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.794+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c90005d00 con 0x7f1c980aa1e0 2020-11-20T12:06:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.795+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90007320 con 0x7f1c980aa1e0 2020-11-20T12:06:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.795+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c98138f30 con 0x7f1c980aa1e0 2020-11-20T12:06:26.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.795+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c98067a40 con 0x7f1c980aa1e0 2020-11-20T12:06:26.805 INFO:tasks.workunit.client.0.smithi044.stdout: create 197/39 file after the snapshot 2020-11-20T12:06:26.805 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/.snap/snap-subdir-test 2020-11-20T12:06:26.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.796+0000 7f1c9d728700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c981366e0 0x7f1c98140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.796+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98067750 con 0x7f1c981366e0 2020-11-20T12:06:26.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.796+0000 7f1c96ffd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c981366e0 0x7f1c98140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.796+0000 7f1c96ffd700 1 -- 172.21.15.44:0/1593201913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c98061650 con 0x7f1c981366e0 2020-11-20T12:06:26.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.796+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1c90010030 con 0x7f1c980aa1e0 2020-11-20T12:06:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.797+0000 7f1c96ffd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c981366e0 0x7f1c98140560 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1c88012010 tx=0x7f1c88016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.797+0000 7f1c7effd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c68056d20 0x7f1c680591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.797+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1c90070220 con 0x7f1c980aa1e0 2020-11-20T12:06:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.797+0000 7f1c967fc700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c68056d20 0x7f1c680591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.797+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8801d070 con 0x7f1c981366e0 2020-11-20T12:06:26.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.798+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f1c88017c20 con 0x7f1c981366e0 2020-11-20T12:06:26.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.798+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c981366e0 msgr2=0x7f1c98140560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.798+0000 7f1c7effd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c981366e0 0x7f1c98140560 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1c88012010 tx=0x7f1c88016000).stop 2020-11-20T12:06:26.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.798+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c981366e0 msgr2=0x7f1c98140560 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:26.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.798+0000 7f1c7effd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c981366e0 0x7f1c98140560 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0x7f1c88012010 tx=0x7f1c88016000).stop 2020-11-20T12:06:26.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.798+0000 7f1c967fc700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c68056d20 0x7f1c680591e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f1c980675c0 tx=0x7f1c8c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.807+0000 7f1c9d728700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c98140a50 0x7f1c98140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:26.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.808+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980aa5c0 con 0x7f1c98140a50 2020-11-20T12:06:26.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.808+0000 7f1c96ffd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c98140a50 0x7f1c98140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:26.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.808+0000 7f1c96ffd700 1 -- 172.21.15.44:0/1593201913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1c98067d00 con 0x7f1c98140a50 2020-11-20T12:06:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.808+0000 7f1c96ffd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c98140a50 0x7f1c98140e60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1c98067890 tx=0x7f1c8801d000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.808+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88024320 con 0x7f1c98140a50 2020-11-20T12:06:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7f1c88018930 con 0x7f1c98140a50 2020-11-20T12:06:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c98140a50 msgr2=0x7f1c98140e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c7effd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c98140a50 0x7f1c98140e60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1c98067890 tx=0x7f1c8801d000).stop 2020-11-20T12:06:26.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c7effd700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c98140a50 msgr2=0x7f1c98140e60 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:26.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c7effd700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c98140a50 0x7f1c98140e60 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0x7f1c98067890 tx=0x7f1c8801d000).stop 2020-11-20T12:06:26.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c68056d20 msgr2=0x7f1c680591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c68056d20 0x7f1c680591e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f1c980675c0 tx=0x7f1c8c009040).stop 2020-11-20T12:06:26.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c980aa1e0 msgr2=0x7f1c981361f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:26.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 --2- 172.21.15.44:0/1593201913 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c980aa1e0 0x7f1c981361f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1c9000cac0 tx=0x7f1c90007ec0).stop 2020-11-20T12:06:26.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c977fe700 1 -- 172.21.15.44:0/1593201913 reap_dead start 2020-11-20T12:06:26.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c977fe700 1 -- 172.21.15.44:0/1593201913 reap_dead start 2020-11-20T12:06:26.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 shutdown_connections 2020-11-20T12:06:26.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 >> 172.21.15.44:0/1593201913 conn(0x7f1c98062690 msgr2=0x7f1c980a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:26.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 shutdown_connections 2020-11-20T12:06:26.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:26.809+0000 7f1c9d728700 1 -- 172.21.15.44:0/1593201913 wait complete. 2020-11-20T12:06:26.817 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:26.817 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:26.818 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:26.818 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:06:26.818 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 26, 2020-11-20T12:06:26.818 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:26.818 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:06:26.819 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:26.819 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:26.819 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:26.819 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:06:26.819 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:26.820 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:26.820 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:26.820 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:26.820 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:26.820 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:26.821 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:26.821 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:26.821 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:26.821 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:26.822 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:26.822 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:26.822 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:26.822 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:26.822 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:26.823 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:26.823 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:26.823 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:26.823 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:26.823 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:26.824 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:26.824 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:26.824 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:26.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.824 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:26.825 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:26.825 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:26.825 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:26.825 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:26.826 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:26.826 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:26.826 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:26.826 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:26.826 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:26.827 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:26.827 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:26.827 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:26.827 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:26.827 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:26.828 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:26.828 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:26.828 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:26.828 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:26.828 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:26.829 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:26.829 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:26.829 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:26.829 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:26.829 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:26.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.830 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:26.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.830 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:26.830 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:26.831 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:26.831 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:26.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.831 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:26.831 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:26.832 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:26.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:26.832 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.833 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:26.834 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:26.834 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:26.834 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:26.834 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:26.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.835 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:26.835 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:26.835 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:26.835 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:26.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:26.836 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:26.836 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:26.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.837 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:26.837 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:26.837 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:26.837 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.838 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:26.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.838 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:26.838 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:26.838 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:26.839 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:26.839 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:26.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.840 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:26.840 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:26.840 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:26.840 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:26.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:26.841 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:26.841 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:26.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.842 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:26.842 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:26.842 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:26.842 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.842 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:26.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.843 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:26.843 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:26.843 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:26.843 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:26.844 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:26.844 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.844 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:26.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.844 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:26.845 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:26.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.845 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:26.845 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:26.845 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:26.846 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.846 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:26.846 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:06:26.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.846 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:26.847 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:26.847 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:06:26.847 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.847 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:26.847 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:06:26.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.848 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:26.848 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:26.848 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:06:26.848 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.849 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:26.849 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:26.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.849 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:26.849 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:26.850 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:06:26.850 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.850 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:26.850 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:06:26.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:26.851 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:26.851 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:26.851 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:26.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:26.851 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:26.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:26.852 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:26.852 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:26.854 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/30 file after the snapshot 2020-11-20T12:06:26.854 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/31 file after the snapshot 2020-11-20T12:06:26.854 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/32 file after the snapshot 2020-11-20T12:06:26.854 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/33 file after the snapshot 2020-11-20T12:06:26.855 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/34 file after the snapshot 2020-11-20T12:06:26.855 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/35 file after the snapshot 2020-11-20T12:06:26.855 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/36 file after the snapshot 2020-11-20T12:06:26.855 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/37 file after the snapshot 2020-11-20T12:06:26.855 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/38 file after the snapshot 2020-11-20T12:06:26.856 INFO:tasks.workunit.client.0.smithi044.stdout: create 198/39 file after the snapshot 2020-11-20T12:06:26.856 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/.snap/snap-subdir-test 2020-11-20T12:06:26.856 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:06:26.885 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/30 file after the snapshot 2020-11-20T12:06:26.935 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/31 file after the snapshot 2020-11-20T12:06:26.938 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/32 file after the snapshot 2020-11-20T12:06:26.941 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/33 file after the snapshot 2020-11-20T12:06:26.943 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/34 file after the snapshot 2020-11-20T12:06:26.945 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/35 file after the snapshot 2020-11-20T12:06:26.948 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/36 file after the snapshot 2020-11-20T12:06:26.950 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/37 file after the snapshot 2020-11-20T12:06:26.952 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/38 file after the snapshot 2020-11-20T12:06:26.956 INFO:tasks.workunit.client.0.smithi044.stdout: create 199/39 file after the snapshot 2020-11-20T12:06:26.959 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/.snap/snap-subdir-test 2020-11-20T12:06:27.014 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/30 file after the snapshot 2020-11-20T12:06:27.016 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/31 file after the snapshot 2020-11-20T12:06:27.018 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/32 file after the snapshot 2020-11-20T12:06:27.021 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/33 file after the snapshot 2020-11-20T12:06:27.024 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/34 file after the snapshot 2020-11-20T12:06:27.026 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/35 file after the snapshot 2020-11-20T12:06:27.031 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/36 file after the snapshot 2020-11-20T12:06:27.033 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/37 file after the snapshot 2020-11-20T12:06:27.035 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/38 file after the snapshot 2020-11-20T12:06:27.038 INFO:tasks.workunit.client.0.smithi044.stdout: create 200/39 file after the snapshot 2020-11-20T12:06:27.041 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/.snap/snap-subdir-test 2020-11-20T12:06:27.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.038+0000 7fcf8f6c3700 1 Processor -- start 2020-11-20T12:06:27.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.039+0000 7fcf8f6c3700 1 -- start start 2020-11-20T12:06:27.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8f6c3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880ab620 0x7fcf88137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8f6c3700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf88067350 con 0x7fcf880abc30 2020-11-20T12:06:27.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8f6c3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf880ab040 con 0x7fcf881380b0 2020-11-20T12:06:27.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8f6c3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf880a9b40 con 0x7fcf880ab620 2020-11-20T12:06:27.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8d45f700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcf880abc30 0x7fcf880666d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:41750/0 (socket says 172.21.15.44:41750) 2020-11-20T12:06:27.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8d45f700 1 -- 172.21.15.44:0/4169450499 learned_addr learned my addr 172.21.15.44:0/4169450499 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:06:27.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8cc5e700 1 --2- 172.21.15.44:0/4169450499 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880ab620 0x7fcf88137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf73fff700 1 -- 172.21.15.44:0/4169450499 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877393504 0 0) 0x7fcf88067350 con 0x7fcf880abc30 2020-11-20T12:06:27.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf73fff700 1 -- 172.21.15.44:0/4169450499 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf6c003240 con 0x7fcf880abc30 2020-11-20T12:06:27.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8cc5e700 1 -- 172.21.15.44:0/4169450499 >> v1:172.21.15.79:6789/0 conn(0x7fcf881380b0 legacy=0x7fcf88145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8cc5e700 1 -- 172.21.15.44:0/4169450499 >> v1:172.21.15.44:6789/0 conn(0x7fcf880abc30 legacy=0x7fcf880666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.040+0000 7fcf8cc5e700 1 -- 172.21.15.44:0/4169450499 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf88067880 con 0x7fcf880ab620 2020-11-20T12:06:27.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.041+0000 7fcf8cc5e700 1 --2- 172.21.15.44:0/4169450499 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880ab620 0x7fcf88137bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fcf7c00b9d0 tx=0x7fcf7c006df0).ready entity=mon.2 client_cookie=22ebafbd330f2ac0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.041+0000 7fcf73fff700 1 -- 172.21.15.44:0/4169450499 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0097b0 con 0x7fcf880ab620 2020-11-20T12:06:27.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.041+0000 7fcf73fff700 1 -- 172.21.15.44:0/4169450499 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c009910 con 0x7fcf880ab620 2020-11-20T12:06:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.041+0000 7fcf73fff700 1 -- 172.21.15.44:0/4169450499 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c009c90 con 0x7fcf880ab620 2020-11-20T12:06:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.042+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/4169450499 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880ab620 msgr2=0x7fcf88137bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.042+0000 7fcf8f6c3700 1 --2- 172.21.15.44:0/4169450499 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880ab620 0x7fcf88137bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fcf7c00b9d0 tx=0x7fcf7c006df0).stop 2020-11-20T12:06:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.042+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/4169450499 shutdown_connections 2020-11-20T12:06:27.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.042+0000 7fcf8f6c3700 1 --2- 172.21.15.44:0/4169450499 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880ab620 0x7fcf88137bc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.042+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/4169450499 >> 172.21.15.44:0/4169450499 conn(0x7fcf88065290 msgr2=0x7fcf88060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.042+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/4169450499 shutdown_connections 2020-11-20T12:06:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.042+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/4169450499 wait complete. 2020-11-20T12:06:27.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.044+0000 7fcf8f6c3700 1 Processor -- start 2020-11-20T12:06:27.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.044+0000 7fcf8f6c3700 1 -- start start 2020-11-20T12:06:27.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.044+0000 7fcf8f6c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880ab620 0x7fcf88063100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8f6c3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880abc30 0x7fcf8813cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8f6c3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf881380b0 0x7fcf8814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8f6c3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88063990 con 0x7fcf881380b0 2020-11-20T12:06:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8f6c3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8813d7e0 con 0x7fcf880ab620 2020-11-20T12:06:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8f6c3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88061a80 con 0x7fcf880abc30 2020-11-20T12:06:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8d45f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880ab620 0x7fcf88063100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8dc60700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf881380b0 0x7fcf8814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880abc30 0x7fcf8813cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880abc30 0x7fcf8813cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49448/0 (socket says 172.21.15.44:49448) 2020-11-20T12:06:27.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 -- 172.21.15.44:0/3027133380 learned_addr learned my addr 172.21.15.44:0/3027133380 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:27.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8d45f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880ab620 0x7fcf88063100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58290/0 (socket says 172.21.15.44:58290) 2020-11-20T12:06:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880ab620 msgr2=0x7fcf88063100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880ab620 0x7fcf88063100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf881380b0 msgr2=0x7fcf8814e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf881380b0 0x7fcf8814e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 -- 172.21.15.44:0/3027133380 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf88061d20 con 0x7fcf880abc30 2020-11-20T12:06:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8dc60700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf881380b0 0x7fcf8814e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf8cc5e700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880abc30 0x7fcf8813cfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fcf7c006e30 tx=0x7fcf7c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c011d50 con 0x7fcf880abc30 2020-11-20T12:06:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c01d3c0 con 0x7fcf880abc30 2020-11-20T12:06:27.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.045+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c01d760 con 0x7fcf880abc30 2020-11-20T12:06:27.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.046+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8813d4a0 con 0x7fcf880abc30 2020-11-20T12:06:27.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.046+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8814ed30 con 0x7fcf880abc30 2020-11-20T12:06:27.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.046+0000 7fcf8f6c3700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880643c0 0x7fcf88131a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.046+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8814ef20 con 0x7fcf880643c0 2020-11-20T12:06:27.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf8d45f700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880643c0 0x7fcf88131a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcf7c01a040 con 0x7fcf880abc30 2020-11-20T12:06:27.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf8d45f700 1 -- 172.21.15.44:0/3027133380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf88061eb0 con 0x7fcf880643c0 2020-11-20T12:06:27.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf79ffb700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf5c056d20 0x7fcf5c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fcf7c071520 con 0x7fcf880abc30 2020-11-20T12:06:27.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf8d45f700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880643c0 0x7fcf88131a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcf8000fba0 tx=0x7fcf80009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf8dc60700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf5c056d20 0x7fcf5c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.047+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80005b60 con 0x7fcf880643c0 2020-11-20T12:06:27.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.048+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fcf80005cc0 con 0x7fcf880643c0 2020-11-20T12:06:27.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.048+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880643c0 msgr2=0x7fcf88131a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.048+0000 7fcf79ffb700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880643c0 0x7fcf88131a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcf8000fba0 tx=0x7fcf80009040).stop 2020-11-20T12:06:27.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.048+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880643c0 msgr2=0x7fcf88131a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:27.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.048+0000 7fcf79ffb700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880643c0 0x7fcf88131a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.048+0000 7fcf8dc60700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf5c056d20 0x7fcf5c0591e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fcf8813ced0 tx=0x7fcf84009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.057+0000 7fcf8f6c3700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880620e0 0x7fcf88131f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.057+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88061bc0 con 0x7fcf880620e0 2020-11-20T12:06:27.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.057+0000 7fcf8d45f700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880620e0 0x7fcf88131f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.057+0000 7fcf8d45f700 1 -- 172.21.15.44:0/3027133380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcf8805c2a0 con 0x7fcf880620e0 2020-11-20T12:06:27.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.057+0000 7fcf8d45f700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880620e0 0x7fcf88131f10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcf80016040 tx=0x7fcf80013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.057+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80013600 con 0x7fcf880620e0 2020-11-20T12:06:27.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7fcf80005950 con 0x7fcf880620e0 2020-11-20T12:06:27.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880620e0 msgr2=0x7fcf88131f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf79ffb700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880620e0 0x7fcf88131f10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcf80016040 tx=0x7fcf80013000).stop 2020-11-20T12:06:27.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf79ffb700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880620e0 msgr2=0x7fcf88131f10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:27.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf79ffb700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf880620e0 0x7fcf88131f10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf5c056d20 msgr2=0x7fcf5c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf5c056d20 0x7fcf5c0591e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fcf8813ced0 tx=0x7fcf84009040).stop 2020-11-20T12:06:27.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880abc30 msgr2=0x7fcf8813cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 --2- 172.21.15.44:0/3027133380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf880abc30 0x7fcf8813cfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fcf7c006e30 tx=0x7fcf7c015000).stop 2020-11-20T12:06:27.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8dc60700 1 -- 172.21.15.44:0/3027133380 reap_dead start 2020-11-20T12:06:27.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8dc60700 1 -- 172.21.15.44:0/3027133380 reap_dead start 2020-11-20T12:06:27.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 shutdown_connections 2020-11-20T12:06:27.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 >> 172.21.15.44:0/3027133380 conn(0x7fcf88065290 msgr2=0x7fcf88137990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:27.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 shutdown_connections 2020-11-20T12:06:27.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.058+0000 7fcf8f6c3700 1 -- 172.21.15.44:0/3027133380 wait complete. 2020-11-20T12:06:27.068 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:27.069 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:27.069 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:27.069 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:06:27.069 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 26, 2020-11-20T12:06:27.070 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:27.070 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:06:27.070 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:27.071 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:27.071 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.071 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:06:27.071 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:27.071 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:27.072 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:27.072 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:27.072 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:27.072 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:27.072 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:27.073 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:27.073 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:27.073 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:27.073 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:27.073 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.074 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:27.074 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:27.074 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:27.074 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:27.075 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:27.075 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:27.075 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:27.075 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:27.075 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:27.076 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:27.076 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.076 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:27.077 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:27.077 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:27.077 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:27.077 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:27.077 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:27.078 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:27.078 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:27.078 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:27.078 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:27.079 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:27.079 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:27.079 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:27.079 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:27.079 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:27.080 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:27.080 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:27.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:27.080 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:27.080 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:27.081 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:27.081 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:27.081 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:27.081 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.081 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:27.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.082 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:27.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.082 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:27.083 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:27.083 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:27.083 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:27.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:27.084 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:27.084 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:27.085 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:27.085 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.085 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.086 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:27.086 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:27.086 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:27.086 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:27.087 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:27.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.087 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:27.088 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:27.088 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:27.088 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:27.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:27.089 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:27.089 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:27.090 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:27.090 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.090 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.090 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.091 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:27.091 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:27.091 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:27.091 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:27.092 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:27.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.092 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:27.093 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:27.093 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:27.093 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:27.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:27.094 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:27.094 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:27.095 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:27.095 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.095 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.096 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.096 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:27.096 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:27.096 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:27.097 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:27.097 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:27.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.098 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.099 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:27.099 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:27.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.099 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:27.099 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:27.100 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:27.100 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.100 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.100 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:27.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.101 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:27.101 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:27.101 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:27.102 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.102 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.103 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:27.103 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:27.105 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/30 file after the snapshot 2020-11-20T12:06:27.105 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/31 file after the snapshot 2020-11-20T12:06:27.105 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/32 file after the snapshot 2020-11-20T12:06:27.105 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/33 file after the snapshot 2020-11-20T12:06:27.106 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/34 file after the snapshot 2020-11-20T12:06:27.106 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/35 file after the snapshot 2020-11-20T12:06:27.106 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/36 file after the snapshot 2020-11-20T12:06:27.106 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/37 file after the snapshot 2020-11-20T12:06:27.106 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/38 file after the snapshot 2020-11-20T12:06:27.107 INFO:tasks.workunit.client.0.smithi044.stdout: create 201/39 file after the snapshot 2020-11-20T12:06:27.107 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/.snap/snap-subdir-test 2020-11-20T12:06:27.107 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:06:27.111 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/30 file after the snapshot 2020-11-20T12:06:27.113 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/31 file after the snapshot 2020-11-20T12:06:27.115 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/32 file after the snapshot 2020-11-20T12:06:27.118 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/33 file after the snapshot 2020-11-20T12:06:27.120 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/34 file after the snapshot 2020-11-20T12:06:27.123 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/35 file after the snapshot 2020-11-20T12:06:27.127 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/36 file after the snapshot 2020-11-20T12:06:27.129 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/37 file after the snapshot 2020-11-20T12:06:27.132 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/38 file after the snapshot 2020-11-20T12:06:27.135 INFO:tasks.workunit.client.0.smithi044.stdout: create 202/39 file after the snapshot 2020-11-20T12:06:27.138 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/.snap/snap-subdir-test 2020-11-20T12:06:27.161 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/30 file after the snapshot 2020-11-20T12:06:27.163 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/31 file after the snapshot 2020-11-20T12:06:27.164 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/32 file after the snapshot 2020-11-20T12:06:27.168 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/33 file after the snapshot 2020-11-20T12:06:27.171 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/34 file after the snapshot 2020-11-20T12:06:27.174 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/35 file after the snapshot 2020-11-20T12:06:27.177 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/36 file after the snapshot 2020-11-20T12:06:27.179 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/37 file after the snapshot 2020-11-20T12:06:27.182 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/38 file after the snapshot 2020-11-20T12:06:27.185 INFO:tasks.workunit.client.0.smithi044.stdout: create 203/39 file after the snapshot 2020-11-20T12:06:27.187 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/.snap/snap-subdir-test 2020-11-20T12:06:27.209 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/30 file after the snapshot 2020-11-20T12:06:27.211 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/31 file after the snapshot 2020-11-20T12:06:27.213 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/32 file after the snapshot 2020-11-20T12:06:27.216 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/33 file after the snapshot 2020-11-20T12:06:27.220 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/34 file after the snapshot 2020-11-20T12:06:27.222 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/35 file after the snapshot 2020-11-20T12:06:27.225 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/36 file after the snapshot 2020-11-20T12:06:27.227 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/37 file after the snapshot 2020-11-20T12:06:27.230 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/38 file after the snapshot 2020-11-20T12:06:27.233 INFO:tasks.workunit.client.0.smithi044.stdout: create 204/39 file after the snapshot 2020-11-20T12:06:27.236 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/.snap/snap-subdir-test 2020-11-20T12:06:27.259 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/30 file after the snapshot 2020-11-20T12:06:27.261 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/31 file after the snapshot 2020-11-20T12:06:27.264 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/32 file after the snapshot 2020-11-20T12:06:27.266 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/33 file after the snapshot 2020-11-20T12:06:27.269 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/34 file after the snapshot 2020-11-20T12:06:27.273 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/35 file after the snapshot 2020-11-20T12:06:27.275 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/36 file after the snapshot 2020-11-20T12:06:27.277 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/37 file after the snapshot 2020-11-20T12:06:27.279 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/38 file after the snapshot 2020-11-20T12:06:27.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.277+0000 7f5f76c14700 1 Processor -- start 2020-11-20T12:06:27.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.277+0000 7f5f76c14700 1 -- start start 2020-11-20T12:06:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f76c14700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f700a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f76c14700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5f700a86d0 0x7f5f7013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f76c14700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f701385f0 con 0x7f5f700ab0e0 2020-11-20T12:06:27.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f76c14700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7013e250 con 0x7f5f700a86d0 2020-11-20T12:06:27.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f76c14700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7013e4f0 con 0x7f5f700aad00 2020-11-20T12:06:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f749b0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f700a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f749b0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f700a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49458/0 (socket says 172.21.15.44:49458) 2020-11-20T12:06:27.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f749b0700 1 -- 172.21.15.44:0/1884629794 learned_addr learned my addr 172.21.15.44:0/1884629794 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.278+0000 7f5f751b1700 1 --2- 172.21.15.44:0/1884629794 >> v2:172.21.15.79:3300/0 conn(0x7f5f700a86d0 0x7f5f7013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f749b0700 1 -- 172.21.15.44:0/1884629794 >> v2:172.21.15.79:3300/0 conn(0x7f5f700a86d0 msgr2=0x7f5f7013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f749b0700 1 --2- 172.21.15.44:0/1884629794 >> v2:172.21.15.79:3300/0 conn(0x7f5f700a86d0 0x7f5f7013ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f749b0700 1 -- 172.21.15.44:0/1884629794 >> v1:172.21.15.44:6789/0 conn(0x7f5f700ab0e0 legacy=0x7f5f700abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f749b0700 1 -- 172.21.15.44:0/1884629794 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700ac340 con 0x7f5f700aad00 2020-11-20T12:06:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f749b0700 1 --2- 172.21.15.44:0/1884629794 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f700a94d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f5f64006fd0 tx=0x7f5f64006eb0).ready entity=mon.2 client_cookie=e47013f4629f5cfc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f6f7fe700 1 -- 172.21.15.44:0/1884629794 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f640098a0 con 0x7f5f700aad00 2020-11-20T12:06:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f6f7fe700 1 -- 172.21.15.44:0/1884629794 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64009a00 con 0x7f5f700aad00 2020-11-20T12:06:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.279+0000 7f5f6f7fe700 1 -- 172.21.15.44:0/1884629794 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64009d80 con 0x7f5f700aad00 2020-11-20T12:06:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.280+0000 7f5f76c14700 1 -- 172.21.15.44:0/1884629794 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 msgr2=0x7f5f700a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.280+0000 7f5f76c14700 1 --2- 172.21.15.44:0/1884629794 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f700a94d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f5f64006fd0 tx=0x7f5f64006eb0).stop 2020-11-20T12:06:27.290 INFO:tasks.workunit.client.0.smithi044.stdout: create 205/39 file after the snapshot 2020-11-20T12:06:27.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.281+0000 7f5f76c14700 1 -- 172.21.15.44:0/1884629794 shutdown_connections 2020-11-20T12:06:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.281+0000 7f5f76c14700 1 --2- 172.21.15.44:0/1884629794 >> v2:172.21.15.79:3300/0 conn(0x7f5f700a86d0 0x7f5f7013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.281+0000 7f5f76c14700 1 --2- 172.21.15.44:0/1884629794 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f700a94d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.281+0000 7f5f76c14700 1 -- 172.21.15.44:0/1884629794 >> 172.21.15.44:0/1884629794 conn(0x7f5f70060170 msgr2=0x7f5f70060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.281+0000 7f5f76c14700 1 -- 172.21.15.44:0/1884629794 shutdown_connections 2020-11-20T12:06:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.281+0000 7f5f76c14700 1 -- 172.21.15.44:0/1884629794 wait complete. 2020-11-20T12:06:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.283+0000 7f5f76c14700 1 Processor -- start 2020-11-20T12:06:27.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.284+0000 7f5f76c14700 1 -- start start 2020-11-20T12:06:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f76c14700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f700a86d0 0x7f5f700ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f76c14700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f7009d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f76c14700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f700ab0e0 0x7f5f7009e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f76c14700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700a0260 con 0x7f5f700ab0e0 2020-11-20T12:06:27.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f76c14700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700a0500 con 0x7f5f700a86d0 2020-11-20T12:06:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f76c14700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700a07a0 con 0x7f5f700aad00 2020-11-20T12:06:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f700a86d0 0x7f5f700ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f700a86d0 0x7f5f700ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58308/0 (socket says 172.21.15.44:58308) 2020-11-20T12:06:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 -- 172.21.15.44:0/353636684 learned_addr learned my addr 172.21.15.44:0/353636684 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f7009d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 msgr2=0x7f5f7009d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f7009d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f700ab0e0 msgr2=0x7f5f7009e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f700ab0e0 0x7f5f7009e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:27.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 -- 172.21.15.44:0/353636684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7009ff90 con 0x7f5f700a86d0 2020-11-20T12:06:27.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f6ffff700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700aad00 0x7f5f7009d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:27.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.285+0000 7f5f749b0700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f700a86d0 0x7f5f700ad8a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5f700b2300 tx=0x7f5f64015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.286+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64011dd0 con 0x7f5f700a86d0 2020-11-20T12:06:27.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.286+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6401e460 con 0x7f5f700a86d0 2020-11-20T12:06:27.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.286+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6401e800 con 0x7f5f700a86d0 2020-11-20T12:06:27.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.286+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7009d9d0 con 0x7f5f700a86d0 2020-11-20T12:06:27.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.286+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7009e800 con 0x7f5f700a86d0 2020-11-20T12:06:27.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.287+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f5f64025030 con 0x7f5f700a86d0 2020-11-20T12:06:27.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.287+0000 7f5f76c14700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a3d80 0x7f5f700a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.287+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700a0120 con 0x7f5f700a3d80 2020-11-20T12:06:27.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.287+0000 7f5f6ffff700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a3d80 0x7f5f700a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.287+0000 7f5f6cff9700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f48056d20 0x7f5f480591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f5f64071870 con 0x7f5f700a86d0 2020-11-20T12:06:27.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6ffff700 1 -- 172.21.15.44:0/353636684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f7009e070 con 0x7f5f700a3d80 2020-11-20T12:06:27.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f751b1700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f48056d20 0x7f5f480591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6ffff700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a3d80 0x7f5f700a4160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f5f7009fd30 tx=0x7f5f5c009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c007240 con 0x7f5f700a3d80 2020-11-20T12:06:27.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f751b1700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f48056d20 0x7f5f480591e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5f68009c00 tx=0x7f5f680091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f5f5c0073a0 con 0x7f5f700a3d80 2020-11-20T12:06:27.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a3d80 msgr2=0x7f5f700a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6cff9700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a3d80 0x7f5f700a4160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f5f7009fd30 tx=0x7f5f5c009040).stop 2020-11-20T12:06:27.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a3d80 msgr2=0x7f5f700a4160 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:27.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.288+0000 7f5f6cff9700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a3d80 0x7f5f700a4160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.309 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/.snap/snap-subdir-test 2020-11-20T12:06:27.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.297+0000 7f5f76c14700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a6070 0x7f5f700a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.297+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7005c5f0 con 0x7f5f700a6070 2020-11-20T12:06:27.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.297+0000 7f5f6ffff700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a6070 0x7f5f700a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.297+0000 7f5f6ffff700 1 -- 172.21.15.44:0/353636684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5f70069680 con 0x7f5f700a6070 2020-11-20T12:06:27.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.297+0000 7f5f6ffff700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a6070 0x7f5f700a6480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5f5c007fd0 tx=0x7f5f5c018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c025410 con 0x7f5f700a6070 2020-11-20T12:06:27.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f5f5c007030 con 0x7f5f700a6070 2020-11-20T12:06:27.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a6070 msgr2=0x7f5f700a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f6cff9700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a6070 0x7f5f700a6480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5f5c007fd0 tx=0x7f5f5c018000).stop 2020-11-20T12:06:27.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f6cff9700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a6070 msgr2=0x7f5f700a6480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:27.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f6cff9700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5f700a6070 0x7f5f700a6480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f48056d20 msgr2=0x7f5f480591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5f48056d20 0x7f5f480591e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5f68009c00 tx=0x7f5f680091d0).stop 2020-11-20T12:06:27.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f700a86d0 msgr2=0x7f5f700ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 --2- 172.21.15.44:0/353636684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f700a86d0 0x7f5f700ad8a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5f700b2300 tx=0x7f5f64015000).stop 2020-11-20T12:06:27.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f751b1700 1 -- 172.21.15.44:0/353636684 reap_dead start 2020-11-20T12:06:27.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f751b1700 1 -- 172.21.15.44:0/353636684 reap_dead start 2020-11-20T12:06:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 shutdown_connections 2020-11-20T12:06:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 >> 172.21.15.44:0/353636684 conn(0x7f5f70060170 msgr2=0x7f5f70060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 shutdown_connections 2020-11-20T12:06:27.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.298+0000 7f5f76c14700 1 -- 172.21.15.44:0/353636684 wait complete. 2020-11-20T12:06:27.317 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:27.317 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:27.317 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:27.317 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:06:27.317 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 26, 2020-11-20T12:06:27.318 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:27.318 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:06:27.318 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:27.318 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:27.319 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.319 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 16, 2020-11-20T12:06:27.319 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:27.319 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:27.319 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:27.320 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:27.320 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:27.320 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:27.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:27.320 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:27.321 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:27.321 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:27.321 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:27.321 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.322 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:27.322 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:27.322 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:27.322 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:27.322 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:27.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:27.323 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:27.323 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:27.323 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:27.324 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:27.324 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.324 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:27.325 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:27.325 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:27.325 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:27.325 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:27.325 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:27.326 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:27.326 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:27.326 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:27.326 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:27.326 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:27.327 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:27.327 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:27.327 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:27.327 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:27.328 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:27.328 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:27.328 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:27.328 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:27.328 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:27.329 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:27.329 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:27.329 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:27.329 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.329 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:27.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.330 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:27.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.330 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:27.330 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:27.331 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:27.331 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:27.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:27.331 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:27.332 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:27.333 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:27.333 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.333 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.334 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:27.334 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:27.334 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:27.334 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:27.334 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:27.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.335 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:27.335 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:27.335 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:27.336 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:27.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:27.336 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:27.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:27.337 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:27.337 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.338 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.338 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:27.339 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:27.339 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:27.339 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:27.339 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:27.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.340 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:27.340 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:27.341 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:27.341 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:27.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:27.341 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:27.342 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.342 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:27.342 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:27.343 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:27.343 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.343 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.344 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:27.344 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:27.344 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:27.344 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:27.345 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:27.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.345 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.346 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:27.346 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:27.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.347 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:27.347 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:27.347 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:27.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.347 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.348 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:06:27.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.348 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:27.348 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:27.348 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:06:27.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.349 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:27.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.349 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:27.350 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:27.350 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:06:27.350 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.350 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:27.350 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:06:27.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:27.351 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:27.351 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:27.351 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:27.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:27.352 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:27.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:27.352 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:27.352 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:27.354 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:06:27.355 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:06:27.355 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:06:27.355 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:06:27.356 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:06:27.356 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:06:27.356 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/30 file after the snapshot 2020-11-20T12:06:27.356 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/31 file after the snapshot 2020-11-20T12:06:27.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/32 file after the snapshot 2020-11-20T12:06:27.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/33 file after the snapshot 2020-11-20T12:06:27.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/34 file after the snapshot 2020-11-20T12:06:27.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/35 file after the snapshot 2020-11-20T12:06:27.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/36 file after the snapshot 2020-11-20T12:06:27.358 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/37 file after the snapshot 2020-11-20T12:06:27.358 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/38 file after the snapshot 2020-11-20T12:06:27.358 INFO:tasks.workunit.client.0.smithi044.stdout: create 206/39 file after the snapshot 2020-11-20T12:06:27.358 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/.snap/snap-subdir-test 2020-11-20T12:06:27.359 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/30 file after the snapshot 2020-11-20T12:06:27.361 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/31 file after the snapshot 2020-11-20T12:06:27.364 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/32 file after the snapshot 2020-11-20T12:06:27.368 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/33 file after the snapshot 2020-11-20T12:06:27.371 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/34 file after the snapshot 2020-11-20T12:06:27.373 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:06:27.373 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:06:27.374 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:06:27.374 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/35 file after the snapshot 2020-11-20T12:06:27.377 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/36 file after the snapshot 2020-11-20T12:06:27.380 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/37 file after the snapshot 2020-11-20T12:06:27.382 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/38 file after the snapshot 2020-11-20T12:06:27.384 INFO:tasks.workunit.client.0.smithi044.stdout: create 207/39 file after the snapshot 2020-11-20T12:06:27.386 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/.snap/snap-subdir-test 2020-11-20T12:06:27.410 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/30 file after the snapshot 2020-11-20T12:06:27.411 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/31 file after the snapshot 2020-11-20T12:06:27.414 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/32 file after the snapshot 2020-11-20T12:06:27.417 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/33 file after the snapshot 2020-11-20T12:06:27.420 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/34 file after the snapshot 2020-11-20T12:06:27.422 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/35 file after the snapshot 2020-11-20T12:06:27.424 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/36 file after the snapshot 2020-11-20T12:06:27.427 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/37 file after the snapshot 2020-11-20T12:06:27.431 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/38 file after the snapshot 2020-11-20T12:06:27.433 INFO:tasks.workunit.client.0.smithi044.stdout: create 208/39 file after the snapshot 2020-11-20T12:06:27.436 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/.snap/snap-subdir-test 2020-11-20T12:06:27.455 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/30 file after the snapshot 2020-11-20T12:06:27.457 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/31 file after the snapshot 2020-11-20T12:06:27.459 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/32 file after the snapshot 2020-11-20T12:06:27.461 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/33 file after the snapshot 2020-11-20T12:06:27.463 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/34 file after the snapshot 2020-11-20T12:06:27.466 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/35 file after the snapshot 2020-11-20T12:06:27.468 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/36 file after the snapshot 2020-11-20T12:06:27.470 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/37 file after the snapshot 2020-11-20T12:06:27.473 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/38 file after the snapshot 2020-11-20T12:06:27.475 INFO:tasks.workunit.client.0.smithi044.stdout: create 209/39 file after the snapshot 2020-11-20T12:06:27.478 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/.snap/snap-subdir-test 2020-11-20T12:06:27.498 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/30 file after the snapshot 2020-11-20T12:06:27.500 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/31 file after the snapshot 2020-11-20T12:06:27.502 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/32 file after the snapshot 2020-11-20T12:06:27.507 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/33 file after the snapshot 2020-11-20T12:06:27.509 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/34 file after the snapshot 2020-11-20T12:06:27.512 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/35 file after the snapshot 2020-11-20T12:06:27.515 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/36 file after the snapshot 2020-11-20T12:06:27.518 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/37 file after the snapshot 2020-11-20T12:06:27.522 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/38 file after the snapshot 2020-11-20T12:06:27.525 INFO:tasks.workunit.client.0.smithi044.stdout: create 210/39 file after the snapshot 2020-11-20T12:06:27.527 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/.snap/snap-subdir-test 2020-11-20T12:06:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.545+0000 7f2820cb6700 1 Processor -- start 2020-11-20T12:06:27.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.545+0000 7f2820cb6700 1 -- start start 2020-11-20T12:06:27.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f2820cb6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f281c13a990 0x7f281c13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f2820cb6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f2820cb6700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c0a6260 con 0x7f281c060330 2020-11-20T12:06:27.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f2820cb6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c0a52e0 con 0x7f281c13a990 2020-11-20T12:06:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f2820cb6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0a6490 con 0x7f281c13f800 2020-11-20T12:06:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281a59c700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f281c060330 legacy=0x7f281c0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:06:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281ad9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f2813fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f281c13a990 0x7f281c13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281ad9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49510/0 (socket says 172.21.15.44:49510) 2020-11-20T12:06:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281ad9d700 1 -- 172.21.15.44:0/1464795590 learned_addr learned my addr 172.21.15.44:0/1464795590 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:27.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281ad9d700 1 -- 172.21.15.44:0/1464795590 >> v2:172.21.15.79:3300/0 conn(0x7f281c13a990 msgr2=0x7f281c13ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281ad9d700 1 --2- 172.21.15.44:0/1464795590 >> v2:172.21.15.79:3300/0 conn(0x7f281c13a990 0x7f281c13ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281ad9d700 1 -- 172.21.15.44:0/1464795590 >> v1:172.21.15.44:6789/0 conn(0x7f281c060330 legacy=0x7f281c0a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f281ad9d700 1 -- 172.21.15.44:0/1464795590 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0a6730 con 0x7f281c13f800 2020-11-20T12:06:27.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.546+0000 7f2813fff700 1 --2- 172.21.15.44:0/1464795590 >> v2:172.21.15.79:3300/0 conn(0x7f281c13a990 0x7f281c13ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f281ad9d700 1 --2- 172.21.15.44:0/1464795590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c141bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2814009d50 tx=0x7f2814009290).ready entity=mon.2 client_cookie=3ab60beaa0b97274 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2819d9b700 1 -- 172.21.15.44:0/1464795590 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400bb80 con 0x7f281c13f800 2020-11-20T12:06:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2819d9b700 1 -- 172.21.15.44:0/1464795590 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281400bce0 con 0x7f281c13f800 2020-11-20T12:06:27.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2819d9b700 1 -- 172.21.15.44:0/1464795590 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814011610 con 0x7f281c13f800 2020-11-20T12:06:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 -- 172.21.15.44:0/1464795590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 msgr2=0x7f281c141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 --2- 172.21.15.44:0/1464795590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c141bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2814009d50 tx=0x7f2814009290).stop 2020-11-20T12:06:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 -- 172.21.15.44:0/1464795590 shutdown_connections 2020-11-20T12:06:27.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 --2- 172.21.15.44:0/1464795590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c141bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 --2- 172.21.15.44:0/1464795590 >> v2:172.21.15.79:3300/0 conn(0x7f281c13a990 0x7f281c13ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 -- 172.21.15.44:0/1464795590 >> 172.21.15.44:0/1464795590 conn(0x7f281c062690 msgr2=0x7f281c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 -- 172.21.15.44:0/1464795590 shutdown_connections 2020-11-20T12:06:27.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.547+0000 7f2820cb6700 1 -- 172.21.15.44:0/1464795590 wait complete. 2020-11-20T12:06:27.559 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/30 file after the snapshot 2020-11-20T12:06:27.559 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/31 file after the snapshot 2020-11-20T12:06:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.548+0000 7f2820cb6700 1 Processor -- start 2020-11-20T12:06:27.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.549+0000 7f2820cb6700 1 -- start start 2020-11-20T12:06:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.549+0000 7f2820cb6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281c060330 0x7f281c14a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.549+0000 7f2820cb6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281c13a990 0x7f281c136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.549+0000 7f2820cb6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c1310c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.549+0000 7f2820cb6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c137190 con 0x7f281c13a990 2020-11-20T12:06:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.549+0000 7f2820cb6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c14a000 con 0x7f281c060330 2020-11-20T12:06:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.549+0000 7f2820cb6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c149cf0 con 0x7f281c13f800 2020-11-20T12:06:27.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f2813fff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281c13a990 msgr2=0x7f281c136760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:27.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f2813fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281c13a990 0x7f281c136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:27.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c1310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c1310c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49514/0 (socket says 172.21.15.44:49514) 2020-11-20T12:06:27.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 -- 172.21.15.44:0/2621225303 learned_addr learned my addr 172.21.15.44:0/2621225303 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:27.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281a59c700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281c060330 0x7f281c14a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 -- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281c060330 msgr2=0x7f281c14a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281c060330 0x7f281c14a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 -- 172.21.15.44:0/2621225303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281c13a990 msgr2=0x7f281c136760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:27.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281c13a990 0x7f281c136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:27.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281ad9d700 1 -- 172.21.15.44:0/2621225303 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0ac360 con 0x7f281c13f800 2020-11-20T12:06:27.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.550+0000 7f281a59c700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281c060330 0x7f281c14a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:06:27.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.551+0000 7f281ad9d700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c1310c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f28140092d0 tx=0x7f2814016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.551+0000 7f28127fc700 1 -- 172.21.15.44:0/2621225303 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814011650 con 0x7f281c13f800 2020-11-20T12:06:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.551+0000 7f28127fc700 1 -- 172.21.15.44:0/2621225303 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281400be50 con 0x7f281c13f800 2020-11-20T12:06:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.551+0000 7f28127fc700 1 -- 172.21.15.44:0/2621225303 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814010e30 con 0x7f281c13f800 2020-11-20T12:06:27.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.551+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c137b90 con 0x7f281c13f800 2020-11-20T12:06:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.551+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c14abd0 con 0x7f281c13f800 2020-11-20T12:06:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.552+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c061650 con 0x7f281c13f800 2020-11-20T12:06:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.552+0000 7f28127fc700 1 -- 172.21.15.44:0/2621225303 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2814023030 con 0x7f281c13f800 2020-11-20T12:06:27.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.553+0000 7f28127fc700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27f8056cd0 0x7f27f8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:27.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.553+0000 7f28127fc700 1 -- 172.21.15.44:0/2621225303 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f28140716f0 con 0x7f281c13f800 2020-11-20T12:06:27.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.553+0000 7f281a59c700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27f8056cd0 0x7f27f8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:27.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.554+0000 7f281a59c700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27f8056cd0 0x7f27f8059190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f280c006fd0 tx=0x7f280c006d30).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:27.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.557+0000 7f28127fc700 1 -- 172.21.15.44:0/2621225303 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f28140469c0 con 0x7f281c13f800 2020-11-20T12:06:27.571 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/32 file after the snapshot 2020-11-20T12:06:27.571 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/33 file after the snapshot 2020-11-20T12:06:27.571 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/34 file after the snapshot 2020-11-20T12:06:27.572 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/35 file after the snapshot 2020-11-20T12:06:27.573 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/36 file after the snapshot 2020-11-20T12:06:27.573 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/37 file after the snapshot 2020-11-20T12:06:27.573 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/38 file after the snapshot 2020-11-20T12:06:27.575 INFO:tasks.workunit.client.0.smithi044.stdout: create 211/39 file after the snapshot 2020-11-20T12:06:27.578 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/.snap/snap-subdir-test 2020-11-20T12:06:27.600 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/30 file after the snapshot 2020-11-20T12:06:27.603 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/31 file after the snapshot 2020-11-20T12:06:27.605 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/32 file after the snapshot 2020-11-20T12:06:27.607 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/33 file after the snapshot 2020-11-20T12:06:27.610 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/34 file after the snapshot 2020-11-20T12:06:27.613 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/35 file after the snapshot 2020-11-20T12:06:27.616 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/36 file after the snapshot 2020-11-20T12:06:27.620 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/37 file after the snapshot 2020-11-20T12:06:27.622 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/38 file after the snapshot 2020-11-20T12:06:27.626 INFO:tasks.workunit.client.0.smithi044.stdout: create 212/39 file after the snapshot 2020-11-20T12:06:27.627 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/.snap/snap-subdir-test 2020-11-20T12:06:27.654 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/30 file after the snapshot 2020-11-20T12:06:27.657 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/31 file after the snapshot 2020-11-20T12:06:27.659 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/32 file after the snapshot 2020-11-20T12:06:27.661 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/33 file after the snapshot 2020-11-20T12:06:27.663 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/34 file after the snapshot 2020-11-20T12:06:27.666 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/35 file after the snapshot 2020-11-20T12:06:27.667 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/36 file after the snapshot 2020-11-20T12:06:27.670 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/37 file after the snapshot 2020-11-20T12:06:27.672 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/38 file after the snapshot 2020-11-20T12:06:27.674 INFO:tasks.workunit.client.0.smithi044.stdout: create 213/39 file after the snapshot 2020-11-20T12:06:27.677 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/.snap/snap-subdir-test 2020-11-20T12:06:27.699 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/30 file after the snapshot 2020-11-20T12:06:27.700 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/31 file after the snapshot 2020-11-20T12:06:27.704 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/32 file after the snapshot 2020-11-20T12:06:27.707 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/33 file after the snapshot 2020-11-20T12:06:27.710 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/34 file after the snapshot 2020-11-20T12:06:27.713 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/35 file after the snapshot 2020-11-20T12:06:27.715 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/36 file after the snapshot 2020-11-20T12:06:27.717 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/37 file after the snapshot 2020-11-20T12:06:27.721 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/38 file after the snapshot 2020-11-20T12:06:27.723 INFO:tasks.workunit.client.0.smithi044.stdout: create 214/39 file after the snapshot 2020-11-20T12:06:27.725 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/.snap/snap-subdir-test 2020-11-20T12:06:27.753 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/30 file after the snapshot 2020-11-20T12:06:27.754 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/31 file after the snapshot 2020-11-20T12:06:27.756 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/32 file after the snapshot 2020-11-20T12:06:27.760 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/33 file after the snapshot 2020-11-20T12:06:27.763 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/34 file after the snapshot 2020-11-20T12:06:27.767 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/35 file after the snapshot 2020-11-20T12:06:27.769 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/36 file after the snapshot 2020-11-20T12:06:27.772 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/37 file after the snapshot 2020-11-20T12:06:27.774 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/38 file after the snapshot 2020-11-20T12:06:27.776 INFO:tasks.workunit.client.0.smithi044.stdout: create 215/39 file after the snapshot 2020-11-20T12:06:27.779 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/.snap/snap-subdir-test 2020-11-20T12:06:27.802 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/30 file after the snapshot 2020-11-20T12:06:27.804 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/31 file after the snapshot 2020-11-20T12:06:27.806 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/32 file after the snapshot 2020-11-20T12:06:27.809 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/33 file after the snapshot 2020-11-20T12:06:27.811 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/34 file after the snapshot 2020-11-20T12:06:27.813 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/35 file after the snapshot 2020-11-20T12:06:27.815 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/36 file after the snapshot 2020-11-20T12:06:27.817 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/37 file after the snapshot 2020-11-20T12:06:27.821 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/38 file after the snapshot 2020-11-20T12:06:27.825 INFO:tasks.workunit.client.0.smithi044.stdout: create 216/39 file after the snapshot 2020-11-20T12:06:27.827 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/.snap/snap-subdir-test 2020-11-20T12:06:27.850 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/30 file after the snapshot 2020-11-20T12:06:27.851 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/31 file after the snapshot 2020-11-20T12:06:27.853 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/32 file after the snapshot 2020-11-20T12:06:27.855 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/33 file after the snapshot 2020-11-20T12:06:27.858 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/34 file after the snapshot 2020-11-20T12:06:27.860 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/35 file after the snapshot 2020-11-20T12:06:27.862 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/36 file after the snapshot 2020-11-20T12:06:27.864 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/37 file after the snapshot 2020-11-20T12:06:27.869 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/38 file after the snapshot 2020-11-20T12:06:27.870 INFO:tasks.workunit.client.0.smithi044.stdout: create 217/39 file after the snapshot 2020-11-20T12:06:27.873 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/.snap/snap-subdir-test 2020-11-20T12:06:27.906 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/30 file after the snapshot 2020-11-20T12:06:27.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.904+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f281c14a1c0 con 0x7f281c13f800 2020-11-20T12:06:27.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.905+0000 7f28127fc700 1 -- 172.21.15.44:0/2621225303 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f2814021080 con 0x7f281c13f800 2020-11-20T12:06:27.909 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:06:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27f8056cd0 msgr2=0x7f27f8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27f8056cd0 0x7f27f8059190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f280c006fd0 tx=0x7f280c006d30).stop 2020-11-20T12:06:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 msgr2=0x7f281c1310c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c1310c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f28140092d0 tx=0x7f2814016000).stop 2020-11-20T12:06:27.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 shutdown_connections 2020-11-20T12:06:27.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27f8056cd0 0x7f27f8059190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f281c13f800 0x7f281c1310c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f281c060330 0x7f281c14a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:27.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 --2- 172.21.15.44:0/2621225303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f281c13a990 0x7f281c136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:27.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 >> 172.21.15.44:0/2621225303 conn(0x7f281c062690 msgr2=0x7f281c0a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:27.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 shutdown_connections 2020-11-20T12:06:27.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:27.907+0000 7f2820cb6700 1 -- 172.21.15.44:0/2621225303 wait complete. 2020-11-20T12:06:27.974 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/31 file after the snapshot 2020-11-20T12:06:27.976 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/32 file after the snapshot 2020-11-20T12:06:27.978 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/33 file after the snapshot 2020-11-20T12:06:27.980 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/34 file after the snapshot 2020-11-20T12:06:27.982 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/35 file after the snapshot 2020-11-20T12:06:27.985 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/36 file after the snapshot 2020-11-20T12:06:27.988 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/37 file after the snapshot 2020-11-20T12:06:27.990 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/38 file after the snapshot 2020-11-20T12:06:27.992 INFO:tasks.workunit.client.0.smithi044.stdout: create 218/39 file after the snapshot 2020-11-20T12:06:27.994 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/.snap/snap-subdir-test 2020-11-20T12:06:28.050 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/30 file after the snapshot 2020-11-20T12:06:28.051 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/31 file after the snapshot 2020-11-20T12:06:28.053 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/32 file after the snapshot 2020-11-20T12:06:28.055 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/33 file after the snapshot 2020-11-20T12:06:28.058 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/34 file after the snapshot 2020-11-20T12:06:28.061 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/35 file after the snapshot 2020-11-20T12:06:28.062 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/36 file after the snapshot 2020-11-20T12:06:28.064 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/37 file after the snapshot 2020-11-20T12:06:28.066 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/38 file after the snapshot 2020-11-20T12:06:28.069 INFO:tasks.workunit.client.0.smithi044.stdout: create 219/39 file after the snapshot 2020-11-20T12:06:28.072 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/.snap/snap-subdir-test 2020-11-20T12:06:28.099 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/30 file after the snapshot 2020-11-20T12:06:28.101 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/31 file after the snapshot 2020-11-20T12:06:28.103 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/32 file after the snapshot 2020-11-20T12:06:28.105 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/33 file after the snapshot 2020-11-20T12:06:28.107 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/34 file after the snapshot 2020-11-20T12:06:28.110 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/35 file after the snapshot 2020-11-20T12:06:28.112 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/36 file after the snapshot 2020-11-20T12:06:28.114 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/37 file after the snapshot 2020-11-20T12:06:28.116 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/38 file after the snapshot 2020-11-20T12:06:28.119 INFO:tasks.workunit.client.0.smithi044.stdout: create 220/39 file after the snapshot 2020-11-20T12:06:28.121 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/.snap/snap-subdir-test 2020-11-20T12:06:28.142 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/30 file after the snapshot 2020-11-20T12:06:28.144 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/31 file after the snapshot 2020-11-20T12:06:28.146 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/32 file after the snapshot 2020-11-20T12:06:28.149 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/33 file after the snapshot 2020-11-20T12:06:28.152 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/34 file after the snapshot 2020-11-20T12:06:28.155 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/35 file after the snapshot 2020-11-20T12:06:28.157 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/36 file after the snapshot 2020-11-20T12:06:28.160 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/37 file after the snapshot 2020-11-20T12:06:28.162 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/38 file after the snapshot 2020-11-20T12:06:28.165 INFO:tasks.workunit.client.0.smithi044.stdout: create 221/39 file after the snapshot 2020-11-20T12:06:28.167 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/.snap/snap-subdir-test 2020-11-20T12:06:28.187 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/30 file after the snapshot 2020-11-20T12:06:28.189 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/31 file after the snapshot 2020-11-20T12:06:28.191 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/32 file after the snapshot 2020-11-20T12:06:28.194 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/33 file after the snapshot 2020-11-20T12:06:28.196 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/34 file after the snapshot 2020-11-20T12:06:28.198 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/35 file after the snapshot 2020-11-20T12:06:28.200 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/36 file after the snapshot 2020-11-20T12:06:28.203 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/37 file after the snapshot 2020-11-20T12:06:28.205 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/38 file after the snapshot 2020-11-20T12:06:28.207 INFO:tasks.workunit.client.0.smithi044.stdout: create 222/39 file after the snapshot 2020-11-20T12:06:28.209 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/.snap/snap-subdir-test 2020-11-20T12:06:28.231 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/30 file after the snapshot 2020-11-20T12:06:28.232 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/31 file after the snapshot 2020-11-20T12:06:28.234 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/32 file after the snapshot 2020-11-20T12:06:28.237 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/33 file after the snapshot 2020-11-20T12:06:28.240 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/34 file after the snapshot 2020-11-20T12:06:28.242 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/35 file after the snapshot 2020-11-20T12:06:28.246 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/36 file after the snapshot 2020-11-20T12:06:28.246 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/37 file after the snapshot 2020-11-20T12:06:28.248 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/38 file after the snapshot 2020-11-20T12:06:28.250 INFO:tasks.workunit.client.0.smithi044.stdout: create 223/39 file after the snapshot 2020-11-20T12:06:28.252 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/.snap/snap-subdir-test 2020-11-20T12:06:28.277 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/30 file after the snapshot 2020-11-20T12:06:28.278 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/31 file after the snapshot 2020-11-20T12:06:28.280 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/32 file after the snapshot 2020-11-20T12:06:28.281 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/33 file after the snapshot 2020-11-20T12:06:28.284 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/34 file after the snapshot 2020-11-20T12:06:28.287 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/35 file after the snapshot 2020-11-20T12:06:28.289 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/36 file after the snapshot 2020-11-20T12:06:28.291 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/37 file after the snapshot 2020-11-20T12:06:28.293 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/38 file after the snapshot 2020-11-20T12:06:28.295 INFO:tasks.workunit.client.0.smithi044.stdout: create 224/39 file after the snapshot 2020-11-20T12:06:28.297 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/.snap/snap-subdir-test 2020-11-20T12:06:28.326 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/30 file after the snapshot 2020-11-20T12:06:28.327 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/31 file after the snapshot 2020-11-20T12:06:28.329 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/32 file after the snapshot 2020-11-20T12:06:28.331 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/33 file after the snapshot 2020-11-20T12:06:28.333 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/34 file after the snapshot 2020-11-20T12:06:28.335 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/35 file after the snapshot 2020-11-20T12:06:28.337 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/36 file after the snapshot 2020-11-20T12:06:28.339 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/37 file after the snapshot 2020-11-20T12:06:28.341 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/38 file after the snapshot 2020-11-20T12:06:28.343 INFO:tasks.workunit.client.0.smithi044.stdout: create 225/39 file after the snapshot 2020-11-20T12:06:28.345 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/.snap/snap-subdir-test 2020-11-20T12:06:28.368 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/30 file after the snapshot 2020-11-20T12:06:28.370 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/31 file after the snapshot 2020-11-20T12:06:28.373 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/32 file after the snapshot 2020-11-20T12:06:28.374 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/33 file after the snapshot 2020-11-20T12:06:28.377 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/34 file after the snapshot 2020-11-20T12:06:28.379 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/35 file after the snapshot 2020-11-20T12:06:28.381 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/36 file after the snapshot 2020-11-20T12:06:28.383 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/37 file after the snapshot 2020-11-20T12:06:28.385 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/38 file after the snapshot 2020-11-20T12:06:28.388 INFO:tasks.workunit.client.0.smithi044.stdout: create 226/39 file after the snapshot 2020-11-20T12:06:28.390 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/.snap/snap-subdir-test 2020-11-20T12:06:28.410 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/30 file after the snapshot 2020-11-20T12:06:28.412 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/31 file after the snapshot 2020-11-20T12:06:28.414 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/32 file after the snapshot 2020-11-20T12:06:28.418 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/33 file after the snapshot 2020-11-20T12:06:28.419 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/34 file after the snapshot 2020-11-20T12:06:28.422 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/35 file after the snapshot 2020-11-20T12:06:28.423 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/36 file after the snapshot 2020-11-20T12:06:28.425 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/37 file after the snapshot 2020-11-20T12:06:28.427 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/38 file after the snapshot 2020-11-20T12:06:28.429 INFO:tasks.workunit.client.0.smithi044.stdout: create 227/39 file after the snapshot 2020-11-20T12:06:28.432 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/.snap/snap-subdir-test 2020-11-20T12:06:28.453 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/30 file after the snapshot 2020-11-20T12:06:28.454 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/31 file after the snapshot 2020-11-20T12:06:28.456 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/32 file after the snapshot 2020-11-20T12:06:28.458 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/33 file after the snapshot 2020-11-20T12:06:28.460 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/34 file after the snapshot 2020-11-20T12:06:28.463 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/35 file after the snapshot 2020-11-20T12:06:28.467 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/36 file after the snapshot 2020-11-20T12:06:28.469 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/37 file after the snapshot 2020-11-20T12:06:28.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/38 file after the snapshot 2020-11-20T12:06:28.473 INFO:tasks.workunit.client.0.smithi044.stdout: create 228/39 file after the snapshot 2020-11-20T12:06:28.476 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/.snap/snap-subdir-test 2020-11-20T12:06:28.497 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/30 file after the snapshot 2020-11-20T12:06:28.498 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/31 file after the snapshot 2020-11-20T12:06:28.500 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/32 file after the snapshot 2020-11-20T12:06:28.504 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/33 file after the snapshot 2020-11-20T12:06:28.505 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/34 file after the snapshot 2020-11-20T12:06:28.507 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/35 file after the snapshot 2020-11-20T12:06:28.509 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/36 file after the snapshot 2020-11-20T12:06:28.512 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/37 file after the snapshot 2020-11-20T12:06:28.513 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/38 file after the snapshot 2020-11-20T12:06:28.516 INFO:tasks.workunit.client.0.smithi044.stdout: create 229/39 file after the snapshot 2020-11-20T12:06:28.518 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/.snap/snap-subdir-test 2020-11-20T12:06:28.542 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/30 file after the snapshot 2020-11-20T12:06:28.544 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/31 file after the snapshot 2020-11-20T12:06:28.546 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/32 file after the snapshot 2020-11-20T12:06:28.548 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/33 file after the snapshot 2020-11-20T12:06:28.551 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/34 file after the snapshot 2020-11-20T12:06:28.553 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/35 file after the snapshot 2020-11-20T12:06:28.555 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/36 file after the snapshot 2020-11-20T12:06:28.557 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/37 file after the snapshot 2020-11-20T12:06:28.560 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/38 file after the snapshot 2020-11-20T12:06:28.562 INFO:tasks.workunit.client.0.smithi044.stdout: create 230/39 file after the snapshot 2020-11-20T12:06:28.564 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/.snap/snap-subdir-test 2020-11-20T12:06:28.587 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/30 file after the snapshot 2020-11-20T12:06:28.588 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/31 file after the snapshot 2020-11-20T12:06:28.591 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/32 file after the snapshot 2020-11-20T12:06:28.594 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/33 file after the snapshot 2020-11-20T12:06:28.597 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/34 file after the snapshot 2020-11-20T12:06:28.598 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/35 file after the snapshot 2020-11-20T12:06:28.600 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/36 file after the snapshot 2020-11-20T12:06:28.603 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/37 file after the snapshot 2020-11-20T12:06:28.606 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/38 file after the snapshot 2020-11-20T12:06:28.607 INFO:tasks.workunit.client.0.smithi044.stdout: create 231/39 file after the snapshot 2020-11-20T12:06:28.610 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/.snap/snap-subdir-test 2020-11-20T12:06:28.634 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/30 file after the snapshot 2020-11-20T12:06:28.636 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/31 file after the snapshot 2020-11-20T12:06:28.638 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/32 file after the snapshot 2020-11-20T12:06:28.641 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/33 file after the snapshot 2020-11-20T12:06:28.643 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/34 file after the snapshot 2020-11-20T12:06:28.646 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/35 file after the snapshot 2020-11-20T12:06:28.647 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/36 file after the snapshot 2020-11-20T12:06:28.651 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/37 file after the snapshot 2020-11-20T12:06:28.653 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/38 file after the snapshot 2020-11-20T12:06:28.655 INFO:tasks.workunit.client.0.smithi044.stdout: create 232/39 file after the snapshot 2020-11-20T12:06:28.658 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/.snap/snap-subdir-test 2020-11-20T12:06:28.678 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/30 file after the snapshot 2020-11-20T12:06:28.680 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/31 file after the snapshot 2020-11-20T12:06:28.681 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/32 file after the snapshot 2020-11-20T12:06:28.685 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/33 file after the snapshot 2020-11-20T12:06:28.686 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/34 file after the snapshot 2020-11-20T12:06:28.690 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/35 file after the snapshot 2020-11-20T12:06:28.692 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/36 file after the snapshot 2020-11-20T12:06:28.694 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/37 file after the snapshot 2020-11-20T12:06:28.696 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/38 file after the snapshot 2020-11-20T12:06:28.699 INFO:tasks.workunit.client.0.smithi044.stdout: create 233/39 file after the snapshot 2020-11-20T12:06:28.702 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/.snap/snap-subdir-test 2020-11-20T12:06:28.722 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/30 file after the snapshot 2020-11-20T12:06:28.724 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/31 file after the snapshot 2020-11-20T12:06:28.726 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/32 file after the snapshot 2020-11-20T12:06:28.728 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/33 file after the snapshot 2020-11-20T12:06:28.730 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/34 file after the snapshot 2020-11-20T12:06:28.733 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/35 file after the snapshot 2020-11-20T12:06:28.737 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/36 file after the snapshot 2020-11-20T12:06:28.740 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/37 file after the snapshot 2020-11-20T12:06:28.746 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/38 file after the snapshot 2020-11-20T12:06:28.748 INFO:tasks.workunit.client.0.smithi044.stdout: create 234/39 file after the snapshot 2020-11-20T12:06:28.750 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/.snap/snap-subdir-test 2020-11-20T12:06:28.773 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/30 file after the snapshot 2020-11-20T12:06:28.774 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/31 file after the snapshot 2020-11-20T12:06:28.776 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/32 file after the snapshot 2020-11-20T12:06:28.779 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/33 file after the snapshot 2020-11-20T12:06:28.781 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/34 file after the snapshot 2020-11-20T12:06:28.783 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/35 file after the snapshot 2020-11-20T12:06:28.785 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/36 file after the snapshot 2020-11-20T12:06:28.788 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/37 file after the snapshot 2020-11-20T12:06:28.790 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/38 file after the snapshot 2020-11-20T12:06:28.792 INFO:tasks.workunit.client.0.smithi044.stdout: create 235/39 file after the snapshot 2020-11-20T12:06:28.794 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/.snap/snap-subdir-test 2020-11-20T12:06:28.815 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/30 file after the snapshot 2020-11-20T12:06:28.817 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/31 file after the snapshot 2020-11-20T12:06:28.818 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/32 file after the snapshot 2020-11-20T12:06:28.820 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/33 file after the snapshot 2020-11-20T12:06:28.822 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/34 file after the snapshot 2020-11-20T12:06:28.824 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/35 file after the snapshot 2020-11-20T12:06:28.826 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/36 file after the snapshot 2020-11-20T12:06:28.828 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/37 file after the snapshot 2020-11-20T12:06:28.830 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/38 file after the snapshot 2020-11-20T12:06:28.832 INFO:tasks.workunit.client.0.smithi044.stdout: create 236/39 file after the snapshot 2020-11-20T12:06:28.834 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/.snap/snap-subdir-test 2020-11-20T12:06:28.854 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/30 file after the snapshot 2020-11-20T12:06:28.856 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/31 file after the snapshot 2020-11-20T12:06:28.858 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/32 file after the snapshot 2020-11-20T12:06:28.860 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/33 file after the snapshot 2020-11-20T12:06:28.863 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/34 file after the snapshot 2020-11-20T12:06:28.865 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/35 file after the snapshot 2020-11-20T12:06:28.868 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/36 file after the snapshot 2020-11-20T12:06:28.870 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/37 file after the snapshot 2020-11-20T12:06:28.873 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/38 file after the snapshot 2020-11-20T12:06:28.876 INFO:tasks.workunit.client.0.smithi044.stdout: create 237/39 file after the snapshot 2020-11-20T12:06:28.878 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/.snap/snap-subdir-test 2020-11-20T12:06:28.912 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/30 file after the snapshot 2020-11-20T12:06:28.984 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/31 file after the snapshot 2020-11-20T12:06:28.986 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/32 file after the snapshot 2020-11-20T12:06:28.988 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/33 file after the snapshot 2020-11-20T12:06:28.989 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/34 file after the snapshot 2020-11-20T12:06:28.991 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/35 file after the snapshot 2020-11-20T12:06:28.994 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/36 file after the snapshot 2020-11-20T12:06:28.996 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/37 file after the snapshot 2020-11-20T12:06:28.999 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/38 file after the snapshot 2020-11-20T12:06:29.001 INFO:tasks.workunit.client.0.smithi044.stdout: create 238/39 file after the snapshot 2020-11-20T12:06:29.003 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/.snap/snap-subdir-test 2020-11-20T12:06:29.073 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/30 file after the snapshot 2020-11-20T12:06:29.075 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/31 file after the snapshot 2020-11-20T12:06:29.077 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/32 file after the snapshot 2020-11-20T12:06:29.079 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/33 file after the snapshot 2020-11-20T12:06:29.081 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/34 file after the snapshot 2020-11-20T12:06:29.084 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/35 file after the snapshot 2020-11-20T12:06:29.086 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/36 file after the snapshot 2020-11-20T12:06:29.089 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/37 file after the snapshot 2020-11-20T12:06:29.090 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/38 file after the snapshot 2020-11-20T12:06:29.092 INFO:tasks.workunit.client.0.smithi044.stdout: create 239/39 file after the snapshot 2020-11-20T12:06:29.095 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/.snap/snap-subdir-test 2020-11-20T12:06:29.117 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/30 file after the snapshot 2020-11-20T12:06:29.119 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/31 file after the snapshot 2020-11-20T12:06:29.122 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/32 file after the snapshot 2020-11-20T12:06:29.123 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/33 file after the snapshot 2020-11-20T12:06:29.126 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/34 file after the snapshot 2020-11-20T12:06:29.128 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/35 file after the snapshot 2020-11-20T12:06:29.130 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/36 file after the snapshot 2020-11-20T12:06:29.132 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/37 file after the snapshot 2020-11-20T12:06:29.135 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/38 file after the snapshot 2020-11-20T12:06:29.138 INFO:tasks.workunit.client.0.smithi044.stdout: create 240/39 file after the snapshot 2020-11-20T12:06:29.140 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/.snap/snap-subdir-test 2020-11-20T12:06:29.160 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/30 file after the snapshot 2020-11-20T12:06:29.162 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/31 file after the snapshot 2020-11-20T12:06:29.163 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/32 file after the snapshot 2020-11-20T12:06:29.166 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/33 file after the snapshot 2020-11-20T12:06:29.168 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/34 file after the snapshot 2020-11-20T12:06:29.172 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/35 file after the snapshot 2020-11-20T12:06:29.174 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/36 file after the snapshot 2020-11-20T12:06:29.177 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/37 file after the snapshot 2020-11-20T12:06:29.248 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/38 file after the snapshot 2020-11-20T12:06:29.249 INFO:tasks.workunit.client.0.smithi044.stdout: create 241/39 file after the snapshot 2020-11-20T12:06:29.249 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/.snap/snap-subdir-test 2020-11-20T12:06:29.249 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/30 file after the snapshot 2020-11-20T12:06:29.249 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/31 file after the snapshot 2020-11-20T12:06:29.249 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/32 file after the snapshot 2020-11-20T12:06:29.250 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/33 file after the snapshot 2020-11-20T12:06:29.250 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/34 file after the snapshot 2020-11-20T12:06:29.250 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/35 file after the snapshot 2020-11-20T12:06:29.251 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/36 file after the snapshot 2020-11-20T12:06:29.251 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/37 file after the snapshot 2020-11-20T12:06:29.251 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/38 file after the snapshot 2020-11-20T12:06:29.252 INFO:tasks.workunit.client.0.smithi044.stdout: create 242/39 file after the snapshot 2020-11-20T12:06:29.252 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/.snap/snap-subdir-test 2020-11-20T12:06:29.252 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/30 file after the snapshot 2020-11-20T12:06:29.253 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/31 file after the snapshot 2020-11-20T12:06:29.253 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/32 file after the snapshot 2020-11-20T12:06:29.253 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/33 file after the snapshot 2020-11-20T12:06:29.255 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/34 file after the snapshot 2020-11-20T12:06:29.257 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/35 file after the snapshot 2020-11-20T12:06:29.260 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/36 file after the snapshot 2020-11-20T12:06:29.263 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/37 file after the snapshot 2020-11-20T12:06:29.266 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/38 file after the snapshot 2020-11-20T12:06:29.268 INFO:tasks.workunit.client.0.smithi044.stdout: create 243/39 file after the snapshot 2020-11-20T12:06:29.271 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/.snap/snap-subdir-test 2020-11-20T12:06:29.295 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/30 file after the snapshot 2020-11-20T12:06:29.297 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/31 file after the snapshot 2020-11-20T12:06:29.298 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/32 file after the snapshot 2020-11-20T12:06:29.300 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/33 file after the snapshot 2020-11-20T12:06:29.302 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/34 file after the snapshot 2020-11-20T12:06:29.304 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/35 file after the snapshot 2020-11-20T12:06:29.307 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/36 file after the snapshot 2020-11-20T12:06:29.309 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/37 file after the snapshot 2020-11-20T12:06:29.312 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/38 file after the snapshot 2020-11-20T12:06:29.314 INFO:tasks.workunit.client.0.smithi044.stdout: create 244/39 file after the snapshot 2020-11-20T12:06:29.316 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/.snap/snap-subdir-test 2020-11-20T12:06:29.341 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/30 file after the snapshot 2020-11-20T12:06:29.342 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/31 file after the snapshot 2020-11-20T12:06:29.345 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/32 file after the snapshot 2020-11-20T12:06:29.346 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/33 file after the snapshot 2020-11-20T12:06:29.349 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/34 file after the snapshot 2020-11-20T12:06:29.355 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/35 file after the snapshot 2020-11-20T12:06:29.356 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/36 file after the snapshot 2020-11-20T12:06:29.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/37 file after the snapshot 2020-11-20T12:06:29.359 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/38 file after the snapshot 2020-11-20T12:06:29.360 INFO:tasks.workunit.client.0.smithi044.stdout: create 245/39 file after the snapshot 2020-11-20T12:06:29.363 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/.snap/snap-subdir-test 2020-11-20T12:06:29.384 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/30 file after the snapshot 2020-11-20T12:06:29.385 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/31 file after the snapshot 2020-11-20T12:06:29.388 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/32 file after the snapshot 2020-11-20T12:06:29.392 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/33 file after the snapshot 2020-11-20T12:06:29.394 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/34 file after the snapshot 2020-11-20T12:06:29.397 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/35 file after the snapshot 2020-11-20T12:06:29.399 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/36 file after the snapshot 2020-11-20T12:06:29.401 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/37 file after the snapshot 2020-11-20T12:06:29.404 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/38 file after the snapshot 2020-11-20T12:06:29.407 INFO:tasks.workunit.client.0.smithi044.stdout: create 246/39 file after the snapshot 2020-11-20T12:06:29.409 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/.snap/snap-subdir-test 2020-11-20T12:06:29.430 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/30 file after the snapshot 2020-11-20T12:06:29.431 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/31 file after the snapshot 2020-11-20T12:06:29.433 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/32 file after the snapshot 2020-11-20T12:06:29.435 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/33 file after the snapshot 2020-11-20T12:06:29.438 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/34 file after the snapshot 2020-11-20T12:06:29.439 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/35 file after the snapshot 2020-11-20T12:06:29.442 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/36 file after the snapshot 2020-11-20T12:06:29.444 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/37 file after the snapshot 2020-11-20T12:06:29.446 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/38 file after the snapshot 2020-11-20T12:06:29.448 INFO:tasks.workunit.client.0.smithi044.stdout: create 247/39 file after the snapshot 2020-11-20T12:06:29.450 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/.snap/snap-subdir-test 2020-11-20T12:06:29.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/30 file after the snapshot 2020-11-20T12:06:29.473 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/31 file after the snapshot 2020-11-20T12:06:29.475 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/32 file after the snapshot 2020-11-20T12:06:29.478 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/33 file after the snapshot 2020-11-20T12:06:29.480 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/34 file after the snapshot 2020-11-20T12:06:29.482 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/35 file after the snapshot 2020-11-20T12:06:29.485 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/36 file after the snapshot 2020-11-20T12:06:29.487 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/37 file after the snapshot 2020-11-20T12:06:29.489 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/38 file after the snapshot 2020-11-20T12:06:29.492 INFO:tasks.workunit.client.0.smithi044.stdout: create 248/39 file after the snapshot 2020-11-20T12:06:29.495 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/.snap/snap-subdir-test 2020-11-20T12:06:29.516 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/30 file after the snapshot 2020-11-20T12:06:29.518 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/31 file after the snapshot 2020-11-20T12:06:29.520 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/32 file after the snapshot 2020-11-20T12:06:29.522 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/33 file after the snapshot 2020-11-20T12:06:29.524 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/34 file after the snapshot 2020-11-20T12:06:29.527 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/35 file after the snapshot 2020-11-20T12:06:29.529 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/36 file after the snapshot 2020-11-20T12:06:29.531 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/37 file after the snapshot 2020-11-20T12:06:29.534 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/38 file after the snapshot 2020-11-20T12:06:29.536 INFO:tasks.workunit.client.0.smithi044.stdout: create 249/39 file after the snapshot 2020-11-20T12:06:29.537 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/.snap/snap-subdir-test 2020-11-20T12:06:29.557 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/30 file after the snapshot 2020-11-20T12:06:29.559 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/31 file after the snapshot 2020-11-20T12:06:29.561 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/32 file after the snapshot 2020-11-20T12:06:29.563 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/33 file after the snapshot 2020-11-20T12:06:29.566 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/34 file after the snapshot 2020-11-20T12:06:29.568 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/35 file after the snapshot 2020-11-20T12:06:29.570 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/36 file after the snapshot 2020-11-20T12:06:29.572 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/37 file after the snapshot 2020-11-20T12:06:29.576 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/38 file after the snapshot 2020-11-20T12:06:29.578 INFO:tasks.workunit.client.0.smithi044.stdout: create 250/39 file after the snapshot 2020-11-20T12:06:29.579 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/.snap/snap-subdir-test 2020-11-20T12:06:29.600 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/30 file after the snapshot 2020-11-20T12:06:29.602 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/31 file after the snapshot 2020-11-20T12:06:29.603 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/32 file after the snapshot 2020-11-20T12:06:29.605 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/33 file after the snapshot 2020-11-20T12:06:29.606 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/34 file after the snapshot 2020-11-20T12:06:29.609 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/35 file after the snapshot 2020-11-20T12:06:29.612 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/36 file after the snapshot 2020-11-20T12:06:29.615 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/37 file after the snapshot 2020-11-20T12:06:29.617 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/38 file after the snapshot 2020-11-20T12:06:29.618 INFO:tasks.workunit.client.0.smithi044.stdout: create 251/39 file after the snapshot 2020-11-20T12:06:29.621 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/.snap/snap-subdir-test 2020-11-20T12:06:29.641 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/30 file after the snapshot 2020-11-20T12:06:29.642 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/31 file after the snapshot 2020-11-20T12:06:29.644 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/32 file after the snapshot 2020-11-20T12:06:29.648 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/33 file after the snapshot 2020-11-20T12:06:29.649 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/34 file after the snapshot 2020-11-20T12:06:29.651 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/35 file after the snapshot 2020-11-20T12:06:29.653 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/36 file after the snapshot 2020-11-20T12:06:29.655 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/37 file after the snapshot 2020-11-20T12:06:29.657 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/38 file after the snapshot 2020-11-20T12:06:29.660 INFO:tasks.workunit.client.0.smithi044.stdout: create 252/39 file after the snapshot 2020-11-20T12:06:29.662 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/.snap/snap-subdir-test 2020-11-20T12:06:29.681 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/30 file after the snapshot 2020-11-20T12:06:29.683 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/31 file after the snapshot 2020-11-20T12:06:29.685 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/32 file after the snapshot 2020-11-20T12:06:29.687 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/33 file after the snapshot 2020-11-20T12:06:29.689 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/34 file after the snapshot 2020-11-20T12:06:29.691 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/35 file after the snapshot 2020-11-20T12:06:29.694 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/36 file after the snapshot 2020-11-20T12:06:29.696 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/37 file after the snapshot 2020-11-20T12:06:29.698 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/38 file after the snapshot 2020-11-20T12:06:29.699 INFO:tasks.workunit.client.0.smithi044.stdout: create 253/39 file after the snapshot 2020-11-20T12:06:29.703 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/.snap/snap-subdir-test 2020-11-20T12:06:29.725 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/30 file after the snapshot 2020-11-20T12:06:29.726 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/31 file after the snapshot 2020-11-20T12:06:29.728 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/32 file after the snapshot 2020-11-20T12:06:29.730 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/33 file after the snapshot 2020-11-20T12:06:29.732 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/34 file after the snapshot 2020-11-20T12:06:29.735 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/35 file after the snapshot 2020-11-20T12:06:29.737 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/36 file after the snapshot 2020-11-20T12:06:29.739 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/37 file after the snapshot 2020-11-20T12:06:29.742 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/38 file after the snapshot 2020-11-20T12:06:29.745 INFO:tasks.workunit.client.0.smithi044.stdout: create 254/39 file after the snapshot 2020-11-20T12:06:29.747 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/.snap/snap-subdir-test 2020-11-20T12:06:29.770 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/30 file after the snapshot 2020-11-20T12:06:29.771 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/31 file after the snapshot 2020-11-20T12:06:29.773 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/32 file after the snapshot 2020-11-20T12:06:29.775 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/33 file after the snapshot 2020-11-20T12:06:29.779 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/34 file after the snapshot 2020-11-20T12:06:29.782 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/35 file after the snapshot 2020-11-20T12:06:29.784 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/36 file after the snapshot 2020-11-20T12:06:29.787 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/37 file after the snapshot 2020-11-20T12:06:29.791 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/38 file after the snapshot 2020-11-20T12:06:29.794 INFO:tasks.workunit.client.0.smithi044.stdout: create 255/39 file after the snapshot 2020-11-20T12:06:29.796 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/.snap/snap-subdir-test 2020-11-20T12:06:29.820 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/30 file after the snapshot 2020-11-20T12:06:29.822 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/31 file after the snapshot 2020-11-20T12:06:29.823 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/32 file after the snapshot 2020-11-20T12:06:29.826 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/33 file after the snapshot 2020-11-20T12:06:29.829 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/34 file after the snapshot 2020-11-20T12:06:29.831 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/35 file after the snapshot 2020-11-20T12:06:29.834 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/36 file after the snapshot 2020-11-20T12:06:29.836 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/37 file after the snapshot 2020-11-20T12:06:29.838 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/38 file after the snapshot 2020-11-20T12:06:29.841 INFO:tasks.workunit.client.0.smithi044.stdout: create 256/39 file after the snapshot 2020-11-20T12:06:29.844 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/.snap/snap-subdir-test 2020-11-20T12:06:29.866 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/30 file after the snapshot 2020-11-20T12:06:29.868 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/31 file after the snapshot 2020-11-20T12:06:29.870 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/32 file after the snapshot 2020-11-20T12:06:29.873 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/33 file after the snapshot 2020-11-20T12:06:29.875 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/34 file after the snapshot 2020-11-20T12:06:29.877 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/35 file after the snapshot 2020-11-20T12:06:29.879 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/36 file after the snapshot 2020-11-20T12:06:29.881 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/37 file after the snapshot 2020-11-20T12:06:29.887 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/38 file after the snapshot 2020-11-20T12:06:29.887 INFO:tasks.workunit.client.0.smithi044.stdout: create 257/39 file after the snapshot 2020-11-20T12:06:29.888 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/.snap/snap-subdir-test 2020-11-20T12:06:29.908 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/30 file after the snapshot 2020-11-20T12:06:29.909 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/31 file after the snapshot 2020-11-20T12:06:29.928 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/32 file after the snapshot 2020-11-20T12:06:29.986 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/33 file after the snapshot 2020-11-20T12:06:29.989 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/34 file after the snapshot 2020-11-20T12:06:29.991 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/35 file after the snapshot 2020-11-20T12:06:29.992 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/36 file after the snapshot 2020-11-20T12:06:29.995 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/37 file after the snapshot 2020-11-20T12:06:29.998 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/38 file after the snapshot 2020-11-20T12:06:30.000 INFO:tasks.workunit.client.0.smithi044.stdout: create 258/39 file after the snapshot 2020-11-20T12:06:30.003 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/.snap/snap-subdir-test 2020-11-20T12:06:30.061 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/30 file after the snapshot 2020-11-20T12:06:30.062 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/31 file after the snapshot 2020-11-20T12:06:30.064 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/32 file after the snapshot 2020-11-20T12:06:30.067 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/33 file after the snapshot 2020-11-20T12:06:30.069 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/34 file after the snapshot 2020-11-20T12:06:30.072 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/35 file after the snapshot 2020-11-20T12:06:30.073 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/36 file after the snapshot 2020-11-20T12:06:30.075 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/37 file after the snapshot 2020-11-20T12:06:30.077 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/38 file after the snapshot 2020-11-20T12:06:30.079 INFO:tasks.workunit.client.0.smithi044.stdout: create 259/39 file after the snapshot 2020-11-20T12:06:30.081 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/.snap/snap-subdir-test 2020-11-20T12:06:30.100 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/30 file after the snapshot 2020-11-20T12:06:30.102 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/31 file after the snapshot 2020-11-20T12:06:30.104 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/32 file after the snapshot 2020-11-20T12:06:30.107 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/33 file after the snapshot 2020-11-20T12:06:30.109 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/34 file after the snapshot 2020-11-20T12:06:30.111 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/35 file after the snapshot 2020-11-20T12:06:30.114 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/36 file after the snapshot 2020-11-20T12:06:30.117 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/37 file after the snapshot 2020-11-20T12:06:30.119 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/38 file after the snapshot 2020-11-20T12:06:30.121 INFO:tasks.workunit.client.0.smithi044.stdout: create 260/39 file after the snapshot 2020-11-20T12:06:30.123 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/.snap/snap-subdir-test 2020-11-20T12:06:30.147 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/30 file after the snapshot 2020-11-20T12:06:30.148 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/31 file after the snapshot 2020-11-20T12:06:30.150 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/32 file after the snapshot 2020-11-20T12:06:30.153 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/33 file after the snapshot 2020-11-20T12:06:30.155 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/34 file after the snapshot 2020-11-20T12:06:30.158 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/35 file after the snapshot 2020-11-20T12:06:30.160 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/36 file after the snapshot 2020-11-20T12:06:30.162 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/37 file after the snapshot 2020-11-20T12:06:30.164 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/38 file after the snapshot 2020-11-20T12:06:30.167 INFO:tasks.workunit.client.0.smithi044.stdout: create 261/39 file after the snapshot 2020-11-20T12:06:30.169 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/.snap/snap-subdir-test 2020-11-20T12:06:30.192 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/30 file after the snapshot 2020-11-20T12:06:30.194 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/31 file after the snapshot 2020-11-20T12:06:30.195 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/32 file after the snapshot 2020-11-20T12:06:30.197 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/33 file after the snapshot 2020-11-20T12:06:30.199 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/34 file after the snapshot 2020-11-20T12:06:30.201 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/35 file after the snapshot 2020-11-20T12:06:30.204 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/36 file after the snapshot 2020-11-20T12:06:30.206 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/37 file after the snapshot 2020-11-20T12:06:30.209 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/38 file after the snapshot 2020-11-20T12:06:30.211 INFO:tasks.workunit.client.0.smithi044.stdout: create 262/39 file after the snapshot 2020-11-20T12:06:30.213 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/.snap/snap-subdir-test 2020-11-20T12:06:30.236 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/30 file after the snapshot 2020-11-20T12:06:30.238 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/31 file after the snapshot 2020-11-20T12:06:30.240 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/32 file after the snapshot 2020-11-20T12:06:30.242 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/33 file after the snapshot 2020-11-20T12:06:30.245 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/34 file after the snapshot 2020-11-20T12:06:30.248 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/35 file after the snapshot 2020-11-20T12:06:30.251 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/36 file after the snapshot 2020-11-20T12:06:30.254 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/37 file after the snapshot 2020-11-20T12:06:30.257 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/38 file after the snapshot 2020-11-20T12:06:30.259 INFO:tasks.workunit.client.0.smithi044.stdout: create 263/39 file after the snapshot 2020-11-20T12:06:30.261 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/.snap/snap-subdir-test 2020-11-20T12:06:30.282 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/30 file after the snapshot 2020-11-20T12:06:30.283 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/31 file after the snapshot 2020-11-20T12:06:30.286 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/32 file after the snapshot 2020-11-20T12:06:30.288 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/33 file after the snapshot 2020-11-20T12:06:30.290 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/34 file after the snapshot 2020-11-20T12:06:30.292 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/35 file after the snapshot 2020-11-20T12:06:30.294 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/36 file after the snapshot 2020-11-20T12:06:30.296 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/37 file after the snapshot 2020-11-20T12:06:30.298 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/38 file after the snapshot 2020-11-20T12:06:30.300 INFO:tasks.workunit.client.0.smithi044.stdout: create 264/39 file after the snapshot 2020-11-20T12:06:30.302 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/.snap/snap-subdir-test 2020-11-20T12:06:30.325 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/30 file after the snapshot 2020-11-20T12:06:30.326 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/31 file after the snapshot 2020-11-20T12:06:30.328 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/32 file after the snapshot 2020-11-20T12:06:30.331 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/33 file after the snapshot 2020-11-20T12:06:30.333 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/34 file after the snapshot 2020-11-20T12:06:30.335 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/35 file after the snapshot 2020-11-20T12:06:30.338 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/36 file after the snapshot 2020-11-20T12:06:30.340 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/37 file after the snapshot 2020-11-20T12:06:30.343 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/38 file after the snapshot 2020-11-20T12:06:30.345 INFO:tasks.workunit.client.0.smithi044.stdout: create 265/39 file after the snapshot 2020-11-20T12:06:30.346 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/.snap/snap-subdir-test 2020-11-20T12:06:30.367 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/30 file after the snapshot 2020-11-20T12:06:30.368 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/31 file after the snapshot 2020-11-20T12:06:30.370 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/32 file after the snapshot 2020-11-20T12:06:30.374 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/33 file after the snapshot 2020-11-20T12:06:30.377 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/34 file after the snapshot 2020-11-20T12:06:30.379 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/35 file after the snapshot 2020-11-20T12:06:30.382 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/36 file after the snapshot 2020-11-20T12:06:30.384 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/37 file after the snapshot 2020-11-20T12:06:30.387 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/38 file after the snapshot 2020-11-20T12:06:30.389 INFO:tasks.workunit.client.0.smithi044.stdout: create 266/39 file after the snapshot 2020-11-20T12:06:30.392 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/.snap/snap-subdir-test 2020-11-20T12:06:30.409 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/30 file after the snapshot 2020-11-20T12:06:30.411 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/31 file after the snapshot 2020-11-20T12:06:30.413 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/32 file after the snapshot 2020-11-20T12:06:30.415 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/33 file after the snapshot 2020-11-20T12:06:30.418 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/34 file after the snapshot 2020-11-20T12:06:30.421 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/35 file after the snapshot 2020-11-20T12:06:30.423 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/36 file after the snapshot 2020-11-20T12:06:30.425 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/37 file after the snapshot 2020-11-20T12:06:30.428 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/38 file after the snapshot 2020-11-20T12:06:30.430 INFO:tasks.workunit.client.0.smithi044.stdout: create 267/39 file after the snapshot 2020-11-20T12:06:30.433 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/.snap/snap-subdir-test 2020-11-20T12:06:30.452 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/30 file after the snapshot 2020-11-20T12:06:30.453 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/31 file after the snapshot 2020-11-20T12:06:30.455 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/32 file after the snapshot 2020-11-20T12:06:30.458 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/33 file after the snapshot 2020-11-20T12:06:30.460 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/34 file after the snapshot 2020-11-20T12:06:30.462 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/35 file after the snapshot 2020-11-20T12:06:30.464 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/36 file after the snapshot 2020-11-20T12:06:30.467 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/37 file after the snapshot 2020-11-20T12:06:30.469 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/38 file after the snapshot 2020-11-20T12:06:30.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 268/39 file after the snapshot 2020-11-20T12:06:30.473 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/.snap/snap-subdir-test 2020-11-20T12:06:30.494 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/30 file after the snapshot 2020-11-20T12:06:30.496 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/31 file after the snapshot 2020-11-20T12:06:30.499 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/32 file after the snapshot 2020-11-20T12:06:30.501 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/33 file after the snapshot 2020-11-20T12:06:30.503 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/34 file after the snapshot 2020-11-20T12:06:30.506 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/35 file after the snapshot 2020-11-20T12:06:30.508 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/36 file after the snapshot 2020-11-20T12:06:30.510 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/37 file after the snapshot 2020-11-20T12:06:30.513 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/38 file after the snapshot 2020-11-20T12:06:30.515 INFO:tasks.workunit.client.0.smithi044.stdout: create 269/39 file after the snapshot 2020-11-20T12:06:30.518 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/.snap/snap-subdir-test 2020-11-20T12:06:30.538 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/30 file after the snapshot 2020-11-20T12:06:30.540 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/31 file after the snapshot 2020-11-20T12:06:30.542 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/32 file after the snapshot 2020-11-20T12:06:30.544 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/33 file after the snapshot 2020-11-20T12:06:30.546 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/34 file after the snapshot 2020-11-20T12:06:30.548 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/35 file after the snapshot 2020-11-20T12:06:30.550 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/36 file after the snapshot 2020-11-20T12:06:30.552 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/37 file after the snapshot 2020-11-20T12:06:30.556 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/38 file after the snapshot 2020-11-20T12:06:30.558 INFO:tasks.workunit.client.0.smithi044.stdout: create 270/39 file after the snapshot 2020-11-20T12:06:30.560 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/.snap/snap-subdir-test 2020-11-20T12:06:30.582 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/30 file after the snapshot 2020-11-20T12:06:30.583 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/31 file after the snapshot 2020-11-20T12:06:30.585 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/32 file after the snapshot 2020-11-20T12:06:30.588 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/33 file after the snapshot 2020-11-20T12:06:30.590 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/34 file after the snapshot 2020-11-20T12:06:30.591 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/35 file after the snapshot 2020-11-20T12:06:30.595 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/36 file after the snapshot 2020-11-20T12:06:30.597 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/37 file after the snapshot 2020-11-20T12:06:30.599 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/38 file after the snapshot 2020-11-20T12:06:30.602 INFO:tasks.workunit.client.0.smithi044.stdout: create 271/39 file after the snapshot 2020-11-20T12:06:30.604 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/.snap/snap-subdir-test 2020-11-20T12:06:30.625 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/30 file after the snapshot 2020-11-20T12:06:30.627 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/31 file after the snapshot 2020-11-20T12:06:30.629 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/32 file after the snapshot 2020-11-20T12:06:30.631 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/33 file after the snapshot 2020-11-20T12:06:30.634 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/34 file after the snapshot 2020-11-20T12:06:30.637 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/35 file after the snapshot 2020-11-20T12:06:30.639 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/36 file after the snapshot 2020-11-20T12:06:30.642 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/37 file after the snapshot 2020-11-20T12:06:30.644 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/38 file after the snapshot 2020-11-20T12:06:30.646 INFO:tasks.workunit.client.0.smithi044.stdout: create 272/39 file after the snapshot 2020-11-20T12:06:30.648 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/.snap/snap-subdir-test 2020-11-20T12:06:30.667 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/30 file after the snapshot 2020-11-20T12:06:30.669 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/31 file after the snapshot 2020-11-20T12:06:30.671 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/32 file after the snapshot 2020-11-20T12:06:30.673 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/33 file after the snapshot 2020-11-20T12:06:30.676 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/34 file after the snapshot 2020-11-20T12:06:30.680 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/35 file after the snapshot 2020-11-20T12:06:30.682 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/36 file after the snapshot 2020-11-20T12:06:30.684 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/37 file after the snapshot 2020-11-20T12:06:30.686 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/38 file after the snapshot 2020-11-20T12:06:30.688 INFO:tasks.workunit.client.0.smithi044.stdout: create 273/39 file after the snapshot 2020-11-20T12:06:30.691 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/.snap/snap-subdir-test 2020-11-20T12:06:30.712 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/30 file after the snapshot 2020-11-20T12:06:30.713 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/31 file after the snapshot 2020-11-20T12:06:30.716 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/32 file after the snapshot 2020-11-20T12:06:30.718 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/33 file after the snapshot 2020-11-20T12:06:30.720 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/34 file after the snapshot 2020-11-20T12:06:30.724 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/35 file after the snapshot 2020-11-20T12:06:30.726 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/36 file after the snapshot 2020-11-20T12:06:30.730 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/37 file after the snapshot 2020-11-20T12:06:30.732 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/38 file after the snapshot 2020-11-20T12:06:30.735 INFO:tasks.workunit.client.0.smithi044.stdout: create 274/39 file after the snapshot 2020-11-20T12:06:30.737 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/.snap/snap-subdir-test 2020-11-20T12:06:30.757 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/30 file after the snapshot 2020-11-20T12:06:30.758 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/31 file after the snapshot 2020-11-20T12:06:30.760 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/32 file after the snapshot 2020-11-20T12:06:30.762 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/33 file after the snapshot 2020-11-20T12:06:30.765 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/34 file after the snapshot 2020-11-20T12:06:30.767 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/35 file after the snapshot 2020-11-20T12:06:30.770 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/36 file after the snapshot 2020-11-20T12:06:30.774 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/37 file after the snapshot 2020-11-20T12:06:30.775 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/38 file after the snapshot 2020-11-20T12:06:30.777 INFO:tasks.workunit.client.0.smithi044.stdout: create 275/39 file after the snapshot 2020-11-20T12:06:30.780 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/.snap/snap-subdir-test 2020-11-20T12:06:30.804 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/30 file after the snapshot 2020-11-20T12:06:30.805 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/31 file after the snapshot 2020-11-20T12:06:30.807 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/32 file after the snapshot 2020-11-20T12:06:30.810 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/33 file after the snapshot 2020-11-20T12:06:30.812 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/34 file after the snapshot 2020-11-20T12:06:30.815 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/35 file after the snapshot 2020-11-20T12:06:30.817 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/36 file after the snapshot 2020-11-20T12:06:30.819 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/37 file after the snapshot 2020-11-20T12:06:30.821 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/38 file after the snapshot 2020-11-20T12:06:30.824 INFO:tasks.workunit.client.0.smithi044.stdout: create 276/39 file after the snapshot 2020-11-20T12:06:30.826 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/.snap/snap-subdir-test 2020-11-20T12:06:30.846 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/30 file after the snapshot 2020-11-20T12:06:30.848 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/31 file after the snapshot 2020-11-20T12:06:30.850 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/32 file after the snapshot 2020-11-20T12:06:30.852 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/33 file after the snapshot 2020-11-20T12:06:30.854 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/34 file after the snapshot 2020-11-20T12:06:30.856 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/35 file after the snapshot 2020-11-20T12:06:30.859 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/36 file after the snapshot 2020-11-20T12:06:30.862 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/37 file after the snapshot 2020-11-20T12:06:30.864 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/38 file after the snapshot 2020-11-20T12:06:30.866 INFO:tasks.workunit.client.0.smithi044.stdout: create 277/39 file after the snapshot 2020-11-20T12:06:30.869 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/.snap/snap-subdir-test 2020-11-20T12:06:30.890 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/30 file after the snapshot 2020-11-20T12:06:30.892 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/31 file after the snapshot 2020-11-20T12:06:30.893 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/32 file after the snapshot 2020-11-20T12:06:30.895 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/33 file after the snapshot 2020-11-20T12:06:30.897 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/34 file after the snapshot 2020-11-20T12:06:30.899 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/35 file after the snapshot 2020-11-20T12:06:30.901 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/36 file after the snapshot 2020-11-20T12:06:30.904 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/37 file after the snapshot 2020-11-20T12:06:30.905 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/38 file after the snapshot 2020-11-20T12:06:30.908 INFO:tasks.workunit.client.0.smithi044.stdout: create 278/39 file after the snapshot 2020-11-20T12:06:30.910 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/.snap/snap-subdir-test 2020-11-20T12:06:30.923 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:06:30.939 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/30 file after the snapshot 2020-11-20T12:06:31.019 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/31 file after the snapshot 2020-11-20T12:06:31.021 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/32 file after the snapshot 2020-11-20T12:06:31.024 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/33 file after the snapshot 2020-11-20T12:06:31.026 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/34 file after the snapshot 2020-11-20T12:06:31.028 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/35 file after the snapshot 2020-11-20T12:06:31.031 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/36 file after the snapshot 2020-11-20T12:06:31.034 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/37 file after the snapshot 2020-11-20T12:06:31.036 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/38 file after the snapshot 2020-11-20T12:06:31.038 INFO:tasks.workunit.client.0.smithi044.stdout: create 279/39 file after the snapshot 2020-11-20T12:06:31.042 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/.snap/snap-subdir-test 2020-11-20T12:06:31.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.067+0000 7fc9c0da7700 1 Processor -- start 2020-11-20T12:06:31.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.067+0000 7fc9c0da7700 1 -- start start 2020-11-20T12:06:31.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.068+0000 7fc9c0da7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc9bc0ae540 0x7fc9bc0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:31.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.068+0000 7fc9c0da7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc13b2d0 0x7fc9bc13b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:31.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.068+0000 7fc9c0da7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc067c60 con 0x7fc9bc0ae920 2020-11-20T12:06:31.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.068+0000 7fc9c0da7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9bc0a9980 con 0x7fc9bc0ae540 2020-11-20T12:06:31.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.068+0000 7fc9c0da7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc13bc00 con 0x7fc9bc13b2d0 2020-11-20T12:06:31.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc13b2d0 0x7fc9bc13b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:31.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bb7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc9bc0ae540 0x7fc9bc0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:31.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc13b2d0 0x7fc9bc13b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49524/0 (socket says 172.21.15.44:49524) 2020-11-20T12:06:31.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 -- 172.21.15.44:0/1091141089 learned_addr learned my addr 172.21.15.44:0/1091141089 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:31.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 -- 172.21.15.44:0/1091141089 >> v2:172.21.15.79:3300/0 conn(0x7fc9bc0ae540 msgr2=0x7fc9bc0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:31.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 --2- 172.21.15.44:0/1091141089 >> v2:172.21.15.79:3300/0 conn(0x7fc9bc0ae540 0x7fc9bc0673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:31.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 -- 172.21.15.44:0/1091141089 >> v1:172.21.15.79:6789/0 conn(0x7fc9bc0ae920 legacy=0x7fc9bc0a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:31.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 -- 172.21.15.44:0/1091141089 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0679c0 con 0x7fc9bc13b2d0 2020-11-20T12:06:31.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9bbfff700 1 --2- 172.21.15.44:0/1091141089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc13b2d0 0x7fc9bc13b710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc9b4009d50 tx=0x7fc9b4009290).ready entity=mon.2 client_cookie=151a337001e585ac server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:31.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9ba7fc700 1 -- 172.21.15.44:0/1091141089 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400bc50 con 0x7fc9bc13b2d0 2020-11-20T12:06:31.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9ba7fc700 1 -- 172.21.15.44:0/1091141089 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b400bdb0 con 0x7fc9bc13b2d0 2020-11-20T12:06:31.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.069+0000 7fc9ba7fc700 1 -- 172.21.15.44:0/1091141089 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4011610 con 0x7fc9bc13b2d0 2020-11-20T12:06:31.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.070+0000 7fc9c0da7700 1 -- 172.21.15.44:0/1091141089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc13b2d0 msgr2=0x7fc9bc13b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:31.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.070+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/1091141089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc13b2d0 0x7fc9bc13b710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc9b4009d50 tx=0x7fc9b4009290).stop 2020-11-20T12:06:31.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.071+0000 7fc9c0da7700 1 -- 172.21.15.44:0/1091141089 shutdown_connections 2020-11-20T12:06:31.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.071+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/1091141089 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc13b2d0 0x7fc9bc13b710 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:31.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.071+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/1091141089 >> v2:172.21.15.79:3300/0 conn(0x7fc9bc0ae540 0x7fc9bc0673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:31.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.071+0000 7fc9c0da7700 1 -- 172.21.15.44:0/1091141089 >> 172.21.15.44:0/1091141089 conn(0x7fc9bc0654e0 msgr2=0x7fc9bc0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:31.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.071+0000 7fc9c0da7700 1 -- 172.21.15.44:0/1091141089 shutdown_connections 2020-11-20T12:06:31.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.071+0000 7fc9c0da7700 1 -- 172.21.15.44:0/1091141089 wait complete. 2020-11-20T12:06:31.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.072+0000 7fc9c0da7700 1 Processor -- start 2020-11-20T12:06:31.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.072+0000 7fc9c0da7700 1 -- start start 2020-11-20T12:06:31.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9c0da7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc0ae540 0x7fc9bc137180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:31.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9c0da7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9bc0ae920 0x7fc9bc09ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:31.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9c0da7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9bc13b2d0 0x7fc9bc146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:31.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9c0da7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0a0480 con 0x7fc9bc13b2d0 2020-11-20T12:06:31.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9c0da7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc09ed90 con 0x7fc9bc0ae920 2020-11-20T12:06:31.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9c0da7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc09f030 con 0x7fc9bc0ae540 2020-11-20T12:06:31.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9bbfff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9bc13b2d0 msgr2=0x7fc9bc146a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:31.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9bbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9bc13b2d0 0x7fc9bc146a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:31.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9bb7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc0ae540 0x7fc9bc137180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:31.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9bb7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc0ae540 0x7fc9bc137180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49526/0 (socket says 172.21.15.44:49526) 2020-11-20T12:06:31.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9baffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9bc0ae920 0x7fc9bc09ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:31.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.073+0000 7fc9bb7fe700 1 -- 172.21.15.44:0/3297697192 learned_addr learned my addr 172.21.15.44:0/3297697192 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:31.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9bb7fe700 1 -- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9bc0ae920 msgr2=0x7fc9bc09ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:31.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9bb7fe700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9bc0ae920 0x7fc9bc09ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:31.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9bb7fe700 1 -- 172.21.15.44:0/3297697192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9bc13b2d0 msgr2=0x7fc9bc146a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:31.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9bb7fe700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9bc13b2d0 0x7fc9bc146a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:31.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9bb7fe700 1 -- 172.21.15.44:0/3297697192 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc09f2d0 con 0x7fc9bc0ae540 2020-11-20T12:06:31.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9baffd700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9bc0ae920 0x7fc9bc09ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:31.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9bb7fe700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc0ae540 0x7fc9bc137180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc9ac00ebd0 tx=0x7fc9ac006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:31.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc99b7fe700 1 -- 172.21.15.44:0/3297697192 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00c9b0 con 0x7fc9bc0ae540 2020-11-20T12:06:31.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc99b7fe700 1 -- 172.21.15.44:0/3297697192 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac00cb10 con 0x7fc9bc0ae540 2020-11-20T12:06:31.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc09e810 con 0x7fc9bc0ae540 2020-11-20T12:06:31.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc99b7fe700 1 -- 172.21.15.44:0/3297697192 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00ce90 con 0x7fc9bc0ae540 2020-11-20T12:06:31.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.074+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc1471a0 con 0x7fc9bc0ae540 2020-11-20T12:06:31.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.075+0000 7fc99b7fe700 1 -- 172.21.15.44:0/3297697192 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc9ac012600 con 0x7fc9bc0ae540 2020-11-20T12:06:31.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.075+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc13abe0 con 0x7fc9bc0ae540 2020-11-20T12:06:31.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.076+0000 7fc99b7fe700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc994056cd0 0x7fc994059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.076+0000 7fc99b7fe700 1 -- 172.21.15.44:0/3297697192 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc9ac0705d0 con 0x7fc9bc0ae540 2020-11-20T12:06:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.076+0000 7fc9baffd700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc994056cd0 0x7fc994059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.076+0000 7fc9baffd700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc994056cd0 0x7fc994059190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fc9bc1378f0 tx=0x7fc9b0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:31.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.080+0000 7fc99b7fe700 1 -- 172.21.15.44:0/3297697192 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc9ac045730 con 0x7fc9bc0ae540 2020-11-20T12:06:31.116 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/30 file after the snapshot 2020-11-20T12:06:31.117 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/31 file after the snapshot 2020-11-20T12:06:31.119 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/32 file after the snapshot 2020-11-20T12:06:31.122 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/33 file after the snapshot 2020-11-20T12:06:31.125 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/34 file after the snapshot 2020-11-20T12:06:31.127 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/35 file after the snapshot 2020-11-20T12:06:31.129 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/36 file after the snapshot 2020-11-20T12:06:31.132 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/37 file after the snapshot 2020-11-20T12:06:31.135 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/38 file after the snapshot 2020-11-20T12:06:31.138 INFO:tasks.workunit.client.0.smithi044.stdout: create 280/39 file after the snapshot 2020-11-20T12:06:31.140 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/.snap/snap-subdir-test 2020-11-20T12:06:31.166 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/30 file after the snapshot 2020-11-20T12:06:31.167 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/31 file after the snapshot 2020-11-20T12:06:31.169 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/32 file after the snapshot 2020-11-20T12:06:31.173 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/33 file after the snapshot 2020-11-20T12:06:31.176 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/34 file after the snapshot 2020-11-20T12:06:31.178 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/35 file after the snapshot 2020-11-20T12:06:31.181 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/36 file after the snapshot 2020-11-20T12:06:31.183 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/37 file after the snapshot 2020-11-20T12:06:31.186 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/38 file after the snapshot 2020-11-20T12:06:31.188 INFO:tasks.workunit.client.0.smithi044.stdout: create 281/39 file after the snapshot 2020-11-20T12:06:31.191 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/.snap/snap-subdir-test 2020-11-20T12:06:31.210 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/30 file after the snapshot 2020-11-20T12:06:31.211 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/31 file after the snapshot 2020-11-20T12:06:31.213 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/32 file after the snapshot 2020-11-20T12:06:31.216 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/33 file after the snapshot 2020-11-20T12:06:31.219 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/34 file after the snapshot 2020-11-20T12:06:31.221 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/35 file after the snapshot 2020-11-20T12:06:31.224 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/36 file after the snapshot 2020-11-20T12:06:31.226 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/37 file after the snapshot 2020-11-20T12:06:31.228 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/38 file after the snapshot 2020-11-20T12:06:31.233 INFO:tasks.workunit.client.0.smithi044.stdout: create 282/39 file after the snapshot 2020-11-20T12:06:31.235 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/.snap/snap-subdir-test 2020-11-20T12:06:31.255 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/30 file after the snapshot 2020-11-20T12:06:31.256 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/31 file after the snapshot 2020-11-20T12:06:31.259 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/32 file after the snapshot 2020-11-20T12:06:31.261 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/33 file after the snapshot 2020-11-20T12:06:31.264 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/34 file after the snapshot 2020-11-20T12:06:31.268 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/35 file after the snapshot 2020-11-20T12:06:31.270 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/36 file after the snapshot 2020-11-20T12:06:31.273 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/37 file after the snapshot 2020-11-20T12:06:31.276 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/38 file after the snapshot 2020-11-20T12:06:31.279 INFO:tasks.workunit.client.0.smithi044.stdout: create 283/39 file after the snapshot 2020-11-20T12:06:31.284 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/.snap/snap-subdir-test 2020-11-20T12:06:31.301 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/30 file after the snapshot 2020-11-20T12:06:31.302 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/31 file after the snapshot 2020-11-20T12:06:31.305 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/32 file after the snapshot 2020-11-20T12:06:31.308 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/33 file after the snapshot 2020-11-20T12:06:31.311 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/34 file after the snapshot 2020-11-20T12:06:31.313 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/35 file after the snapshot 2020-11-20T12:06:31.315 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/36 file after the snapshot 2020-11-20T12:06:31.317 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/37 file after the snapshot 2020-11-20T12:06:31.319 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/38 file after the snapshot 2020-11-20T12:06:31.322 INFO:tasks.workunit.client.0.smithi044.stdout: create 284/39 file after the snapshot 2020-11-20T12:06:31.325 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/.snap/snap-subdir-test 2020-11-20T12:06:31.353 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/30 file after the snapshot 2020-11-20T12:06:31.355 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/31 file after the snapshot 2020-11-20T12:06:31.357 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/32 file after the snapshot 2020-11-20T12:06:31.360 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/33 file after the snapshot 2020-11-20T12:06:31.362 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/34 file after the snapshot 2020-11-20T12:06:31.365 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/35 file after the snapshot 2020-11-20T12:06:31.367 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/36 file after the snapshot 2020-11-20T12:06:31.369 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/37 file after the snapshot 2020-11-20T12:06:31.372 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/38 file after the snapshot 2020-11-20T12:06:31.373 INFO:tasks.workunit.client.0.smithi044.stdout: create 285/39 file after the snapshot 2020-11-20T12:06:31.377 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/.snap/snap-subdir-test 2020-11-20T12:06:31.403 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/30 file after the snapshot 2020-11-20T12:06:31.404 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/31 file after the snapshot 2020-11-20T12:06:31.406 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/32 file after the snapshot 2020-11-20T12:06:31.409 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/33 file after the snapshot 2020-11-20T12:06:31.411 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/34 file after the snapshot 2020-11-20T12:06:31.414 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/35 file after the snapshot 2020-11-20T12:06:31.416 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/36 file after the snapshot 2020-11-20T12:06:31.420 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/37 file after the snapshot 2020-11-20T12:06:31.423 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/38 file after the snapshot 2020-11-20T12:06:31.426 INFO:tasks.workunit.client.0.smithi044.stdout: create 286/39 file after the snapshot 2020-11-20T12:06:31.429 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/.snap/snap-subdir-test 2020-11-20T12:06:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.427+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc9bc057d80 con 0x7fc9bc0ae540 2020-11-20T12:06:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.428+0000 7fc99b7fe700 1 -- 172.21.15.44:0/3297697192 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fc9ac048d50 con 0x7fc9bc0ae540 2020-11-20T12:06:31.432 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:06:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc994056cd0 msgr2=0x7fc994059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc994056cd0 0x7fc994059190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fc9bc1378f0 tx=0x7fc9b0009040).stop 2020-11-20T12:06:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc0ae540 msgr2=0x7fc9bc137180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc0ae540 0x7fc9bc137180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc9ac00ebd0 tx=0x7fc9ac006df0).stop 2020-11-20T12:06:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 shutdown_connections 2020-11-20T12:06:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc994056cd0 0x7fc994059190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9bc13b2d0 0x7fc9bc146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc9bc0ae540 0x7fc9bc137180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 --2- 172.21.15.44:0/3297697192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9bc0ae920 0x7fc9bc09ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 >> 172.21.15.44:0/3297697192 conn(0x7fc9bc0654e0 msgr2=0x7fc9bc0addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:31.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 shutdown_connections 2020-11-20T12:06:31.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:31.430+0000 7fc9c0da7700 1 -- 172.21.15.44:0/3297697192 wait complete. 2020-11-20T12:06:31.449 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/30 file after the snapshot 2020-11-20T12:06:31.451 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/31 file after the snapshot 2020-11-20T12:06:31.452 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/32 file after the snapshot 2020-11-20T12:06:31.454 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/33 file after the snapshot 2020-11-20T12:06:31.457 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/34 file after the snapshot 2020-11-20T12:06:31.459 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/35 file after the snapshot 2020-11-20T12:06:31.461 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/36 file after the snapshot 2020-11-20T12:06:31.464 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/37 file after the snapshot 2020-11-20T12:06:31.466 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/38 file after the snapshot 2020-11-20T12:06:31.469 INFO:tasks.workunit.client.0.smithi044.stdout: create 287/39 file after the snapshot 2020-11-20T12:06:31.471 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/.snap/snap-subdir-test 2020-11-20T12:06:31.491 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/30 file after the snapshot 2020-11-20T12:06:31.492 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/31 file after the snapshot 2020-11-20T12:06:31.494 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/32 file after the snapshot 2020-11-20T12:06:31.497 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/33 file after the snapshot 2020-11-20T12:06:31.499 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/34 file after the snapshot 2020-11-20T12:06:31.501 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/35 file after the snapshot 2020-11-20T12:06:31.504 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/36 file after the snapshot 2020-11-20T12:06:31.506 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/37 file after the snapshot 2020-11-20T12:06:31.508 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/38 file after the snapshot 2020-11-20T12:06:31.512 INFO:tasks.workunit.client.0.smithi044.stdout: create 288/39 file after the snapshot 2020-11-20T12:06:31.514 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/.snap/snap-subdir-test 2020-11-20T12:06:31.540 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/30 file after the snapshot 2020-11-20T12:06:31.541 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/31 file after the snapshot 2020-11-20T12:06:31.543 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/32 file after the snapshot 2020-11-20T12:06:31.546 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/33 file after the snapshot 2020-11-20T12:06:31.548 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/34 file after the snapshot 2020-11-20T12:06:31.551 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/35 file after the snapshot 2020-11-20T12:06:31.554 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/36 file after the snapshot 2020-11-20T12:06:31.557 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/37 file after the snapshot 2020-11-20T12:06:31.560 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/38 file after the snapshot 2020-11-20T12:06:31.563 INFO:tasks.workunit.client.0.smithi044.stdout: create 289/39 file after the snapshot 2020-11-20T12:06:31.565 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/.snap/snap-subdir-test 2020-11-20T12:06:31.592 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/30 file after the snapshot 2020-11-20T12:06:31.594 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/31 file after the snapshot 2020-11-20T12:06:31.596 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/32 file after the snapshot 2020-11-20T12:06:31.598 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/33 file after the snapshot 2020-11-20T12:06:31.600 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/34 file after the snapshot 2020-11-20T12:06:31.602 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/35 file after the snapshot 2020-11-20T12:06:31.605 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/36 file after the snapshot 2020-11-20T12:06:31.607 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/37 file after the snapshot 2020-11-20T12:06:31.610 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/38 file after the snapshot 2020-11-20T12:06:31.612 INFO:tasks.workunit.client.0.smithi044.stdout: create 290/39 file after the snapshot 2020-11-20T12:06:31.614 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/.snap/snap-subdir-test 2020-11-20T12:06:31.632 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/30 file after the snapshot 2020-11-20T12:06:31.634 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/31 file after the snapshot 2020-11-20T12:06:31.636 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/32 file after the snapshot 2020-11-20T12:06:31.639 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/33 file after the snapshot 2020-11-20T12:06:31.641 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/34 file after the snapshot 2020-11-20T12:06:31.643 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/35 file after the snapshot 2020-11-20T12:06:31.645 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/36 file after the snapshot 2020-11-20T12:06:31.647 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/37 file after the snapshot 2020-11-20T12:06:31.650 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/38 file after the snapshot 2020-11-20T12:06:31.653 INFO:tasks.workunit.client.0.smithi044.stdout: create 291/39 file after the snapshot 2020-11-20T12:06:31.655 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/.snap/snap-subdir-test 2020-11-20T12:06:31.677 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/30 file after the snapshot 2020-11-20T12:06:31.678 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/31 file after the snapshot 2020-11-20T12:06:31.680 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/32 file after the snapshot 2020-11-20T12:06:31.683 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/33 file after the snapshot 2020-11-20T12:06:31.685 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/34 file after the snapshot 2020-11-20T12:06:31.688 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/35 file after the snapshot 2020-11-20T12:06:31.690 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/36 file after the snapshot 2020-11-20T12:06:31.694 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/37 file after the snapshot 2020-11-20T12:06:31.696 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/38 file after the snapshot 2020-11-20T12:06:31.698 INFO:tasks.workunit.client.0.smithi044.stdout: create 292/39 file after the snapshot 2020-11-20T12:06:31.700 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/.snap/snap-subdir-test 2020-11-20T12:06:31.722 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/30 file after the snapshot 2020-11-20T12:06:31.724 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/31 file after the snapshot 2020-11-20T12:06:31.725 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/32 file after the snapshot 2020-11-20T12:06:31.728 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/33 file after the snapshot 2020-11-20T12:06:31.730 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/34 file after the snapshot 2020-11-20T12:06:31.733 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/35 file after the snapshot 2020-11-20T12:06:31.736 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/36 file after the snapshot 2020-11-20T12:06:31.738 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/37 file after the snapshot 2020-11-20T12:06:31.741 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/38 file after the snapshot 2020-11-20T12:06:31.743 INFO:tasks.workunit.client.0.smithi044.stdout: create 293/39 file after the snapshot 2020-11-20T12:06:31.746 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/.snap/snap-subdir-test 2020-11-20T12:06:31.764 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/30 file after the snapshot 2020-11-20T12:06:31.766 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/31 file after the snapshot 2020-11-20T12:06:31.768 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/32 file after the snapshot 2020-11-20T12:06:31.770 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/33 file after the snapshot 2020-11-20T12:06:31.772 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/34 file after the snapshot 2020-11-20T12:06:31.774 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/35 file after the snapshot 2020-11-20T12:06:31.777 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/36 file after the snapshot 2020-11-20T12:06:31.779 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/37 file after the snapshot 2020-11-20T12:06:31.781 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/38 file after the snapshot 2020-11-20T12:06:31.784 INFO:tasks.workunit.client.0.smithi044.stdout: create 294/39 file after the snapshot 2020-11-20T12:06:31.786 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/.snap/snap-subdir-test 2020-11-20T12:06:31.805 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/30 file after the snapshot 2020-11-20T12:06:31.806 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/31 file after the snapshot 2020-11-20T12:06:31.808 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/32 file after the snapshot 2020-11-20T12:06:31.811 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/33 file after the snapshot 2020-11-20T12:06:31.814 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/34 file after the snapshot 2020-11-20T12:06:31.817 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/35 file after the snapshot 2020-11-20T12:06:31.819 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/36 file after the snapshot 2020-11-20T12:06:31.821 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/37 file after the snapshot 2020-11-20T12:06:31.825 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/38 file after the snapshot 2020-11-20T12:06:31.827 INFO:tasks.workunit.client.0.smithi044.stdout: create 295/39 file after the snapshot 2020-11-20T12:06:31.829 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/.snap/snap-subdir-test 2020-11-20T12:06:31.852 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/30 file after the snapshot 2020-11-20T12:06:31.854 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/31 file after the snapshot 2020-11-20T12:06:31.856 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/32 file after the snapshot 2020-11-20T12:06:31.860 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/33 file after the snapshot 2020-11-20T12:06:31.862 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/34 file after the snapshot 2020-11-20T12:06:31.864 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/35 file after the snapshot 2020-11-20T12:06:31.867 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/36 file after the snapshot 2020-11-20T12:06:31.869 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/37 file after the snapshot 2020-11-20T12:06:31.871 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/38 file after the snapshot 2020-11-20T12:06:31.874 INFO:tasks.workunit.client.0.smithi044.stdout: create 296/39 file after the snapshot 2020-11-20T12:06:31.876 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/.snap/snap-subdir-test 2020-11-20T12:06:31.905 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/30 file after the snapshot 2020-11-20T12:06:31.905 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/31 file after the snapshot 2020-11-20T12:06:31.905 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/32 file after the snapshot 2020-11-20T12:06:31.908 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/33 file after the snapshot 2020-11-20T12:06:31.911 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/34 file after the snapshot 2020-11-20T12:06:31.913 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/35 file after the snapshot 2020-11-20T12:06:31.915 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/36 file after the snapshot 2020-11-20T12:06:31.918 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/37 file after the snapshot 2020-11-20T12:06:31.920 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/38 file after the snapshot 2020-11-20T12:06:31.922 INFO:tasks.workunit.client.0.smithi044.stdout: create 297/39 file after the snapshot 2020-11-20T12:06:31.925 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/.snap/snap-subdir-test 2020-11-20T12:06:31.951 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/30 file after the snapshot 2020-11-20T12:06:32.016 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/31 file after the snapshot 2020-11-20T12:06:32.019 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/32 file after the snapshot 2020-11-20T12:06:32.022 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/33 file after the snapshot 2020-11-20T12:06:32.024 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/34 file after the snapshot 2020-11-20T12:06:32.026 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/35 file after the snapshot 2020-11-20T12:06:32.028 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/36 file after the snapshot 2020-11-20T12:06:32.030 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/37 file after the snapshot 2020-11-20T12:06:32.032 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/38 file after the snapshot 2020-11-20T12:06:32.034 INFO:tasks.workunit.client.0.smithi044.stdout: create 298/39 file after the snapshot 2020-11-20T12:06:32.036 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/.snap/snap-subdir-test 2020-11-20T12:06:32.095 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/30 file after the snapshot 2020-11-20T12:06:32.095 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/31 file after the snapshot 2020-11-20T12:06:32.097 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/32 file after the snapshot 2020-11-20T12:06:32.099 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/33 file after the snapshot 2020-11-20T12:06:32.102 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/34 file after the snapshot 2020-11-20T12:06:32.104 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/35 file after the snapshot 2020-11-20T12:06:32.107 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/36 file after the snapshot 2020-11-20T12:06:32.109 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/37 file after the snapshot 2020-11-20T12:06:32.111 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/38 file after the snapshot 2020-11-20T12:06:32.114 INFO:tasks.workunit.client.0.smithi044.stdout: create 299/39 file after the snapshot 2020-11-20T12:06:32.116 INFO:tasks.workunit.client.0.smithi044.stdout:Take second snapshot .snap/test2 2020-11-20T12:06:32.263 INFO:tasks.workunit.client.0.smithi044.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2020-11-20T12:06:33.586 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2020-11-20T12:06:33.586 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2020-11-20T12:06:33.587 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2020-11-20T12:06:33.587 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2020-11-20T12:06:33.587 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2020-11-20T12:06:33.588 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2020-11-20T12:06:33.588 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2020-11-20T12:06:33.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2020-11-20T12:06:33.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2020-11-20T12:06:33.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2020-11-20T12:06:33.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2020-11-20T12:06:33.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2020-11-20T12:06:33.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2020-11-20T12:06:33.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2020-11-20T12:06:33.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2020-11-20T12:06:33.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2020-11-20T12:06:33.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2020-11-20T12:06:33.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2020-11-20T12:06:33.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2020-11-20T12:06:33.593 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2020-11-20T12:06:33.593 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2020-11-20T12:06:33.593 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2020-11-20T12:06:33.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2020-11-20T12:06:33.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2020-11-20T12:06:33.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2020-11-20T12:06:33.595 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2020-11-20T12:06:33.595 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2020-11-20T12:06:33.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2020-11-20T12:06:33.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2020-11-20T12:06:33.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2020-11-20T12:06:33.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2020-11-20T12:06:33.597 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2020-11-20T12:06:33.597 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2020-11-20T12:06:33.597 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2020-11-20T12:06:33.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2020-11-20T12:06:33.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2020-11-20T12:06:33.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2020-11-20T12:06:33.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2020-11-20T12:06:33.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2020-11-20T12:06:33.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2020-11-20T12:06:33.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2020-11-20T12:06:33.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2020-11-20T12:06:33.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2020-11-20T12:06:33.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2020-11-20T12:06:33.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2020-11-20T12:06:33.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2020-11-20T12:06:33.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2020-11-20T12:06:33.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2020-11-20T12:06:33.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2020-11-20T12:06:33.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2020-11-20T12:06:33.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2020-11-20T12:06:33.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2020-11-20T12:06:33.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2020-11-20T12:06:33.604 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2020-11-20T12:06:33.604 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2020-11-20T12:06:33.604 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2020-11-20T12:06:33.604 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2020-11-20T12:06:33.605 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2020-11-20T12:06:33.605 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2020-11-20T12:06:33.605 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2020-11-20T12:06:33.606 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2020-11-20T12:06:33.606 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2020-11-20T12:06:33.606 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2020-11-20T12:06:33.607 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2020-11-20T12:06:33.607 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2020-11-20T12:06:33.607 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2020-11-20T12:06:33.608 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2020-11-20T12:06:33.608 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2020-11-20T12:06:33.608 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2020-11-20T12:06:33.608 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2020-11-20T12:06:33.609 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2020-11-20T12:06:33.609 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2020-11-20T12:06:33.609 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2020-11-20T12:06:33.610 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2020-11-20T12:06:33.610 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2020-11-20T12:06:33.610 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2020-11-20T12:06:33.611 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2020-11-20T12:06:33.611 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2020-11-20T12:06:33.611 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2020-11-20T12:06:33.611 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2020-11-20T12:06:33.612 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2020-11-20T12:06:34.446 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2020-11-20T12:06:34.446 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2020-11-20T12:06:34.447 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2020-11-20T12:06:34.447 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2020-11-20T12:06:34.447 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2020-11-20T12:06:34.448 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2020-11-20T12:06:34.448 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2020-11-20T12:06:34.449 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2020-11-20T12:06:34.449 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2020-11-20T12:06:34.449 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2020-11-20T12:06:34.450 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2020-11-20T12:06:34.450 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2020-11-20T12:06:34.451 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2020-11-20T12:06:34.451 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2020-11-20T12:06:34.451 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2020-11-20T12:06:34.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2020-11-20T12:06:34.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2020-11-20T12:06:34.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2020-11-20T12:06:34.453 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2020-11-20T12:06:34.453 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2020-11-20T12:06:34.453 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2020-11-20T12:06:34.454 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2020-11-20T12:06:34.454 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2020-11-20T12:06:34.455 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2020-11-20T12:06:34.455 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2020-11-20T12:06:34.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2020-11-20T12:06:34.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2020-11-20T12:06:34.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2020-11-20T12:06:34.457 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2020-11-20T12:06:34.457 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2020-11-20T12:06:34.458 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2020-11-20T12:06:34.458 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2020-11-20T12:06:34.458 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2020-11-20T12:06:34.459 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2020-11-20T12:06:34.459 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2020-11-20T12:06:34.460 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2020-11-20T12:06:34.460 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2020-11-20T12:06:34.461 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2020-11-20T12:06:34.461 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2020-11-20T12:06:34.461 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2020-11-20T12:06:34.462 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2020-11-20T12:06:34.462 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2020-11-20T12:06:34.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2020-11-20T12:06:34.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2020-11-20T12:06:34.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2020-11-20T12:06:34.464 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2020-11-20T12:06:34.464 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2020-11-20T12:06:34.465 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2020-11-20T12:06:34.465 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2020-11-20T12:06:34.465 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2020-11-20T12:06:34.466 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2020-11-20T12:06:34.466 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2020-11-20T12:06:34.467 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2020-11-20T12:06:34.467 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2020-11-20T12:06:34.467 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2020-11-20T12:06:34.468 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2020-11-20T12:06:34.468 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2020-11-20T12:06:34.469 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2020-11-20T12:06:34.469 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2020-11-20T12:06:34.469 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2020-11-20T12:06:34.470 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2020-11-20T12:06:34.470 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2020-11-20T12:06:34.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2020-11-20T12:06:34.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2020-11-20T12:06:34.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2020-11-20T12:06:34.472 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2020-11-20T12:06:34.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2020-11-20T12:06:34.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2020-11-20T12:06:34.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2020-11-20T12:06:34.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2020-11-20T12:06:34.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2020-11-20T12:06:34.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2020-11-20T12:06:34.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2020-11-20T12:06:34.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2020-11-20T12:06:34.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2020-11-20T12:06:34.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2020-11-20T12:06:34.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2020-11-20T12:06:34.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2020-11-20T12:06:34.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2020-11-20T12:06:34.476 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2020-11-20T12:06:34.476 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2020-11-20T12:06:34.477 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:06:34.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.634+0000 7ff030454700 1 Processor -- start 2020-11-20T12:06:34.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.634+0000 7ff030454700 1 -- start start 2020-11-20T12:06:34.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.634+0000 7ff030454700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0280aad00 0x7ff0280a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:34.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.634+0000 7ff030454700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff0280a86d0 0x7ff02813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff030454700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0281385f0 con 0x7ff0280ab0e0 2020-11-20T12:06:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff030454700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02813e250 con 0x7ff0280aad00 2020-11-20T12:06:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff030454700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02813e4f0 con 0x7ff0280a86d0 2020-11-20T12:06:34.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e1f0700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7ff0280aad00 msgr2=0x7ff0280a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:34.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e1f0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0280aad00 0x7ff0280a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:34.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e9f1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff0280a86d0 0x7ff02813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:34.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02d9ef700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff0280ab0e0 0x7ff0280abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44720/0 (socket says 172.21.15.44:44720) 2020-11-20T12:06:34.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02d9ef700 1 -- 172.21.15.44:0/564994888 learned_addr learned my addr 172.21.15.44:0/564994888 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:06:34.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e9f1700 1 -- 172.21.15.44:0/564994888 >> v2:172.21.15.44:3300/0 conn(0x7ff0280aad00 msgr2=0x7ff0280a94d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:34.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e9f1700 1 --2- 172.21.15.44:0/564994888 >> v2:172.21.15.44:3300/0 conn(0x7ff0280aad00 0x7ff0280a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:34.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e9f1700 1 -- 172.21.15.44:0/564994888 >> v1:172.21.15.79:6789/0 conn(0x7ff0280ab0e0 legacy=0x7ff0280abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:34.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e9f1700 1 -- 172.21.15.44:0/564994888 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0280ac340 con 0x7ff0280a86d0 2020-11-20T12:06:34.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.635+0000 7ff02e9f1700 1 --2- 172.21.15.44:0/564994888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280a86d0 0x7ff02813ec30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff0280a9d40 tx=0x7ff0200091d0).ready entity=mon.1 client_cookie=7f037e2900933d61 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:34.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff02d1ee700 1 -- 172.21.15.44:0/564994888 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02000bbc0 con 0x7ff0280a86d0 2020-11-20T12:06:34.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff02d1ee700 1 -- 172.21.15.44:0/564994888 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02000bd20 con 0x7ff0280a86d0 2020-11-20T12:06:34.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff02d1ee700 1 -- 172.21.15.44:0/564994888 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020011610 con 0x7ff0280a86d0 2020-11-20T12:06:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 -- 172.21.15.44:0/564994888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280a86d0 msgr2=0x7ff02813ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:34.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 --2- 172.21.15.44:0/564994888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280a86d0 0x7ff02813ec30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff0280a9d40 tx=0x7ff0200091d0).stop 2020-11-20T12:06:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 -- 172.21.15.44:0/564994888 shutdown_connections 2020-11-20T12:06:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 --2- 172.21.15.44:0/564994888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280a86d0 0x7ff02813ec30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 --2- 172.21.15.44:0/564994888 >> v2:172.21.15.44:3300/0 conn(0x7ff0280aad00 0x7ff0280a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:34.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 -- 172.21.15.44:0/564994888 >> 172.21.15.44:0/564994888 conn(0x7ff028060170 msgr2=0x7ff028060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 -- 172.21.15.44:0/564994888 shutdown_connections 2020-11-20T12:06:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.636+0000 7ff030454700 1 -- 172.21.15.44:0/564994888 wait complete. 2020-11-20T12:06:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.638+0000 7ff030454700 1 Processor -- start 2020-11-20T12:06:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.638+0000 7ff030454700 1 -- start start 2020-11-20T12:06:34.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff030454700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0280a86d0 0x7ff0280a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff030454700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280aad00 0x7ff02814db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff030454700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0280ab0e0 0x7ff02814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff030454700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02814e520 con 0x7ff0280a86d0 2020-11-20T12:06:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff030454700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02809eab0 con 0x7ff0280aad00 2020-11-20T12:06:34.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff030454700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02809ed50 con 0x7ff0280ab0e0 2020-11-20T12:06:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02e1f0700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0280a86d0 msgr2=0x7ff0280a7780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02e1f0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0280a86d0 0x7ff0280a7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280aad00 0x7ff02814db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02e9f1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0280ab0e0 0x7ff02814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:34.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280aad00 0x7ff02814db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58384/0 (socket says 172.21.15.44:58384) 2020-11-20T12:06:34.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02e9f1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0280ab0e0 0x7ff02814e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49542/0 (socket says 172.21.15.44:49542) 2020-11-20T12:06:34.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 -- 172.21.15.44:0/255487736 learned_addr learned my addr 172.21.15.44:0/255487736 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:34.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 -- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0280ab0e0 msgr2=0x7ff02814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:34.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0280ab0e0 0x7ff02814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:34.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 -- 172.21.15.44:0/255487736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0280a86d0 msgr2=0x7ff0280a7780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0280a86d0 0x7ff0280a7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02d9ef700 1 -- 172.21.15.44:0/255487736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02814d6a0 con 0x7ff0280aad00 2020-11-20T12:06:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.639+0000 7ff02e9f1700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0280ab0e0 0x7ff02814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.640+0000 7ff02d9ef700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280aad00 0x7ff02814db40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ff01800c740 tx=0x7ff018007ca0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:34.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.640+0000 7ff0167fc700 1 -- 172.21.15.44:0/255487736 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0180072e0 con 0x7ff0280aad00 2020-11-20T12:06:34.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.640+0000 7ff0167fc700 1 -- 172.21.15.44:0/255487736 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018007440 con 0x7ff0280aad00 2020-11-20T12:06:34.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.640+0000 7ff0167fc700 1 -- 172.21.15.44:0/255487736 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018005800 con 0x7ff0280aad00 2020-11-20T12:06:34.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.640+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02813add0 con 0x7ff0280aad00 2020-11-20T12:06:34.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.640+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02809e7e0 con 0x7ff0280aad00 2020-11-20T12:06:34.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.641+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff028069680 con 0x7ff0280aad00 2020-11-20T12:06:34.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.642+0000 7ff0167fc700 1 -- 172.21.15.44:0/255487736 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff018010070 con 0x7ff0280aad00 2020-11-20T12:06:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.642+0000 7ff0167fc700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056cd0 0x7ff000059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.642+0000 7ff0167fc700 1 -- 172.21.15.44:0/255487736 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff01806f9e0 con 0x7ff0280aad00 2020-11-20T12:06:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.643+0000 7ff02e1f0700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056cd0 0x7ff000059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.643+0000 7ff02e1f0700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056cd0 0x7ff000059190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff0240036a0 tx=0x7ff024006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:34.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.646+0000 7ff0167fc700 1 -- 172.21.15.44:0/255487736 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff018044d30 con 0x7ff0280aad00 2020-11-20T12:06:34.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.976+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff0280a80d0 con 0x7ff0280aad00 2020-11-20T12:06:34.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.976+0000 7ff0167fc700 1 -- 172.21.15.44:0/255487736 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7ff018007de0 con 0x7ff0280aad00 2020-11-20T12:06:34.980 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:06:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056cd0 msgr2=0x7ff000059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056cd0 0x7ff000059190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff0240036a0 tx=0x7ff024006ae0).stop 2020-11-20T12:06:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280aad00 msgr2=0x7ff02814db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:34.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280aad00 0x7ff02814db40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ff01800c740 tx=0x7ff018007ca0).stop 2020-11-20T12:06:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 shutdown_connections 2020-11-20T12:06:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056cd0 0x7ff000059190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0280ab0e0 0x7ff02814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0280a86d0 0x7ff0280a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:34.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 --2- 172.21.15.44:0/255487736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0280aad00 0x7ff02814db40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 >> 172.21.15.44:0/255487736 conn(0x7ff028060170 msgr2=0x7ff028137a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 shutdown_connections 2020-11-20T12:06:34.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:34.978+0000 7ff030454700 1 -- 172.21.15.44:0/255487736 wait complete. 2020-11-20T12:06:35.352 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2020-11-20T12:06:35.353 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2020-11-20T12:06:35.353 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2020-11-20T12:06:35.353 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2020-11-20T12:06:35.354 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2020-11-20T12:06:35.354 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2020-11-20T12:06:35.354 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2020-11-20T12:06:35.354 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2020-11-20T12:06:35.355 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2020-11-20T12:06:35.355 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2020-11-20T12:06:35.355 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2020-11-20T12:06:35.355 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2020-11-20T12:06:35.356 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2020-11-20T12:06:35.356 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2020-11-20T12:06:35.356 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2020-11-20T12:06:35.356 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2020-11-20T12:06:35.357 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2020-11-20T12:06:35.357 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2020-11-20T12:06:35.357 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2020-11-20T12:06:35.357 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2020-11-20T12:06:35.358 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2020-11-20T12:06:35.358 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2020-11-20T12:06:35.358 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2020-11-20T12:06:35.358 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2020-11-20T12:06:35.359 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2020-11-20T12:06:35.359 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2020-11-20T12:06:35.359 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2020-11-20T12:06:35.359 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2020-11-20T12:06:35.360 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2020-11-20T12:06:35.360 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2020-11-20T12:06:35.360 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2020-11-20T12:06:35.360 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2020-11-20T12:06:35.360 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2020-11-20T12:06:35.361 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2020-11-20T12:06:35.361 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2020-11-20T12:06:35.361 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2020-11-20T12:06:35.361 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2020-11-20T12:06:35.361 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2020-11-20T12:06:35.362 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2020-11-20T12:06:35.362 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2020-11-20T12:06:35.362 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2020-11-20T12:06:35.362 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2020-11-20T12:06:35.362 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2020-11-20T12:06:35.363 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2020-11-20T12:06:35.363 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2020-11-20T12:06:35.363 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2020-11-20T12:06:35.363 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2020-11-20T12:06:35.364 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2020-11-20T12:06:35.364 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2020-11-20T12:06:35.364 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2020-11-20T12:06:35.364 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2020-11-20T12:06:35.365 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2020-11-20T12:06:35.365 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2020-11-20T12:06:35.365 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2020-11-20T12:06:35.366 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2020-11-20T12:06:35.366 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2020-11-20T12:06:35.366 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2020-11-20T12:06:35.366 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2020-11-20T12:06:35.366 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2020-11-20T12:06:35.367 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2020-11-20T12:06:35.367 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2020-11-20T12:06:35.367 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2020-11-20T12:06:35.368 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2020-11-20T12:06:35.368 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2020-11-20T12:06:35.368 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2020-11-20T12:06:35.368 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2020-11-20T12:06:35.369 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2020-11-20T12:06:35.369 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2020-11-20T12:06:35.369 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2020-11-20T12:06:35.370 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2020-11-20T12:06:35.370 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2020-11-20T12:06:35.370 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2020-11-20T12:06:35.370 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2020-11-20T12:06:35.371 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2020-11-20T12:06:35.371 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2020-11-20T12:06:35.371 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2020-11-20T12:06:35.371 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2020-11-20T12:06:35.371 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2020-11-20T12:06:35.372 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2020-11-20T12:06:35.372 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2020-11-20T12:06:35.372 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2020-11-20T12:06:35.948 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:06:35.950 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:06:36.280 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2020-11-20T12:06:36.280 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2020-11-20T12:06:36.280 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2020-11-20T12:06:36.281 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2020-11-20T12:06:36.281 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2020-11-20T12:06:36.281 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2020-11-20T12:06:36.281 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2020-11-20T12:06:36.281 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2020-11-20T12:06:36.282 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2020-11-20T12:06:36.282 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2020-11-20T12:06:36.282 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2020-11-20T12:06:36.282 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2020-11-20T12:06:36.283 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2020-11-20T12:06:36.283 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2020-11-20T12:06:36.283 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2020-11-20T12:06:36.283 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2020-11-20T12:06:36.284 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2020-11-20T12:06:36.284 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2020-11-20T12:06:36.285 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2020-11-20T12:06:36.285 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2020-11-20T12:06:36.285 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2020-11-20T12:06:36.286 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2020-11-20T12:06:36.286 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2020-11-20T12:06:36.287 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2020-11-20T12:06:36.287 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2020-11-20T12:06:36.287 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2020-11-20T12:06:36.288 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2020-11-20T12:06:36.288 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2020-11-20T12:06:36.289 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2020-11-20T12:06:36.289 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2020-11-20T12:06:36.289 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2020-11-20T12:06:36.290 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2020-11-20T12:06:36.290 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2020-11-20T12:06:36.291 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2020-11-20T12:06:36.291 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2020-11-20T12:06:36.291 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2020-11-20T12:06:36.292 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2020-11-20T12:06:36.292 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2020-11-20T12:06:36.293 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2020-11-20T12:06:36.293 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2020-11-20T12:06:36.293 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2020-11-20T12:06:36.294 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2020-11-20T12:06:36.294 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2020-11-20T12:06:36.295 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2020-11-20T12:06:36.295 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2020-11-20T12:06:36.295 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2020-11-20T12:06:36.296 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2020-11-20T12:06:36.296 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2020-11-20T12:06:36.297 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2020-11-20T12:06:36.297 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2020-11-20T12:06:36.297 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2020-11-20T12:06:36.298 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2020-11-20T12:06:36.298 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2020-11-20T12:06:36.299 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2020-11-20T12:06:36.299 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2020-11-20T12:06:36.299 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2020-11-20T12:06:36.300 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2020-11-20T12:06:36.300 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2020-11-20T12:06:36.300 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2020-11-20T12:06:36.300 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2020-11-20T12:06:36.301 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2020-11-20T12:06:36.301 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2020-11-20T12:06:36.301 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2020-11-20T12:06:36.301 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2020-11-20T12:06:36.301 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2020-11-20T12:06:36.302 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2020-11-20T12:06:36.302 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2020-11-20T12:06:36.302 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2020-11-20T12:06:36.302 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2020-11-20T12:06:36.302 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2020-11-20T12:06:36.303 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2020-11-20T12:06:36.303 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2020-11-20T12:06:36.303 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2020-11-20T12:06:36.303 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2020-11-20T12:06:36.304 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2020-11-20T12:06:36.304 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2020-11-20T12:06:36.304 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2020-11-20T12:06:36.304 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2020-11-20T12:06:36.304 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2020-11-20T12:06:36.305 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2020-11-20T12:06:36.305 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2020-11-20T12:06:36.988 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2020-11-20T12:06:36.989 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2020-11-20T12:06:36.989 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2020-11-20T12:06:36.989 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2020-11-20T12:06:36.989 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2020-11-20T12:06:36.989 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2020-11-20T12:06:36.990 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2020-11-20T12:06:36.990 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2020-11-20T12:06:36.990 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2020-11-20T12:06:36.990 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2020-11-20T12:06:36.991 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2020-11-20T12:06:36.991 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2020-11-20T12:06:36.991 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2020-11-20T12:06:36.991 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2020-11-20T12:06:36.991 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2020-11-20T12:06:36.992 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2020-11-20T12:06:36.992 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2020-11-20T12:06:36.992 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2020-11-20T12:06:36.992 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2020-11-20T12:06:36.992 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2020-11-20T12:06:36.993 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2020-11-20T12:06:36.993 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2020-11-20T12:06:36.993 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2020-11-20T12:06:36.993 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2020-11-20T12:06:36.993 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2020-11-20T12:06:36.994 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2020-11-20T12:06:36.994 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2020-11-20T12:06:36.994 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2020-11-20T12:06:36.994 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2020-11-20T12:06:36.995 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2020-11-20T12:06:36.995 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2020-11-20T12:06:36.995 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2020-11-20T12:06:36.995 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2020-11-20T12:06:36.995 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2020-11-20T12:06:36.996 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2020-11-20T12:06:36.996 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2020-11-20T12:06:36.996 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2020-11-20T12:06:36.996 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2020-11-20T12:06:36.996 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2020-11-20T12:06:36.997 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2020-11-20T12:06:36.997 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2020-11-20T12:06:36.997 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2020-11-20T12:06:36.997 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2020-11-20T12:06:36.997 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2020-11-20T12:06:36.998 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2020-11-20T12:06:36.998 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2020-11-20T12:06:36.998 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2020-11-20T12:06:36.998 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2020-11-20T12:06:36.998 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2020-11-20T12:06:36.999 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2020-11-20T12:06:36.999 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2020-11-20T12:06:36.999 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2020-11-20T12:06:36.999 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2020-11-20T12:06:37.000 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2020-11-20T12:06:37.000 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2020-11-20T12:06:37.000 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2020-11-20T12:06:37.000 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2020-11-20T12:06:37.000 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2020-11-20T12:06:37.001 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2020-11-20T12:06:37.001 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2020-11-20T12:06:37.001 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2020-11-20T12:06:37.001 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2020-11-20T12:06:37.001 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2020-11-20T12:06:37.002 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2020-11-20T12:06:37.002 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2020-11-20T12:06:37.002 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2020-11-20T12:06:37.002 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2020-11-20T12:06:37.002 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2020-11-20T12:06:37.003 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2020-11-20T12:06:37.003 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2020-11-20T12:06:37.003 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2020-11-20T12:06:37.003 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2020-11-20T12:06:37.003 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2020-11-20T12:06:37.004 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2020-11-20T12:06:37.004 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2020-11-20T12:06:37.004 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2020-11-20T12:06:37.004 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2020-11-20T12:06:37.004 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2020-11-20T12:06:37.005 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2020-11-20T12:06:37.005 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2020-11-20T12:06:37.005 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2020-11-20T12:06:37.826 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2020-11-20T12:06:37.826 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2020-11-20T12:06:37.826 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2020-11-20T12:06:37.827 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2020-11-20T12:06:37.827 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2020-11-20T12:06:37.827 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2020-11-20T12:06:37.827 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2020-11-20T12:06:37.827 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2020-11-20T12:06:37.828 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2020-11-20T12:06:37.828 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2020-11-20T12:06:37.828 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2020-11-20T12:06:37.828 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2020-11-20T12:06:37.829 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2020-11-20T12:06:37.829 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2020-11-20T12:06:37.829 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2020-11-20T12:06:37.829 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2020-11-20T12:06:37.830 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2020-11-20T12:06:37.830 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2020-11-20T12:06:37.830 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2020-11-20T12:06:37.830 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2020-11-20T12:06:37.831 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2020-11-20T12:06:37.831 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2020-11-20T12:06:37.831 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2020-11-20T12:06:37.831 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2020-11-20T12:06:37.831 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2020-11-20T12:06:37.832 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2020-11-20T12:06:37.832 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2020-11-20T12:06:37.832 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2020-11-20T12:06:37.832 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2020-11-20T12:06:37.832 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2020-11-20T12:06:37.833 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2020-11-20T12:06:37.833 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2020-11-20T12:06:37.833 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2020-11-20T12:06:37.833 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2020-11-20T12:06:37.833 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2020-11-20T12:06:37.834 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2020-11-20T12:06:37.834 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2020-11-20T12:06:37.834 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2020-11-20T12:06:37.834 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2020-11-20T12:06:37.834 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2020-11-20T12:06:37.835 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2020-11-20T12:06:37.835 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2020-11-20T12:06:37.835 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2020-11-20T12:06:37.835 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2020-11-20T12:06:37.835 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2020-11-20T12:06:37.836 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2020-11-20T12:06:37.836 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2020-11-20T12:06:37.836 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2020-11-20T12:06:37.836 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2020-11-20T12:06:37.836 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2020-11-20T12:06:37.837 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2020-11-20T12:06:37.837 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2020-11-20T12:06:37.837 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2020-11-20T12:06:37.837 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2020-11-20T12:06:37.837 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2020-11-20T12:06:37.838 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2020-11-20T12:06:37.838 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2020-11-20T12:06:37.838 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2020-11-20T12:06:37.838 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2020-11-20T12:06:37.838 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2020-11-20T12:06:37.839 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2020-11-20T12:06:37.839 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2020-11-20T12:06:37.839 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2020-11-20T12:06:37.839 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2020-11-20T12:06:37.839 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2020-11-20T12:06:37.840 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2020-11-20T12:06:37.840 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2020-11-20T12:06:37.840 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2020-11-20T12:06:37.840 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2020-11-20T12:06:37.840 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2020-11-20T12:06:37.841 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2020-11-20T12:06:37.841 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2020-11-20T12:06:37.841 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2020-11-20T12:06:37.841 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2020-11-20T12:06:37.841 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2020-11-20T12:06:37.842 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2020-11-20T12:06:37.842 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2020-11-20T12:06:37.842 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2020-11-20T12:06:37.842 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2020-11-20T12:06:37.842 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2020-11-20T12:06:37.843 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2020-11-20T12:06:37.995 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:06:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.148+0000 7fe7b86a8700 1 Processor -- start 2020-11-20T12:06:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.148+0000 7fe7b86a8700 1 -- start start 2020-11-20T12:06:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b86a8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b86a8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b86a8700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b0136380 con 0x7fe7b00adec0 2020-11-20T12:06:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b86a8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b00607a0 con 0x7fe7b00a8170 2020-11-20T12:06:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b86a8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b0060a40 con 0x7fe7b00ae2a0 2020-11-20T12:06:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b6444700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fe7b00adec0 legacy=0x7fe7b00a7a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:06:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b6c45700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b5c43700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.149+0000 7fe7b6c45700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58394/0 (socket says 172.21.15.44:58394) 2020-11-20T12:06:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 learned_addr learned my addr 172.21.15.44:0/1454180919 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:06:38.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:38.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.150+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:38.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.350+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> v1:172.21.15.44:6789/0 conn(0x7fe7b00adec0 legacy=0x7fe7b00a7a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:06:38.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:38.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:38.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:38.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:06:38.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:38.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:38.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:38.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:38.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:06:38.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.351+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:06:38.587 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2020-11-20T12:06:38.587 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2020-11-20T12:06:38.587 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2020-11-20T12:06:38.588 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2020-11-20T12:06:38.588 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2020-11-20T12:06:38.588 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2020-11-20T12:06:38.588 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2020-11-20T12:06:38.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2020-11-20T12:06:38.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2020-11-20T12:06:38.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2020-11-20T12:06:38.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2020-11-20T12:06:38.589 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2020-11-20T12:06:38.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2020-11-20T12:06:38.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2020-11-20T12:06:38.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2020-11-20T12:06:38.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2020-11-20T12:06:38.590 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2020-11-20T12:06:38.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2020-11-20T12:06:38.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2020-11-20T12:06:38.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2020-11-20T12:06:38.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2020-11-20T12:06:38.591 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2020-11-20T12:06:38.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2020-11-20T12:06:38.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2020-11-20T12:06:38.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2020-11-20T12:06:38.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2020-11-20T12:06:38.592 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2020-11-20T12:06:38.593 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2020-11-20T12:06:38.593 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2020-11-20T12:06:38.593 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2020-11-20T12:06:38.593 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2020-11-20T12:06:38.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2020-11-20T12:06:38.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2020-11-20T12:06:38.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2020-11-20T12:06:38.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2020-11-20T12:06:38.594 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2020-11-20T12:06:38.595 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2020-11-20T12:06:38.595 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2020-11-20T12:06:38.595 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2020-11-20T12:06:38.595 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2020-11-20T12:06:38.595 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2020-11-20T12:06:38.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2020-11-20T12:06:38.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2020-11-20T12:06:38.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2020-11-20T12:06:38.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2020-11-20T12:06:38.596 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2020-11-20T12:06:38.597 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2020-11-20T12:06:38.597 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2020-11-20T12:06:38.597 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2020-11-20T12:06:38.597 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2020-11-20T12:06:38.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2020-11-20T12:06:38.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2020-11-20T12:06:38.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2020-11-20T12:06:38.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2020-11-20T12:06:38.598 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2020-11-20T12:06:38.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2020-11-20T12:06:38.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2020-11-20T12:06:38.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2020-11-20T12:06:38.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2020-11-20T12:06:38.599 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2020-11-20T12:06:38.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2020-11-20T12:06:38.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2020-11-20T12:06:38.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2020-11-20T12:06:38.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2020-11-20T12:06:38.600 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2020-11-20T12:06:38.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2020-11-20T12:06:38.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2020-11-20T12:06:38.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2020-11-20T12:06:38.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2020-11-20T12:06:38.601 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2020-11-20T12:06:38.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2020-11-20T12:06:38.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2020-11-20T12:06:38.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2020-11-20T12:06:38.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2020-11-20T12:06:38.602 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2020-11-20T12:06:38.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2020-11-20T12:06:38.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2020-11-20T12:06:38.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2020-11-20T12:06:38.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2020-11-20T12:06:38.603 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2020-11-20T12:06:38.604 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2020-11-20T12:06:38.604 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2020-11-20T12:06:38.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.751+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> v1:172.21.15.44:6789/0 conn(0x7fe7b00adec0 legacy=0x7fe7b00a7a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:06:38.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:38.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:06:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:06:38.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:38.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:38.752+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:06:39.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.552+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> v1:172.21.15.44:6789/0 conn(0x7fe7b00adec0 legacy=0x7fe7b00a7a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:06:39.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.553+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:39.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.553+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:06:39.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.553+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:39.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.553+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:39.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.553+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:06:39.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.553+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:39.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.554+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:06:39.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.554+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:06:39.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.554+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:06:39.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:39.554+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:06:41.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 msgr2=0x7fe7b00a9520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:41.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe7b00ae2a0 0x7fe7b00a9520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:41.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 msgr2=0x7fe7b0138960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:41.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.79:3300/0 conn(0x7fe7b00a8170 0x7fe7b0138960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:41.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 -- 172.21.15.44:0/1454180919 >> v1:172.21.15.44:6789/0 conn(0x7fe7b00adec0 legacy=0x7fe7b00a7a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:41.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 0x7fe7900057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 -- 172.21.15.44:0/1454180919 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b0060ce0 con 0x7fe790005e80 2020-11-20T12:06:41.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7a3fff700 1 -- 172.21.15.44:0/1454180919 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b00607a0 con 0x7fe7900033b0 2020-11-20T12:06:41.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.149+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 msgr2=0x7fe7900057f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:41.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.150+0000 7fe7b6444700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 0x7fe7900057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:41.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.351+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 msgr2=0x7fe7900057f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:41.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.351+0000 7fe7b6444700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 0x7fe7900057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:06:41.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.752+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 msgr2=0x7fe7900057f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:41.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.752+0000 7fe7b6444700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 0x7fe7900057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:06:41.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.759+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408008530 0 0) 0x7fe7b0060ce0 con 0x7fe790005e80 2020-11-20T12:06:41.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.759+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe798003240 con 0x7fe790005e80 2020-11-20T12:06:41.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3013317037 0 0) 0x7fe798003240 con 0x7fe790005e80 2020-11-20T12:06:41.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 msgr2=0x7fe7900057f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:41.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 --2- 172.21.15.44:0/1454180919 >> v2:172.21.15.44:3300/0 conn(0x7fe7900033b0 0x7fe7900057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:41.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b0136380 con 0x7fe790005e80 2020-11-20T12:06:41.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fe7ac00d3b0 con 0x7fe790005e80 2020-11-20T12:06:41.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 >> v1:172.21.15.79:6789/0 conn(0x7fe790005e80 legacy=0x7fe790008340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:41.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b6c45700 1 -- 172.21.15.44:0/1454180919 reap_dead start 2020-11-20T12:06:41.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798003730 0x7fe798003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004040 0x7fe798004520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004bc0 0x7fe79800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00607a0 con 0x7fe798004040 2020-11-20T12:06:41.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798005260 con 0x7fe798003730 2020-11-20T12:06:41.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79800bc20 con 0x7fe798004bc0 2020-11-20T12:06:41.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5c43700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004040 msgr2=0x7fe798004520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:41.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004040 0x7fe798004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:41.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b6444700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798003730 0x7fe798003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:41.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.761+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004bc0 0x7fe79800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:41.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.762+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004bc0 msgr2=0x7fe79800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:06:41.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.762+0000 7fe7b6444700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004bc0 0x7fe79800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:41.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.762+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004040 msgr2=0x7fe798004520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:41.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.762+0000 7fe7b6444700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004040 0x7fe798004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:41.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.762+0000 7fe7b6444700 1 -- 172.21.15.44:0/1454180919 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798003240 con 0x7fe798003730 2020-11-20T12:06:41.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.762+0000 7fe7b6444700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798003730 0x7fe798003b30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe7a800c290 tx=0x7fe7a80076d0).ready entity=mon.1 client_cookie=e05b25b6fea9442b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:41.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.764+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8013420 con 0x7fe798003730 2020-11-20T12:06:41.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.764+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a800d040 con 0x7fe798003730 2020-11-20T12:06:41.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.764+0000 7fe7b5442700 1 -- 172.21.15.44:0/1454180919 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80137d0 con 0x7fe798003730 2020-11-20T12:06:41.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 -- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798003730 msgr2=0x7fe798003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:41.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798003730 0x7fe798003b30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe7a800c290 tx=0x7fe7a80076d0).stop 2020-11-20T12:06:41.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 -- 172.21.15.44:0/1454180919 shutdown_connections 2020-11-20T12:06:41.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004bc0 0x7fe79800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:41.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798004040 0x7fe798004520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:41.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/1454180919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798003730 0x7fe798003b30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:41.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 -- 172.21.15.44:0/1454180919 >> 172.21.15.44:0/1454180919 conn(0x7fe7b00658a0 msgr2=0x7fe7b0062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:41.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 -- 172.21.15.44:0/1454180919 shutdown_connections 2020-11-20T12:06:41.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.765+0000 7fe7b86a8700 1 -- 172.21.15.44:0/1454180919 wait complete. 2020-11-20T12:06:41.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 Processor -- start 2020-11-20T12:06:41.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 -- start start 2020-11-20T12:06:41.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798003730 0x7fe7b00a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004040 0x7fe7b00a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798004bc0 0x7fe7b014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b009e660 con 0x7fe798003730 2020-11-20T12:06:41.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b009e890 con 0x7fe798004bc0 2020-11-20T12:06:41.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b86a8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b009eb30 con 0x7fe798004040 2020-11-20T12:06:41.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b6444700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798003730 msgr2=0x7fe7b00a2d10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:41.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.767+0000 7fe7b6444700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798003730 0x7fe7b00a2d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:41.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b5c43700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004040 0x7fe7b00a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:41.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b6c45700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798004bc0 0x7fe7b014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:41.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b5c43700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004040 0x7fe7b00a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49584/0 (socket says 172.21.15.44:49584) 2020-11-20T12:06:41.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b5c43700 1 -- 172.21.15.44:0/686451463 learned_addr learned my addr 172.21.15.44:0/686451463 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b6c45700 1 -- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004040 msgr2=0x7fe7b00a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004040 0x7fe7b00a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b6c45700 1 -- 172.21.15.44:0/686451463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798003730 msgr2=0x7fe7b00a2d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798003730 0x7fe7b00a2d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:41.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b6c45700 1 -- 172.21.15.44:0/686451463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b0139b30 con 0x7fe798004bc0 2020-11-20T12:06:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b5c43700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004040 0x7fe7b00a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b6c45700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798004bc0 0x7fe7b014d7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe7b00aa800 tx=0x7fe7a4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7a27fc700 1 -- 172.21.15.44:0/686451463 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4013a80 con 0x7fe798004bc0 2020-11-20T12:06:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7a27fc700 1 -- 172.21.15.44:0/686451463 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a4013be0 con 0x7fe798004bc0 2020-11-20T12:06:41.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b00a0df0 con 0x7fe798004bc0 2020-11-20T12:06:41.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.768+0000 7fe7a27fc700 1 -- 172.21.15.44:0/686451463 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4017440 con 0x7fe798004bc0 2020-11-20T12:06:41.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.769+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b009d6b0 con 0x7fe798004bc0 2020-11-20T12:06:41.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.770+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b0062dc0 con 0x7fe798004bc0 2020-11-20T12:06:41.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.770+0000 7fe7a27fc700 1 -- 172.21.15.44:0/686451463 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe7a4028070 con 0x7fe798004bc0 2020-11-20T12:06:41.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.770+0000 7fe7a27fc700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe788056d00 0x7fe7880591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.770+0000 7fe7a27fc700 1 -- 172.21.15.44:0/686451463 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe7a4070880 con 0x7fe798004bc0 2020-11-20T12:06:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.770+0000 7fe7b6444700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe788056d00 0x7fe7880591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.771+0000 7fe7b6444700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe788056d00 0x7fe7880591c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe7a8007800 tx=0x7fe7a80076d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:41.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:41.774+0000 7fe7a27fc700 1 -- 172.21.15.44:0/686451463 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe7a4045af0 con 0x7fe798004bc0 2020-11-20T12:06:42.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.017+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe7b009ecf0 con 0x7fe798004bc0 2020-11-20T12:06:42.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.018+0000 7fe7a27fc700 1 -- 172.21.15.44:0/686451463 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fe7a4026020 con 0x7fe798004bc0 2020-11-20T12:06:42.022 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":28,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:06:42.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.019+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe788056d00 msgr2=0x7fe7880591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.019+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe788056d00 0x7fe7880591c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe7a8007800 tx=0x7fe7a80076d0).stop 2020-11-20T12:06:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.019+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798004bc0 msgr2=0x7fe7b014d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.019+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798004bc0 0x7fe7b014d7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fe7b00aa800 tx=0x7fe7a4008040).stop 2020-11-20T12:06:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 shutdown_connections 2020-11-20T12:06:42.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe788056d00 0x7fe7880591c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe798004bc0 0x7fe7b014d7b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe798003730 0x7fe7b00a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 --2- 172.21.15.44:0/686451463 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe798004040 0x7fe7b00a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 >> 172.21.15.44:0/686451463 conn(0x7fe7b00658a0 msgr2=0x7fe7b01390b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:42.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 shutdown_connections 2020-11-20T12:06:42.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.020+0000 7fe7b86a8700 1 -- 172.21.15.44:0/686451463 wait complete. 2020-11-20T12:06:42.034 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:06:42.035 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:06:42.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.161+0000 7f70f1b1f700 1 Processor -- start 2020-11-20T12:06:42.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.161+0000 7f70f1b1f700 1 -- start start 2020-11-20T12:06:42.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70f1b1f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a74d0 0x7f70ec138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70f1b1f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec0a6d90 con 0x7f70ec061340 2020-11-20T12:06:42.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70f1b1f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec0a7030 con 0x7f70ec0a4e10 2020-11-20T12:06:42.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70f1b1f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0a5900 con 0x7f70ec0a74d0 2020-11-20T12:06:42.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70eb7fe700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f70ec061340 legacy=0x7f70ec137a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:06:42.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70ebfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a74d0 0x7f70ec138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70eaffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f70ec0a4e10 0x7f70ec1433f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:44780/0 (socket says 172.21.15.44:44780) 2020-11-20T12:06:42.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70eaffd700 1 -- 172.21.15.44:0/3181266329 learned_addr learned my addr 172.21.15.44:0/3181266329 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:06:42.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70ebfff700 1 -- 172.21.15.44:0/3181266329 >> v1:172.21.15.79:6789/0 conn(0x7f70ec0a4e10 legacy=0x7f70ec1433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70ea7fc700 1 -- 172.21.15.44:0/3181266329 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308596244 0 0) 0x7f70ec0a7030 con 0x7f70ec0a4e10 2020-11-20T12:06:42.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70ebfff700 1 -- 172.21.15.44:0/3181266329 >> v1:172.21.15.44:6789/0 conn(0x7f70ec061340 legacy=0x7f70ec137a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:42.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70ebfff700 1 -- 172.21.15.44:0/3181266329 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec061720 con 0x7f70ec0a74d0 2020-11-20T12:06:42.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.162+0000 7f70ebfff700 1 --2- 172.21.15.44:0/3181266329 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a74d0 0x7f70ec138190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f70e4009cc0 tx=0x7f70e40091d0).ready entity=mon.2 client_cookie=509410360f19aadc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70ea7fc700 1 -- 172.21.15.44:0/3181266329 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400bb80 con 0x7f70ec0a74d0 2020-11-20T12:06:42.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70ea7fc700 1 -- 172.21.15.44:0/3181266329 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e400bce0 con 0x7f70ec0a74d0 2020-11-20T12:06:42.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70ea7fc700 1 -- 172.21.15.44:0/3181266329 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4011610 con 0x7f70ec0a74d0 2020-11-20T12:06:42.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70f1b1f700 1 -- 172.21.15.44:0/3181266329 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a74d0 msgr2=0x7f70ec138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70f1b1f700 1 --2- 172.21.15.44:0/3181266329 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a74d0 0x7f70ec138190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f70e4009cc0 tx=0x7f70e40091d0).stop 2020-11-20T12:06:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70f1b1f700 1 -- 172.21.15.44:0/3181266329 shutdown_connections 2020-11-20T12:06:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70f1b1f700 1 --2- 172.21.15.44:0/3181266329 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a74d0 0x7f70ec138190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70f1b1f700 1 -- 172.21.15.44:0/3181266329 >> 172.21.15.44:0/3181266329 conn(0x7f70ec0ad6e0 msgr2=0x7f70ec0adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:42.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70f1b1f700 1 -- 172.21.15.44:0/3181266329 shutdown_connections 2020-11-20T12:06:42.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.163+0000 7f70f1b1f700 1 -- 172.21.15.44:0/3181266329 wait complete. 2020-11-20T12:06:42.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.164+0000 7f70f1b1f700 1 Processor -- start 2020-11-20T12:06:42.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.164+0000 7f70f1b1f700 1 -- start start 2020-11-20T12:06:42.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70f1b1f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec061340 0x7f70ec067330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70f1b1f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a4e10 0x7f70ec0a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70f1b1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70ec0a74d0 0x7f70ec14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70f1b1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec149fc0 con 0x7f70ec0a74d0 2020-11-20T12:06:42.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70f1b1f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0a4850 con 0x7f70ec061340 2020-11-20T12:06:42.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70f1b1f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0a4650 con 0x7f70ec0a4e10 2020-11-20T12:06:42.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70ebfff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70ec0a74d0 msgr2=0x7f70ec14a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:42.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70ebfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70ec0a74d0 0x7f70ec14a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:42.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec061340 0x7f70ec067330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eaffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a4e10 0x7f70ec0a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec061340 0x7f70ec067330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58442/0 (socket says 172.21.15.44:58442) 2020-11-20T12:06:42.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eb7fe700 1 -- 172.21.15.44:0/1051911134 learned_addr learned my addr 172.21.15.44:0/1051911134 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:42.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eb7fe700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a4e10 msgr2=0x7f70ec0a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eb7fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a4e10 0x7f70ec0a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eb7fe700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70ec0a74d0 msgr2=0x7f70ec14a8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:42.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.165+0000 7f70eb7fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70ec0a74d0 0x7f70ec14a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:42.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70eb7fe700 1 -- 172.21.15.44:0/1051911134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec14a1b0 con 0x7f70ec061340 2020-11-20T12:06:42.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70eaffd700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f70ec0a4e10 0x7f70ec0a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:42.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70eb7fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec061340 0x7f70ec067330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f70dc009740 tx=0x7f70dc006e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc006170 con 0x7f70ec061340 2020-11-20T12:06:42.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0062d0 con 0x7f70ec061340 2020-11-20T12:06:42.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec14a410 con 0x7f70ec061340 2020-11-20T12:06:42.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0038e0 con 0x7f70ec061340 2020-11-20T12:06:42.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.166+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec14ada0 con 0x7f70ec061340 2020-11-20T12:06:42.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.167+0000 7f70f1b1f700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec064dd0 0x7f70ec065540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.167+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0a5410 con 0x7f70ec064dd0 2020-11-20T12:06:42.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.167+0000 7f70eaffd700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec064dd0 0x7f70ec065540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.167+0000 7f70eaffd700 1 -- 172.21.15.44:0/1051911134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70ec137540 con 0x7f70ec064dd0 2020-11-20T12:06:42.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.167+0000 7f70eaffd700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec064dd0 0x7f70ec065540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f70ec0acd60 tx=0x7f70d40011a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.167+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400fac0 con 0x7f70ec064dd0 2020-11-20T12:06:42.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.167+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f70dc00f070 con 0x7f70ec061340 2020-11-20T12:06:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70e37fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f70c8056dc0 0x7f70c8059280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f70dc06f880 con 0x7f70ec061340 2020-11-20T12:06:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70ebfff700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f70c8056dc0 0x7f70c8059280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f70d400fc20 con 0x7f70ec064dd0 2020-11-20T12:06:42.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec064dd0 msgr2=0x7f70ec065540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70e37fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec064dd0 0x7f70ec065540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f70ec0acd60 tx=0x7f70d40011a0).stop 2020-11-20T12:06:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec064dd0 msgr2=0x7f70ec065540 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70e37fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec064dd0 0x7f70ec065540 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0x7f70ec0acd60 tx=0x7f70d40011a0).stop 2020-11-20T12:06:42.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.168+0000 7f70ebfff700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f70c8056dc0 0x7f70c8059280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f70e4009fb0 tx=0x7f70e4016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.176+0000 7f70f1b1f700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec0619a0 0x7f70ec065a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.176+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec14a5a0 con 0x7f70ec0619a0 2020-11-20T12:06:42.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.176+0000 7f70eaffd700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec0619a0 0x7f70ec065a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.176+0000 7f70eaffd700 1 -- 172.21.15.44:0/1051911134 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f70ec0446d0 con 0x7f70ec0619a0 2020-11-20T12:06:42.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.176+0000 7f70eaffd700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec0619a0 0x7f70ec065a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f70d4016000 tx=0x7f70d4017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40136f0 con 0x7f70ec0619a0 2020-11-20T12:06:42.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4766 (secure 0 0 0) 0x7f70d400f8b0 con 0x7f70ec0619a0 2020-11-20T12:06:42.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec0619a0 msgr2=0x7f70ec065a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70e37fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec0619a0 0x7f70ec065a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f70d4016000 tx=0x7f70d4017000).stop 2020-11-20T12:06:42.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70e37fe700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec0619a0 msgr2=0x7f70ec065a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:42.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70e37fe700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec0619a0 0x7f70ec065a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f70c8056dc0 msgr2=0x7f70c8059280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70f1b1f700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f70c8056dc0 0x7f70c8059280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f70e4009fb0 tx=0x7f70e4016000).stop 2020-11-20T12:06:42.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec061340 msgr2=0x7f70ec067330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70f1b1f700 1 --2- 172.21.15.44:0/1051911134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f70ec061340 0x7f70ec067330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f70dc009740 tx=0x7f70dc006e20).stop 2020-11-20T12:06:42.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70ebfff700 1 -- 172.21.15.44:0/1051911134 reap_dead start 2020-11-20T12:06:42.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.177+0000 7f70ebfff700 1 -- 172.21.15.44:0/1051911134 reap_dead start 2020-11-20T12:06:42.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.178+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 shutdown_connections 2020-11-20T12:06:42.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.178+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 >> 172.21.15.44:0/1051911134 conn(0x7f70ec0ad6e0 msgr2=0x7f70ec0a9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:42.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.178+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 shutdown_connections 2020-11-20T12:06:42.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.178+0000 7f70f1b1f700 1 -- 172.21.15.44:0/1051911134 wait complete. 2020-11-20T12:06:42.195 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:42.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:42.195 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:42.195 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:06:42.195 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 28, 2020-11-20T12:06:42.196 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:42.196 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:42.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:42.196 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.197 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:06:42.197 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:42.197 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:42.197 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:42.197 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:42.198 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:42.198 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:42.198 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:42.198 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:42.198 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:42.199 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:42.199 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:42.199 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.199 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:42.199 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:42.200 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:42.200 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:42.200 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:42.200 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:42.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:42.201 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:42.201 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:42.201 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:42.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.202 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:42.202 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:42.202 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:42.202 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:42.203 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:42.203 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:42.203 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:42.203 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:42.204 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:42.204 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:42.204 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:42.204 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:42.204 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:42.205 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:42.205 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:42.205 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:42.205 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:42.205 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:42.206 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:42.206 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:42.206 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:42.206 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:42.206 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:42.207 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.207 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:42.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.207 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:42.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.208 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:42.208 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:42.208 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:42.208 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:42.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:42.209 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:42.209 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:42.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:42.210 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:42.211 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:42.212 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:42.212 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:42.212 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:42.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.213 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:42.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:42.213 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:42.213 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:42.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.214 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:42.214 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:42.214 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:42.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:42.215 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:42.216 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:42.216 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:42.216 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:42.217 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:42.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.217 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:42.218 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:42.218 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:42.218 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:42.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:42.219 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:42.219 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.219 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:42.220 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:42.220 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.220 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.220 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:42.221 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:42.221 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:42.221 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:42.222 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:42.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.222 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.223 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:42.223 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:42.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.223 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.223 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.224 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:42.224 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.224 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.224 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:06:42.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.225 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.225 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.225 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:42.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.226 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.226 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:06:42.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.226 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.226 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.227 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:42.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.227 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.227 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:42.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.228 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.228 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.228 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:06:42.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.229 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.229 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:06:42.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.229 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.229 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.230 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:42.230 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.230 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.230 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:42.231 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:42.232 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:06:42.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd9a10fc700 1 Processor -- start 2020-11-20T12:06:42.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd9a10fc700 1 -- start start 2020-11-20T12:06:42.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd9a10fc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9130 0x7fd99c0adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd9a10fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd99c0a9510 0x7fd99c13b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd9a10fc700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c0ab630 con 0x7fd99c0aabd0 2020-11-20T12:06:42.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd9a10fc700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99c0ab110 con 0x7fd99c0a9510 2020-11-20T12:06:42.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd9a10fc700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0ab3b0 con 0x7fd99c0a9130 2020-11-20T12:06:42.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd993fff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fd99c0a9510 msgr2=0x7fd99c13b060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:42.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd99b59e700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fd99c0aabd0 legacy=0x7fd99c13e080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:06:42.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd993fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd99c0a9510 0x7fd99c13b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:42.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd99ad9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9130 0x7fd99c0adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd99ad9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9130 0x7fd99c0adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49610/0 (socket says 172.21.15.44:49610) 2020-11-20T12:06:42.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.377+0000 7fd99ad9d700 1 -- 172.21.15.44:0/481035525 learned_addr learned my addr 172.21.15.44:0/481035525 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:42.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99ad9d700 1 -- 172.21.15.44:0/481035525 >> v2:172.21.15.44:3300/0 conn(0x7fd99c0a9510 msgr2=0x7fd99c13b060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:42.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99ad9d700 1 --2- 172.21.15.44:0/481035525 >> v2:172.21.15.44:3300/0 conn(0x7fd99c0a9510 0x7fd99c13b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:42.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99ad9d700 1 -- 172.21.15.44:0/481035525 >> v1:172.21.15.44:6789/0 conn(0x7fd99c0aabd0 legacy=0x7fd99c13e080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:06:42.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99ad9d700 1 -- 172.21.15.44:0/481035525 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0ad7f0 con 0x7fd99c0a9130 2020-11-20T12:06:42.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99ad9d700 1 --2- 172.21.15.44:0/481035525 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9130 0x7fd99c0adc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd98c006fd0 tx=0x7fd98c006eb0).ready entity=mon.2 client_cookie=57ff3d570c2ca102 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99a59c700 1 -- 172.21.15.44:0/481035525 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c009890 con 0x7fd99c0a9130 2020-11-20T12:06:42.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99a59c700 1 -- 172.21.15.44:0/481035525 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c0099f0 con 0x7fd99c0a9130 2020-11-20T12:06:42.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.378+0000 7fd99a59c700 1 -- 172.21.15.44:0/481035525 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c009d70 con 0x7fd99c0a9130 2020-11-20T12:06:42.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.379+0000 7fd9a10fc700 1 -- 172.21.15.44:0/481035525 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9130 msgr2=0x7fd99c0adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.379+0000 7fd9a10fc700 1 --2- 172.21.15.44:0/481035525 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9130 0x7fd99c0adc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd98c006fd0 tx=0x7fd98c006eb0).stop 2020-11-20T12:06:42.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.379+0000 7fd9a10fc700 1 -- 172.21.15.44:0/481035525 shutdown_connections 2020-11-20T12:06:42.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.379+0000 7fd9a10fc700 1 --2- 172.21.15.44:0/481035525 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9130 0x7fd99c0adc90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.379+0000 7fd9a10fc700 1 --2- 172.21.15.44:0/481035525 >> v2:172.21.15.44:3300/0 conn(0x7fd99c0a9510 0x7fd99c13b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:42.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.379+0000 7fd9a10fc700 1 -- 172.21.15.44:0/481035525 >> 172.21.15.44:0/481035525 conn(0x7fd99c060170 msgr2=0x7fd99c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:42.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.380+0000 7fd9a10fc700 1 -- 172.21.15.44:0/481035525 shutdown_connections 2020-11-20T12:06:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.381+0000 7fd9a10fc700 1 -- 172.21.15.44:0/481035525 wait complete. 2020-11-20T12:06:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.382+0000 7fd9a10fc700 1 Processor -- start 2020-11-20T12:06:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.382+0000 7fd9a10fc700 1 -- start start 2020-11-20T12:06:42.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd9a10fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c0a9130 0x7fd99c0a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd9a10fc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9510 0x7fd99c09f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd9a10fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99c0aabd0 0x7fd99c149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd9a10fc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0b1a50 con 0x7fd99c0a9130 2020-11-20T12:06:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd9a10fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c09e030 con 0x7fd99c0aabd0 2020-11-20T12:06:42.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd9a10fc700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c09e260 con 0x7fd99c0a9510 2020-11-20T12:06:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd99ad9d700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c0a9130 msgr2=0x7fd99c0a1230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:06:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd99ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c0a9130 0x7fd99c0a1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:06:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9510 0x7fd99c09f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd99b59e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99c0aabd0 0x7fd99c149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9510 0x7fd99c09f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49616/0 (socket says 172.21.15.44:49616) 2020-11-20T12:06:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 -- 172.21.15.44:0/3410345757 learned_addr learned my addr 172.21.15.44:0/3410345757 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:06:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99c0aabd0 msgr2=0x7fd99c149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99c0aabd0 0x7fd99c149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c0a9130 msgr2=0x7fd99c0a1230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:06:42.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd99c0a9130 0x7fd99c0a1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:06:42.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 -- 172.21.15.44:0/3410345757 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c09e500 con 0x7fd99c0a9510 2020-11-20T12:06:42.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd99b59e700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99c0aabd0 0x7fd99c149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:06:42.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.383+0000 7fd993fff700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9510 0x7fd99c09f5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd9840036a0 tx=0x7fd984006e70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.384+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9840050d0 con 0x7fd99c0a9510 2020-11-20T12:06:42.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.384+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd984005230 con 0x7fd99c0a9510 2020-11-20T12:06:42.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.384+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9840038e0 con 0x7fd99c0a9510 2020-11-20T12:06:42.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.384+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c09e7a0 con 0x7fd99c0a9510 2020-11-20T12:06:42.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.384+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c138870 con 0x7fd99c0a9510 2020-11-20T12:06:42.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd9a10fc700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c14a2b0 0x7fd99c0a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c05c7a0 con 0x7fd99c14a2b0 2020-11-20T12:06:42.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd99ad9d700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c14a2b0 0x7fd99c0a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd98400f070 con 0x7fd99c0a9510 2020-11-20T12:06:42.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd99ad9d700 1 -- 172.21.15.44:0/3410345757 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd99c09fc30 con 0x7fd99c14a2b0 2020-11-20T12:06:42.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd99ad9d700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c14a2b0 0x7fd99c0a6ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fd98c006ef0 tx=0x7fd98c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd992ffd700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd978056cd0 0x7fd978059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.385+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd98406f7d0 con 0x7fd99c0a9510 2020-11-20T12:06:42.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd99b59e700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd978056cd0 0x7fd978059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c01e460 con 0x7fd99c14a2b0 2020-11-20T12:06:42.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fd98c01e5c0 con 0x7fd99c14a2b0 2020-11-20T12:06:42.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c14a2b0 msgr2=0x7fd99c0a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd992ffd700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c14a2b0 0x7fd99c0a6ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fd98c006ef0 tx=0x7fd98c015000).stop 2020-11-20T12:06:42.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c14a2b0 msgr2=0x7fd99c0a6ce0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:42.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd992ffd700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c14a2b0 0x7fd99c0a6ce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.386+0000 7fd99b59e700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd978056cd0 0x7fd978059190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fd994003a20 tx=0x7fd994009060).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.398+0000 7fd9a10fc700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a71d0 0x7fd99c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:06:42.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.398+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c09eb40 con 0x7fd99c0a71d0 2020-11-20T12:06:42.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.398+0000 7fd99ad9d700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a71d0 0x7fd99c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:06:42.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.398+0000 7fd99ad9d700 1 -- 172.21.15.44:0/3410345757 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd99c0a75e0 con 0x7fd99c0a71d0 2020-11-20T12:06:42.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.399+0000 7fd99ad9d700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a71d0 0x7fd99c14d390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd99c147c90 tx=0x7fd98c01e000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:06:42.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.399+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c01dae0 con 0x7fd99c0a71d0 2020-11-20T12:06:42.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.399+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4764 (secure 0 0 0) 0x7fd98c011cc0 con 0x7fd99c0a71d0 2020-11-20T12:06:42.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.399+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a71d0 msgr2=0x7fd99c14d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.399+0000 7fd992ffd700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a71d0 0x7fd99c14d390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd99c147c90 tx=0x7fd98c01e000).stop 2020-11-20T12:06:42.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.399+0000 7fd992ffd700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a71d0 msgr2=0x7fd99c14d390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:06:42.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.399+0000 7fd992ffd700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a71d0 0x7fd99c14d390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:06:42.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd978056cd0 msgr2=0x7fd978059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd978056cd0 0x7fd978059190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fd994003a20 tx=0x7fd994009060).stop 2020-11-20T12:06:42.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9510 msgr2=0x7fd99c09f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:06:42.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 --2- 172.21.15.44:0/3410345757 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd99c0a9510 0x7fd99c09f5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd9840036a0 tx=0x7fd984006e70).stop 2020-11-20T12:06:42.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd99b59e700 1 -- 172.21.15.44:0/3410345757 reap_dead start 2020-11-20T12:06:42.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd99b59e700 1 -- 172.21.15.44:0/3410345757 reap_dead start 2020-11-20T12:06:42.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 shutdown_connections 2020-11-20T12:06:42.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 >> 172.21.15.44:0/3410345757 conn(0x7fd99c060170 msgr2=0x7fd99c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:06:42.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 shutdown_connections 2020-11-20T12:06:42.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:06:42.400+0000 7fd9a10fc700 1 -- 172.21.15.44:0/3410345757 wait complete. 2020-11-20T12:06:42.409 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:06:42.409 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:42.409 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:42.410 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:06:42.410 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 28, 2020-11-20T12:06:42.410 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:06:42.410 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:06:42.410 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:06:42.411 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.411 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:06:42.411 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:42.411 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:06:42.411 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:06:42.412 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:42.412 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:42.412 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:42.412 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:42.412 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:42.413 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:42.413 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:42.413 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:42.413 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.413 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:06:42.414 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:06:42.414 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:42.414 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:42.414 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:42.414 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:42.415 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:42.415 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:42.415 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:42.415 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:42.415 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.416 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:06:42.416 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:06:42.416 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:06:42.416 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:06:42.417 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:06:42.417 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:06:42.417 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:42.417 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:06:42.417 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:06:42.418 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:06:42.418 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:06:42.418 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:06:42.418 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:06:42.418 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:06:42.419 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:06:42.419 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:06:42.419 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:06:42.419 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:06:42.419 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:06:42.420 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:06:42.420 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:06:42.420 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:06:42.420 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:06:42.420 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.421 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:06:42.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.421 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:06:42.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.421 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:06:42.422 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:06:42.422 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:42.422 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:42.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:42.422 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:06:42.423 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.423 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:42.423 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:06:42.424 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.424 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.424 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.424 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:06:42.425 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:06:42.425 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:42.425 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:42.425 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:42.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.426 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:06:42.426 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:06:42.426 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:42.426 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:42.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.427 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:42.427 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:06:42.427 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:42.428 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:06:42.428 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.428 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.429 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.429 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:06:42.429 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:06:42.429 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:42.430 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:42.430 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:42.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.430 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:06:42.431 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:06:42.431 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:06:42.431 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:06:42.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.431 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:06:42.432 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:06:42.432 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:06:42.432 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:06:42.433 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:06:42.433 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.433 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.433 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:06:42.434 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:06:42.434 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:06:42.434 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:06:42.434 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:06:42.434 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.435 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.435 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:06:42.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:06:42.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.436 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.436 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.436 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:42.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.436 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.437 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:06:42.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.437 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.437 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.437 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:42.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.438 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.438 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:06:42.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.438 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.439 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.439 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:06:42.439 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.439 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:06:42.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.440 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.440 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.440 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:06:42.440 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.441 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:06:42.441 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:06:42.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:06:42.441 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:06:42.441 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:06:42.442 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:06:42.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:06:42.442 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:06:42.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:06:42.442 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:06:42.443 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:06:42.444 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:06:42.482 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2020-11-20T12:06:42.482 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2020-11-20T12:06:42.483 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2020-11-20T12:06:42.483 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2020-11-20T12:06:42.483 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2020-11-20T12:06:42.483 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2020-11-20T12:06:42.483 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2020-11-20T12:06:42.484 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2020-11-20T12:06:42.484 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2020-11-20T12:06:42.484 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2020-11-20T12:06:42.484 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2020-11-20T12:06:42.484 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2020-11-20T12:06:42.485 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2020-11-20T12:06:42.485 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2020-11-20T12:06:42.485 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2020-11-20T12:06:42.485 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2020-11-20T12:06:42.486 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2020-11-20T12:06:42.486 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2020-11-20T12:06:42.486 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2020-11-20T12:06:42.486 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2020-11-20T12:06:42.486 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2020-11-20T12:06:42.487 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2020-11-20T12:06:42.487 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2020-11-20T12:06:42.487 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2020-11-20T12:06:42.487 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2020-11-20T12:06:42.487 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2020-11-20T12:06:42.488 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2020-11-20T12:06:42.488 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2020-11-20T12:06:42.488 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2020-11-20T12:06:42.488 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2020-11-20T12:06:42.488 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2020-11-20T12:06:42.489 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2020-11-20T12:06:42.489 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2020-11-20T12:06:42.489 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2020-11-20T12:06:42.489 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2020-11-20T12:06:42.489 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2020-11-20T12:06:42.490 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2020-11-20T12:06:42.490 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2020-11-20T12:06:42.490 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2020-11-20T12:06:42.490 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2020-11-20T12:06:42.490 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2020-11-20T12:06:42.491 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2020-11-20T12:06:42.491 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2020-11-20T12:06:42.491 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2020-11-20T12:06:42.491 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2020-11-20T12:06:42.491 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2020-11-20T12:06:42.492 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2020-11-20T12:06:42.492 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2020-11-20T12:06:42.492 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2020-11-20T12:06:42.492 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2020-11-20T12:06:42.492 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2020-11-20T12:06:42.493 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2020-11-20T12:06:42.493 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2020-11-20T12:06:42.493 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2020-11-20T12:06:42.493 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2020-11-20T12:06:42.493 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2020-11-20T12:06:42.494 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2020-11-20T12:06:42.494 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2020-11-20T12:06:42.494 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2020-11-20T12:06:42.494 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2020-11-20T12:06:42.494 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2020-11-20T12:06:42.495 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2020-11-20T12:06:42.495 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2020-11-20T12:06:42.495 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2020-11-20T12:06:42.495 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2020-11-20T12:06:42.496 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2020-11-20T12:06:42.496 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2020-11-20T12:06:42.496 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2020-11-20T12:06:42.496 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2020-11-20T12:06:42.496 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2020-11-20T12:06:42.497 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2020-11-20T12:06:42.497 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2020-11-20T12:06:42.497 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2020-11-20T12:06:42.497 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2020-11-20T12:06:42.497 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2020-11-20T12:06:42.498 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2020-11-20T12:06:42.498 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2020-11-20T12:06:42.498 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2020-11-20T12:06:42.498 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2020-11-20T12:06:42.498 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2020-11-20T12:06:42.499 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2020-11-20T12:06:43.239 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2020-11-20T12:06:43.239 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2020-11-20T12:06:43.240 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2020-11-20T12:06:43.240 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2020-11-20T12:06:43.240 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2020-11-20T12:06:43.241 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2020-11-20T12:06:43.241 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2020-11-20T12:06:43.241 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2020-11-20T12:06:43.241 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2020-11-20T12:06:43.242 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2020-11-20T12:06:43.242 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2020-11-20T12:06:43.242 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2020-11-20T12:06:43.242 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2020-11-20T12:06:43.243 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2020-11-20T12:06:43.243 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2020-11-20T12:06:43.243 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2020-11-20T12:06:43.243 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2020-11-20T12:06:43.243 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2020-11-20T12:06:43.244 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2020-11-20T12:06:43.244 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2020-11-20T12:06:43.244 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2020-11-20T12:06:43.244 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2020-11-20T12:06:43.245 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2020-11-20T12:06:43.245 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2020-11-20T12:06:43.245 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2020-11-20T12:06:43.245 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2020-11-20T12:06:43.245 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2020-11-20T12:06:43.246 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2020-11-20T12:06:43.246 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2020-11-20T12:06:43.246 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2020-11-20T12:06:43.246 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2020-11-20T12:06:43.247 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2020-11-20T12:06:43.247 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2020-11-20T12:06:43.247 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2020-11-20T12:06:43.247 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2020-11-20T12:06:43.248 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2020-11-20T12:06:43.248 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2020-11-20T12:06:43.248 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2020-11-20T12:06:43.248 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2020-11-20T12:06:43.249 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2020-11-20T12:06:43.249 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2020-11-20T12:06:43.249 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2020-11-20T12:06:43.249 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2020-11-20T12:06:43.249 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2020-11-20T12:06:43.250 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2020-11-20T12:06:43.250 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2020-11-20T12:06:43.250 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2020-11-20T12:06:43.250 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2020-11-20T12:06:43.250 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2020-11-20T12:06:43.251 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2020-11-20T12:06:43.251 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2020-11-20T12:06:43.251 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2020-11-20T12:06:43.251 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2020-11-20T12:06:43.252 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2020-11-20T12:06:43.252 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2020-11-20T12:06:43.252 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2020-11-20T12:06:43.252 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2020-11-20T12:06:43.252 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2020-11-20T12:06:43.253 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2020-11-20T12:06:43.253 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2020-11-20T12:06:43.253 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2020-11-20T12:06:43.253 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2020-11-20T12:06:43.253 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2020-11-20T12:06:43.254 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2020-11-20T12:06:43.254 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2020-11-20T12:06:43.254 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2020-11-20T12:06:43.254 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2020-11-20T12:06:43.255 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2020-11-20T12:06:43.255 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2020-11-20T12:06:43.255 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2020-11-20T12:06:43.255 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2020-11-20T12:06:43.255 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2020-11-20T12:06:43.256 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2020-11-20T12:06:43.256 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2020-11-20T12:06:43.256 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2020-11-20T12:06:43.256 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2020-11-20T12:06:43.256 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2020-11-20T12:06:43.257 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2020-11-20T12:06:43.257 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2020-11-20T12:06:43.257 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2020-11-20T12:06:43.257 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2020-11-20T12:06:43.940 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2020-11-20T12:06:43.941 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2020-11-20T12:06:43.941 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2020-11-20T12:06:43.941 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2020-11-20T12:06:43.942 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2020-11-20T12:06:43.942 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2020-11-20T12:06:43.942 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2020-11-20T12:06:43.942 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2020-11-20T12:06:43.942 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2020-11-20T12:06:43.943 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2020-11-20T12:06:43.943 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2020-11-20T12:06:43.943 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2020-11-20T12:06:43.943 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2020-11-20T12:06:43.943 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2020-11-20T12:06:43.944 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2020-11-20T12:06:43.944 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2020-11-20T12:06:43.944 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2020-11-20T12:06:43.944 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2020-11-20T12:06:43.945 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2020-11-20T12:06:43.945 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2020-11-20T12:06:43.945 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2020-11-20T12:06:43.945 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2020-11-20T12:06:43.945 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2020-11-20T12:06:43.946 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2020-11-20T12:06:43.946 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2020-11-20T12:06:43.946 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2020-11-20T12:06:43.946 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2020-11-20T12:06:43.946 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2020-11-20T12:06:43.947 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2020-11-20T12:06:43.947 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2020-11-20T12:06:43.947 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2020-11-20T12:06:43.947 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2020-11-20T12:06:43.947 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2020-11-20T12:06:43.948 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2020-11-20T12:06:43.948 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2020-11-20T12:06:43.948 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2020-11-20T12:06:43.948 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2020-11-20T12:06:43.948 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2020-11-20T12:06:43.949 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2020-11-20T12:06:43.949 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2020-11-20T12:06:43.949 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2020-11-20T12:06:43.950 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2020-11-20T12:06:43.950 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2020-11-20T12:06:43.950 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2020-11-20T12:06:43.950 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2020-11-20T12:06:43.951 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2020-11-20T12:06:43.951 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2020-11-20T12:06:43.951 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2020-11-20T12:06:43.951 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2020-11-20T12:06:43.952 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2020-11-20T12:06:43.952 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2020-11-20T12:06:43.952 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2020-11-20T12:06:43.953 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2020-11-20T12:06:43.953 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2020-11-20T12:06:43.953 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2020-11-20T12:06:43.953 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2020-11-20T12:06:43.954 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2020-11-20T12:06:43.954 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2020-11-20T12:06:43.954 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2020-11-20T12:06:43.954 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2020-11-20T12:06:43.955 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2020-11-20T12:06:43.955 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2020-11-20T12:06:43.955 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2020-11-20T12:06:43.955 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2020-11-20T12:06:43.956 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2020-11-20T12:06:43.956 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2020-11-20T12:06:43.956 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2020-11-20T12:06:43.956 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2020-11-20T12:06:43.957 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2020-11-20T12:06:43.957 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2020-11-20T12:06:43.957 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2020-11-20T12:06:43.957 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2020-11-20T12:06:43.957 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2020-11-20T12:06:43.958 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2020-11-20T12:06:43.958 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2020-11-20T12:06:43.958 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2020-11-20T12:06:43.958 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2020-11-20T12:06:43.959 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2020-11-20T12:06:43.959 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2020-11-20T12:06:43.959 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2020-11-20T12:06:43.960 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2020-11-20T12:06:44.710 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2020-11-20T12:06:44.710 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2020-11-20T12:06:44.711 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2020-11-20T12:06:44.711 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2020-11-20T12:06:44.711 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2020-11-20T12:06:44.711 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2020-11-20T12:06:44.712 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2020-11-20T12:06:44.712 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2020-11-20T12:06:44.712 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2020-11-20T12:06:44.712 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2020-11-20T12:06:44.712 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2020-11-20T12:06:44.713 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2020-11-20T12:06:44.713 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2020-11-20T12:06:44.713 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2020-11-20T12:06:44.713 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2020-11-20T12:06:44.714 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2020-11-20T12:06:44.714 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2020-11-20T12:06:44.714 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2020-11-20T12:06:44.714 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2020-11-20T12:06:44.714 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2020-11-20T12:06:44.715 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2020-11-20T12:06:44.715 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2020-11-20T12:06:44.715 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2020-11-20T12:06:44.715 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2020-11-20T12:06:44.715 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2020-11-20T12:06:44.716 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2020-11-20T12:06:44.716 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2020-11-20T12:06:44.716 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2020-11-20T12:06:44.716 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2020-11-20T12:06:44.716 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2020-11-20T12:06:44.717 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2020-11-20T12:06:44.717 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2020-11-20T12:06:44.717 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2020-11-20T12:06:44.717 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2020-11-20T12:06:44.717 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2020-11-20T12:06:44.718 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2020-11-20T12:06:44.718 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2020-11-20T12:06:44.718 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2020-11-20T12:06:44.718 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2020-11-20T12:06:44.718 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2020-11-20T12:06:44.719 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2020-11-20T12:06:44.719 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2020-11-20T12:06:44.719 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2020-11-20T12:06:44.719 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2020-11-20T12:06:44.719 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2020-11-20T12:06:44.720 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2020-11-20T12:06:44.720 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2020-11-20T12:06:44.720 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2020-11-20T12:06:44.720 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2020-11-20T12:06:44.720 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2020-11-20T12:06:44.721 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2020-11-20T12:06:44.721 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2020-11-20T12:06:44.721 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2020-11-20T12:06:44.721 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2020-11-20T12:06:44.722 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2020-11-20T12:06:44.722 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2020-11-20T12:06:44.722 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2020-11-20T12:06:44.722 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2020-11-20T12:06:44.722 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2020-11-20T12:06:44.723 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2020-11-20T12:06:44.723 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2020-11-20T12:06:44.723 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2020-11-20T12:06:44.723 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2020-11-20T12:06:44.723 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2020-11-20T12:06:44.724 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2020-11-20T12:06:44.724 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2020-11-20T12:06:44.724 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2020-11-20T12:06:44.724 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2020-11-20T12:06:44.724 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2020-11-20T12:06:44.725 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2020-11-20T12:06:44.725 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2020-11-20T12:06:44.725 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2020-11-20T12:06:44.725 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2020-11-20T12:06:44.725 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2020-11-20T12:06:44.726 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2020-11-20T12:06:44.726 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2020-11-20T12:06:44.726 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2020-11-20T12:06:44.726 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2020-11-20T12:06:44.726 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2020-11-20T12:06:44.727 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2020-11-20T12:06:44.727 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2020-11-20T12:06:45.438 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2020-11-20T12:06:45.439 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2020-11-20T12:06:45.439 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2020-11-20T12:06:45.439 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2020-11-20T12:06:45.439 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2020-11-20T12:06:45.440 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2020-11-20T12:06:45.440 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2020-11-20T12:06:45.440 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2020-11-20T12:06:45.440 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2020-11-20T12:06:45.441 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2020-11-20T12:06:45.441 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2020-11-20T12:06:45.441 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2020-11-20T12:06:45.441 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2020-11-20T12:06:45.441 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2020-11-20T12:06:45.442 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2020-11-20T12:06:45.442 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2020-11-20T12:06:45.442 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2020-11-20T12:06:45.442 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2020-11-20T12:06:45.443 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2020-11-20T12:06:45.443 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2020-11-20T12:06:45.443 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2020-11-20T12:06:45.443 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2020-11-20T12:06:45.443 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2020-11-20T12:06:45.444 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2020-11-20T12:06:45.444 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2020-11-20T12:06:45.444 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2020-11-20T12:06:45.444 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2020-11-20T12:06:45.445 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2020-11-20T12:06:45.445 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2020-11-20T12:06:45.445 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2020-11-20T12:06:45.446 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2020-11-20T12:06:45.446 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2020-11-20T12:06:45.446 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2020-11-20T12:06:45.446 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2020-11-20T12:06:45.446 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2020-11-20T12:06:45.447 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2020-11-20T12:06:45.447 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2020-11-20T12:06:45.447 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2020-11-20T12:06:45.447 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2020-11-20T12:06:45.448 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2020-11-20T12:06:45.448 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2020-11-20T12:06:45.448 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2020-11-20T12:06:45.448 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2020-11-20T12:06:45.448 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2020-11-20T12:06:45.449 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2020-11-20T12:06:45.449 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2020-11-20T12:06:45.449 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2020-11-20T12:06:45.450 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2020-11-20T12:06:45.450 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2020-11-20T12:06:45.450 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2020-11-20T12:06:45.450 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2020-11-20T12:06:45.451 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2020-11-20T12:06:45.451 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2020-11-20T12:06:45.451 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2020-11-20T12:06:45.451 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2020-11-20T12:06:45.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2020-11-20T12:06:45.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2020-11-20T12:06:45.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2020-11-20T12:06:45.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2020-11-20T12:06:45.452 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2020-11-20T12:06:45.453 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2020-11-20T12:06:45.453 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2020-11-20T12:06:45.453 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2020-11-20T12:06:45.453 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2020-11-20T12:06:45.454 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2020-11-20T12:06:45.454 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2020-11-20T12:06:45.454 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2020-11-20T12:06:45.454 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2020-11-20T12:06:45.454 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2020-11-20T12:06:45.455 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2020-11-20T12:06:45.455 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2020-11-20T12:06:45.455 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2020-11-20T12:06:45.455 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2020-11-20T12:06:45.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2020-11-20T12:06:45.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2020-11-20T12:06:45.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2020-11-20T12:06:45.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2020-11-20T12:06:45.456 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2020-11-20T12:06:45.457 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2020-11-20T12:06:45.457 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2020-11-20T12:06:45.457 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2020-11-20T12:06:46.147 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2020-11-20T12:06:46.147 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2020-11-20T12:06:46.147 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2020-11-20T12:06:46.147 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2020-11-20T12:06:46.148 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2020-11-20T12:06:46.148 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2020-11-20T12:06:46.148 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2020-11-20T12:06:46.148 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2020-11-20T12:06:46.148 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2020-11-20T12:06:46.149 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2020-11-20T12:06:46.149 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2020-11-20T12:06:46.149 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2020-11-20T12:06:46.149 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2020-11-20T12:06:46.150 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2020-11-20T12:06:46.150 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2020-11-20T12:06:46.150 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2020-11-20T12:06:46.150 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2020-11-20T12:06:46.150 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2020-11-20T12:06:46.151 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2020-11-20T12:06:46.151 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2020-11-20T12:06:46.151 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2020-11-20T12:06:46.151 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2020-11-20T12:06:46.151 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2020-11-20T12:06:46.152 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2020-11-20T12:06:46.152 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2020-11-20T12:06:46.152 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2020-11-20T12:06:46.152 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2020-11-20T12:06:46.152 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2020-11-20T12:06:46.153 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2020-11-20T12:06:46.153 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2020-11-20T12:06:46.153 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2020-11-20T12:06:46.153 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2020-11-20T12:06:46.153 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2020-11-20T12:06:46.154 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2020-11-20T12:06:46.154 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2020-11-20T12:06:46.154 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2020-11-20T12:06:46.154 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2020-11-20T12:06:46.154 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2020-11-20T12:06:46.155 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2020-11-20T12:06:46.155 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2020-11-20T12:06:46.155 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2020-11-20T12:06:46.155 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2020-11-20T12:06:46.155 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2020-11-20T12:06:46.156 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2020-11-20T12:06:46.156 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2020-11-20T12:06:46.156 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2020-11-20T12:06:46.156 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2020-11-20T12:06:46.156 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2020-11-20T12:06:46.157 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2020-11-20T12:06:46.157 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2020-11-20T12:06:46.157 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2020-11-20T12:06:46.157 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2020-11-20T12:06:46.157 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2020-11-20T12:06:46.158 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2020-11-20T12:06:46.158 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2020-11-20T12:06:46.158 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2020-11-20T12:06:46.158 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2020-11-20T12:06:46.158 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2020-11-20T12:06:46.159 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2020-11-20T12:06:46.159 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2020-11-20T12:06:46.159 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2020-11-20T12:06:46.159 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2020-11-20T12:06:46.159 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2020-11-20T12:06:46.160 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2020-11-20T12:06:46.160 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2020-11-20T12:06:46.160 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2020-11-20T12:06:46.160 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2020-11-20T12:06:46.160 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2020-11-20T12:06:46.161 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2020-11-20T12:06:46.161 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2020-11-20T12:06:46.161 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2020-11-20T12:06:46.161 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2020-11-20T12:06:46.162 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2020-11-20T12:06:46.162 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2020-11-20T12:06:46.162 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2020-11-20T12:06:46.162 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2020-11-20T12:06:46.162 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2020-11-20T12:06:46.162 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2020-11-20T12:06:46.163 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2020-11-20T12:06:46.163 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2020-11-20T12:06:46.163 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2020-11-20T12:06:46.865 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2020-11-20T12:06:46.865 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2020-11-20T12:06:46.865 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2020-11-20T12:06:46.866 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2020-11-20T12:06:46.866 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2020-11-20T12:06:46.866 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2020-11-20T12:06:46.866 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2020-11-20T12:06:46.867 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2020-11-20T12:06:46.867 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2020-11-20T12:06:46.867 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2020-11-20T12:06:46.867 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2020-11-20T12:06:46.867 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2020-11-20T12:06:46.868 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2020-11-20T12:06:46.868 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2020-11-20T12:06:46.868 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2020-11-20T12:06:46.868 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2020-11-20T12:06:46.868 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2020-11-20T12:06:46.869 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2020-11-20T12:06:46.869 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2020-11-20T12:06:46.869 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2020-11-20T12:06:46.869 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2020-11-20T12:06:46.870 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2020-11-20T12:06:46.870 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2020-11-20T12:06:46.870 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2020-11-20T12:06:46.870 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2020-11-20T12:06:46.870 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2020-11-20T12:06:46.871 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2020-11-20T12:06:46.871 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2020-11-20T12:06:46.871 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2020-11-20T12:06:46.871 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2020-11-20T12:06:46.871 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2020-11-20T12:06:46.872 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2020-11-20T12:06:46.872 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2020-11-20T12:06:46.872 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2020-11-20T12:06:46.872 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2020-11-20T12:06:46.872 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2020-11-20T12:06:46.873 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2020-11-20T12:06:46.873 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2020-11-20T12:06:46.873 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2020-11-20T12:06:46.873 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2020-11-20T12:06:46.874 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2020-11-20T12:06:46.874 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2020-11-20T12:06:46.874 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2020-11-20T12:06:46.874 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2020-11-20T12:06:46.875 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2020-11-20T12:06:46.875 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2020-11-20T12:06:46.875 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2020-11-20T12:06:46.875 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2020-11-20T12:06:46.876 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2020-11-20T12:06:46.876 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2020-11-20T12:06:46.876 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2020-11-20T12:06:46.876 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2020-11-20T12:06:46.876 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2020-11-20T12:06:46.877 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2020-11-20T12:06:46.877 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2020-11-20T12:06:46.877 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2020-11-20T12:06:46.877 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2020-11-20T12:06:46.877 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2020-11-20T12:06:46.878 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2020-11-20T12:06:46.878 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2020-11-20T12:06:46.878 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2020-11-20T12:06:46.878 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2020-11-20T12:06:46.878 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2020-11-20T12:06:46.879 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2020-11-20T12:06:46.879 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2020-11-20T12:06:46.879 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2020-11-20T12:06:46.879 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2020-11-20T12:06:46.879 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2020-11-20T12:06:46.880 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2020-11-20T12:06:46.880 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2020-11-20T12:06:46.880 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2020-11-20T12:06:46.880 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2020-11-20T12:06:46.880 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2020-11-20T12:06:46.881 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2020-11-20T12:06:46.881 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2020-11-20T12:06:46.881 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2020-11-20T12:06:46.881 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2020-11-20T12:06:46.882 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2020-11-20T12:06:46.882 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2020-11-20T12:06:46.882 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2020-11-20T12:06:46.882 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2020-11-20T12:06:47.462 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2020-11-20T12:06:47.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2020-11-20T12:06:47.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2020-11-20T12:06:47.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2020-11-20T12:06:47.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2020-11-20T12:06:47.463 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2020-11-20T12:06:47.464 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2020-11-20T12:06:47.464 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2020-11-20T12:06:47.464 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2020-11-20T12:06:47.464 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2020-11-20T12:06:47.465 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2020-11-20T12:06:47.465 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2020-11-20T12:06:47.465 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2020-11-20T12:06:47.465 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2020-11-20T12:06:47.466 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2020-11-20T12:06:47.466 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2020-11-20T12:06:47.466 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2020-11-20T12:06:47.466 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2020-11-20T12:06:47.467 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2020-11-20T12:06:47.467 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2020-11-20T12:06:47.467 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2020-11-20T12:06:47.467 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2020-11-20T12:06:47.468 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2020-11-20T12:06:47.468 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2020-11-20T12:06:47.468 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2020-11-20T12:06:47.468 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2020-11-20T12:06:47.468 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2020-11-20T12:06:47.469 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2020-11-20T12:06:47.469 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2020-11-20T12:06:47.469 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2020-11-20T12:06:47.469 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2020-11-20T12:06:47.470 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2020-11-20T12:06:47.470 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2020-11-20T12:06:47.470 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2020-11-20T12:06:47.470 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2020-11-20T12:06:47.470 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2020-11-20T12:06:47.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2020-11-20T12:06:47.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2020-11-20T12:06:47.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2020-11-20T12:06:47.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2020-11-20T12:06:47.471 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2020-11-20T12:06:47.472 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2020-11-20T12:06:47.472 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2020-11-20T12:06:47.472 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2020-11-20T12:06:47.472 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2020-11-20T12:06:47.472 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2020-11-20T12:06:47.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2020-11-20T12:06:47.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2020-11-20T12:06:47.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2020-11-20T12:06:47.473 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2020-11-20T12:06:47.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2020-11-20T12:06:47.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2020-11-20T12:06:47.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2020-11-20T12:06:47.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2020-11-20T12:06:47.474 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2020-11-20T12:06:47.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2020-11-20T12:06:47.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2020-11-20T12:06:47.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2020-11-20T12:06:47.475 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2020-11-20T12:06:47.476 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2020-11-20T12:06:47.476 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2020-11-20T12:06:47.476 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2020-11-20T12:06:47.476 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2020-11-20T12:06:47.476 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2020-11-20T12:06:47.477 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2020-11-20T12:06:47.477 INFO:tasks.workunit.client.0.smithi044.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2020-11-20T12:06:47.477 INFO:tasks.workunit.client.0.smithi044.stdout:Take third snapshot .snap/test3 2020-11-20T12:06:47.567 INFO:tasks.workunit.client.0.smithi044.stdout:Delete the snapshots... 2020-11-20T12:06:48.606 INFO:tasks.workunit.client.0.smithi044.stdout:.//.snap/test1 2020-11-20T12:06:48.606 INFO:tasks.workunit.client.0.smithi044.stdout:.//.snap/test2 2020-11-20T12:06:48.606 INFO:tasks.workunit.client.0.smithi044.stdout:.//.snap/test3 2020-11-20T12:06:48.614 INFO:tasks.workunit.client.0.smithi044.stdout:./274/.snap/snap-subdir-test 2020-11-20T12:06:48.623 INFO:tasks.workunit.client.0.smithi044.stdout:./166/.snap/snap-subdir-test 2020-11-20T12:06:48.632 INFO:tasks.workunit.client.0.smithi044.stdout:./256/.snap/snap-subdir-test 2020-11-20T12:06:48.641 INFO:tasks.workunit.client.0.smithi044.stdout:./247/.snap/snap-subdir-test 2020-11-20T12:06:48.649 INFO:tasks.workunit.client.0.smithi044.stdout:./248/.snap/snap-subdir-test 2020-11-20T12:06:48.658 INFO:tasks.workunit.client.0.smithi044.stdout:./147/.snap/snap-subdir-test 2020-11-20T12:06:48.668 INFO:tasks.workunit.client.0.smithi044.stdout:./151/.snap/snap-subdir-test 2020-11-20T12:06:48.677 INFO:tasks.workunit.client.0.smithi044.stdout:./170/.snap/snap-subdir-test 2020-11-20T12:06:48.688 INFO:tasks.workunit.client.0.smithi044.stdout:./167/.snap/snap-subdir-test 2020-11-20T12:06:48.698 INFO:tasks.workunit.client.0.smithi044.stdout:./140/.snap/snap-subdir-test 2020-11-20T12:06:48.707 INFO:tasks.workunit.client.0.smithi044.stdout:./142/.snap/snap-subdir-test 2020-11-20T12:06:48.716 INFO:tasks.workunit.client.0.smithi044.stdout:./150/.snap/snap-subdir-test 2020-11-20T12:06:48.725 INFO:tasks.workunit.client.0.smithi044.stdout:./273/.snap/snap-subdir-test 2020-11-20T12:06:48.733 INFO:tasks.workunit.client.0.smithi044.stdout:./269/.snap/snap-subdir-test 2020-11-20T12:06:48.777 INFO:tasks.workunit.client.0.smithi044.stdout:./105/.snap/snap-subdir-test 2020-11-20T12:06:48.878 INFO:tasks.workunit.client.0.smithi044.stdout:./210/.snap/snap-subdir-test 2020-11-20T12:06:48.887 INFO:tasks.workunit.client.0.smithi044.stdout:./265/.snap/snap-subdir-test 2020-11-20T12:06:48.897 INFO:tasks.workunit.client.0.smithi044.stdout:./218/.snap/snap-subdir-test 2020-11-20T12:06:48.906 INFO:tasks.workunit.client.0.smithi044.stdout:./290/.snap/snap-subdir-test 2020-11-20T12:06:48.918 INFO:tasks.workunit.client.0.smithi044.stdout:./149/.snap/snap-subdir-test 2020-11-20T12:06:48.928 INFO:tasks.workunit.client.0.smithi044.stdout:./109/.snap/snap-subdir-test 2020-11-20T12:06:48.937 INFO:tasks.workunit.client.0.smithi044.stdout:./292/.snap/snap-subdir-test 2020-11-20T12:06:48.947 INFO:tasks.workunit.client.0.smithi044.stdout:./242/.snap/snap-subdir-test 2020-11-20T12:06:48.956 INFO:tasks.workunit.client.0.smithi044.stdout:./158/.snap/snap-subdir-test 2020-11-20T12:06:48.966 INFO:tasks.workunit.client.0.smithi044.stdout:./243/.snap/snap-subdir-test 2020-11-20T12:06:48.977 INFO:tasks.workunit.client.0.smithi044.stdout:./106/.snap/snap-subdir-test 2020-11-20T12:06:48.988 INFO:tasks.workunit.client.0.smithi044.stdout:./250/.snap/snap-subdir-test 2020-11-20T12:06:48.998 INFO:tasks.workunit.client.0.smithi044.stdout:./221/.snap/snap-subdir-test 2020-11-20T12:06:49.009 INFO:tasks.workunit.client.0.smithi044.stdout:./244/.snap/snap-subdir-test 2020-11-20T12:06:49.018 INFO:tasks.workunit.client.0.smithi044.stdout:./258/.snap/snap-subdir-test 2020-11-20T12:06:49.027 INFO:tasks.workunit.client.0.smithi044.stdout:./268/.snap/snap-subdir-test 2020-11-20T12:06:49.036 INFO:tasks.workunit.client.0.smithi044.stdout:./189/.snap/snap-subdir-test 2020-11-20T12:06:49.045 INFO:tasks.workunit.client.0.smithi044.stdout:./285/.snap/snap-subdir-test 2020-11-20T12:06:49.054 INFO:tasks.workunit.client.0.smithi044.stdout:./208/.snap/snap-subdir-test 2020-11-20T12:06:49.063 INFO:tasks.workunit.client.0.smithi044.stdout:./288/.snap/snap-subdir-test 2020-11-20T12:06:49.072 INFO:tasks.workunit.client.0.smithi044.stdout:./209/.snap/snap-subdir-test 2020-11-20T12:06:49.081 INFO:tasks.workunit.client.0.smithi044.stdout:./253/.snap/snap-subdir-test 2020-11-20T12:06:49.090 INFO:tasks.workunit.client.0.smithi044.stdout:./249/.snap/snap-subdir-test 2020-11-20T12:06:49.099 INFO:tasks.workunit.client.0.smithi044.stdout:./133/.snap/snap-subdir-test 2020-11-20T12:06:49.108 INFO:tasks.workunit.client.0.smithi044.stdout:./291/.snap/snap-subdir-test 2020-11-20T12:06:49.117 INFO:tasks.workunit.client.0.smithi044.stdout:./226/.snap/snap-subdir-test 2020-11-20T12:06:49.130 INFO:tasks.workunit.client.0.smithi044.stdout:./173/.snap/snap-subdir-test 2020-11-20T12:06:49.192 INFO:tasks.workunit.client.0.smithi044.stdout:./174/.snap/snap-subdir-test 2020-11-20T12:06:49.202 INFO:tasks.workunit.client.0.smithi044.stdout:./299/.snap/snap-subdir-test 2020-11-20T12:06:49.210 INFO:tasks.workunit.client.0.smithi044.stdout:./262/.snap/snap-subdir-test 2020-11-20T12:06:49.219 INFO:tasks.workunit.client.0.smithi044.stdout:./231/.snap/snap-subdir-test 2020-11-20T12:06:49.228 INFO:tasks.workunit.client.0.smithi044.stdout:./214/.snap/snap-subdir-test 2020-11-20T12:06:49.236 INFO:tasks.workunit.client.0.smithi044.stdout:./281/.snap/snap-subdir-test 2020-11-20T12:06:49.246 INFO:tasks.workunit.client.0.smithi044.stdout:./289/.snap/snap-subdir-test 2020-11-20T12:06:49.255 INFO:tasks.workunit.client.0.smithi044.stdout:./283/.snap/snap-subdir-test 2020-11-20T12:06:49.264 INFO:tasks.workunit.client.0.smithi044.stdout:./232/.snap/snap-subdir-test 2020-11-20T12:06:49.273 INFO:tasks.workunit.client.0.smithi044.stdout:./298/.snap/snap-subdir-test 2020-11-20T12:06:49.282 INFO:tasks.workunit.client.0.smithi044.stdout:./233/.snap/snap-subdir-test 2020-11-20T12:06:49.292 INFO:tasks.workunit.client.0.smithi044.stdout:./155/.snap/snap-subdir-test 2020-11-20T12:06:49.303 INFO:tasks.workunit.client.0.smithi044.stdout:./114/.snap/snap-subdir-test 2020-11-20T12:06:49.311 INFO:tasks.workunit.client.0.smithi044.stdout:./159/.snap/snap-subdir-test 2020-11-20T12:06:49.321 INFO:tasks.workunit.client.0.smithi044.stdout:./187/.snap/snap-subdir-test 2020-11-20T12:06:49.330 INFO:tasks.workunit.client.0.smithi044.stdout:./185/.snap/snap-subdir-test 2020-11-20T12:06:49.339 INFO:tasks.workunit.client.0.smithi044.stdout:./293/.snap/snap-subdir-test 2020-11-20T12:06:49.348 INFO:tasks.workunit.client.0.smithi044.stdout:./228/.snap/snap-subdir-test 2020-11-20T12:06:49.357 INFO:tasks.workunit.client.0.smithi044.stdout:./120/.snap/snap-subdir-test 2020-11-20T12:06:49.372 INFO:tasks.workunit.client.0.smithi044.stdout:./110/.snap/snap-subdir-test 2020-11-20T12:06:49.377 INFO:tasks.workunit.client.0.smithi044.stdout:./267/.snap/snap-subdir-test 2020-11-20T12:06:49.385 INFO:tasks.workunit.client.0.smithi044.stdout:./202/.snap/snap-subdir-test 2020-11-20T12:06:49.395 INFO:tasks.workunit.client.0.smithi044.stdout:./137/.snap/snap-subdir-test 2020-11-20T12:06:49.404 INFO:tasks.workunit.client.0.smithi044.stdout:./246/.snap/snap-subdir-test 2020-11-20T12:06:49.413 INFO:tasks.workunit.client.0.smithi044.stdout:./219/.snap/snap-subdir-test 2020-11-20T12:06:49.423 INFO:tasks.workunit.client.0.smithi044.stdout:./198/.snap/snap-subdir-test 2020-11-20T12:06:49.433 INFO:tasks.workunit.client.0.smithi044.stdout:./136/.snap/snap-subdir-test 2020-11-20T12:06:49.441 INFO:tasks.workunit.client.0.smithi044.stdout:./270/.snap/snap-subdir-test 2020-11-20T12:06:49.451 INFO:tasks.workunit.client.0.smithi044.stdout:./125/.snap/snap-subdir-test 2020-11-20T12:06:49.516 INFO:tasks.workunit.client.0.smithi044.stdout:./194/.snap/snap-subdir-test 2020-11-20T12:06:49.516 INFO:tasks.workunit.client.0.smithi044.stdout:./119/.snap/snap-subdir-test 2020-11-20T12:06:49.517 INFO:tasks.workunit.client.0.smithi044.stdout:./297/.snap/snap-subdir-test 2020-11-20T12:06:49.517 INFO:tasks.workunit.client.0.smithi044.stdout:./230/.snap/snap-subdir-test 2020-11-20T12:06:49.517 INFO:tasks.workunit.client.0.smithi044.stdout:./260/.snap/snap-subdir-test 2020-11-20T12:06:49.517 INFO:tasks.workunit.client.0.smithi044.stdout:./157/.snap/snap-subdir-test 2020-11-20T12:06:49.518 INFO:tasks.workunit.client.0.smithi044.stdout:./172/.snap/snap-subdir-test 2020-11-20T12:06:49.525 INFO:tasks.workunit.client.0.smithi044.stdout:./277/.snap/snap-subdir-test 2020-11-20T12:06:49.534 INFO:tasks.workunit.client.0.smithi044.stdout:./186/.snap/snap-subdir-test 2020-11-20T12:06:49.543 INFO:tasks.workunit.client.0.smithi044.stdout:./237/.snap/snap-subdir-test 2020-11-20T12:06:49.554 INFO:tasks.workunit.client.0.smithi044.stdout:./190/.snap/snap-subdir-test 2020-11-20T12:06:49.561 INFO:tasks.workunit.client.0.smithi044.stdout:./213/.snap/snap-subdir-test 2020-11-20T12:06:49.571 INFO:tasks.workunit.client.0.smithi044.stdout:./104/.snap/snap-subdir-test 2020-11-20T12:06:49.580 INFO:tasks.workunit.client.0.smithi044.stdout:./164/.snap/snap-subdir-test 2020-11-20T12:06:49.590 INFO:tasks.workunit.client.0.smithi044.stdout:./112/.snap/snap-subdir-test 2020-11-20T12:06:49.599 INFO:tasks.workunit.client.0.smithi044.stdout:./295/.snap/snap-subdir-test 2020-11-20T12:06:49.606 INFO:tasks.workunit.client.0.smithi044.stdout:./272/.snap/snap-subdir-test 2020-11-20T12:06:49.616 INFO:tasks.workunit.client.0.smithi044.stdout:./100/.snap/snap-subdir-test 2020-11-20T12:06:49.624 INFO:tasks.workunit.client.0.smithi044.stdout:./134/.snap/snap-subdir-test 2020-11-20T12:06:49.633 INFO:tasks.workunit.client.0.smithi044.stdout:./200/.snap/snap-subdir-test 2020-11-20T12:06:49.643 INFO:tasks.workunit.client.0.smithi044.stdout:./115/.snap/snap-subdir-test 2020-11-20T12:06:49.652 INFO:tasks.workunit.client.0.smithi044.stdout:./284/.snap/snap-subdir-test 2020-11-20T12:06:49.661 INFO:tasks.workunit.client.0.smithi044.stdout:./280/.snap/snap-subdir-test 2020-11-20T12:06:49.670 INFO:tasks.workunit.client.0.smithi044.stdout:./116/.snap/snap-subdir-test 2020-11-20T12:06:49.679 INFO:tasks.workunit.client.0.smithi044.stdout:./245/.snap/snap-subdir-test 2020-11-20T12:06:49.691 INFO:tasks.workunit.client.0.smithi044.stdout:./195/.snap/snap-subdir-test 2020-11-20T12:06:49.701 INFO:tasks.workunit.client.0.smithi044.stdout:./138/.snap/snap-subdir-test 2020-11-20T12:06:49.710 INFO:tasks.workunit.client.0.smithi044.stdout:./135/.snap/snap-subdir-test 2020-11-20T12:06:49.721 INFO:tasks.workunit.client.0.smithi044.stdout:./117/.snap/snap-subdir-test 2020-11-20T12:06:49.729 INFO:tasks.workunit.client.0.smithi044.stdout:./241/.snap/snap-subdir-test 2020-11-20T12:06:49.739 INFO:tasks.workunit.client.0.smithi044.stdout:./143/.snap/snap-subdir-test 2020-11-20T12:06:49.749 INFO:tasks.workunit.client.0.smithi044.stdout:./180/.snap/snap-subdir-test 2020-11-20T12:06:49.757 INFO:tasks.workunit.client.0.smithi044.stdout:./126/.snap/snap-subdir-test 2020-11-20T12:06:49.767 INFO:tasks.workunit.client.0.smithi044.stdout:./161/.snap/snap-subdir-test 2020-11-20T12:06:49.777 INFO:tasks.workunit.client.0.smithi044.stdout:./204/.snap/snap-subdir-test 2020-11-20T12:06:49.785 INFO:tasks.workunit.client.0.smithi044.stdout:./176/.snap/snap-subdir-test 2020-11-20T12:06:49.794 INFO:tasks.workunit.client.0.smithi044.stdout:./255/.snap/snap-subdir-test 2020-11-20T12:06:49.803 INFO:tasks.workunit.client.0.smithi044.stdout:./215/.snap/snap-subdir-test 2020-11-20T12:06:49.813 INFO:tasks.workunit.client.0.smithi044.stdout:./111/.snap/snap-subdir-test 2020-11-20T12:06:49.822 INFO:tasks.workunit.client.0.smithi044.stdout:./238/.snap/snap-subdir-test 2020-11-20T12:06:49.831 INFO:tasks.workunit.client.0.smithi044.stdout:./191/.snap/snap-subdir-test 2020-11-20T12:06:49.841 INFO:tasks.workunit.client.0.smithi044.stdout:./184/.snap/snap-subdir-test 2020-11-20T12:06:49.851 INFO:tasks.workunit.client.0.smithi044.stdout:./192/.snap/snap-subdir-test 2020-11-20T12:06:49.861 INFO:tasks.workunit.client.0.smithi044.stdout:./103/.snap/snap-subdir-test 2020-11-20T12:06:49.870 INFO:tasks.workunit.client.0.smithi044.stdout:./128/.snap/snap-subdir-test 2020-11-20T12:06:49.881 INFO:tasks.workunit.client.0.smithi044.stdout:./229/.snap/snap-subdir-test 2020-11-20T12:06:49.891 INFO:tasks.workunit.client.0.smithi044.stdout:./144/.snap/snap-subdir-test 2020-11-20T12:06:49.901 INFO:tasks.workunit.client.0.smithi044.stdout:./146/.snap/snap-subdir-test 2020-11-20T12:06:49.910 INFO:tasks.workunit.client.0.smithi044.stdout:./227/.snap/snap-subdir-test 2020-11-20T12:06:49.919 INFO:tasks.workunit.client.0.smithi044.stdout:./278/.snap/snap-subdir-test 2020-11-20T12:06:49.928 INFO:tasks.workunit.client.0.smithi044.stdout:./141/.snap/snap-subdir-test 2020-11-20T12:06:49.937 INFO:tasks.workunit.client.0.smithi044.stdout:./234/.snap/snap-subdir-test 2020-11-20T12:06:49.946 INFO:tasks.workunit.client.0.smithi044.stdout:./225/.snap/snap-subdir-test 2020-11-20T12:06:49.956 INFO:tasks.workunit.client.0.smithi044.stdout:./182/.snap/snap-subdir-test 2020-11-20T12:06:49.965 INFO:tasks.workunit.client.0.smithi044.stdout:./107/.snap/snap-subdir-test 2020-11-20T12:06:49.975 INFO:tasks.workunit.client.0.smithi044.stdout:./178/.snap/snap-subdir-test 2020-11-20T12:06:49.984 INFO:tasks.workunit.client.0.smithi044.stdout:./266/.snap/snap-subdir-test 2020-11-20T12:06:49.993 INFO:tasks.workunit.client.0.smithi044.stdout:./240/.snap/snap-subdir-test 2020-11-20T12:06:50.002 INFO:tasks.workunit.client.0.smithi044.stdout:./156/.snap/snap-subdir-test 2020-11-20T12:06:50.012 INFO:tasks.workunit.client.0.smithi044.stdout:./131/.snap/snap-subdir-test 2020-11-20T12:06:50.021 INFO:tasks.workunit.client.0.smithi044.stdout:./123/.snap/snap-subdir-test 2020-11-20T12:06:50.031 INFO:tasks.workunit.client.0.smithi044.stdout:./160/.snap/snap-subdir-test 2020-11-20T12:06:50.042 INFO:tasks.workunit.client.0.smithi044.stdout:./165/.snap/snap-subdir-test 2020-11-20T12:06:50.050 INFO:tasks.workunit.client.0.smithi044.stdout:./101/.snap/snap-subdir-test 2020-11-20T12:06:50.059 INFO:tasks.workunit.client.0.smithi044.stdout:./148/.snap/snap-subdir-test 2020-11-20T12:06:50.069 INFO:tasks.workunit.client.0.smithi044.stdout:./168/.snap/snap-subdir-test 2020-11-20T12:06:50.079 INFO:tasks.workunit.client.0.smithi044.stdout:./179/.snap/snap-subdir-test 2020-11-20T12:06:50.088 INFO:tasks.workunit.client.0.smithi044.stdout:./257/.snap/snap-subdir-test 2020-11-20T12:06:50.096 INFO:tasks.workunit.client.0.smithi044.stdout:./252/.snap/snap-subdir-test 2020-11-20T12:06:50.105 INFO:tasks.workunit.client.0.smithi044.stdout:./132/.snap/snap-subdir-test 2020-11-20T12:06:50.114 INFO:tasks.workunit.client.0.smithi044.stdout:./236/.snap/snap-subdir-test 2020-11-20T12:06:50.123 INFO:tasks.workunit.client.0.smithi044.stdout:./211/.snap/snap-subdir-test 2020-11-20T12:06:50.132 INFO:tasks.workunit.client.0.smithi044.stdout:./199/.snap/snap-subdir-test 2020-11-20T12:06:50.141 INFO:tasks.workunit.client.0.smithi044.stdout:./205/.snap/snap-subdir-test 2020-11-20T12:06:50.152 INFO:tasks.workunit.client.0.smithi044.stdout:./163/.snap/snap-subdir-test 2020-11-20T12:06:50.161 INFO:tasks.workunit.client.0.smithi044.stdout:./261/.snap/snap-subdir-test 2020-11-20T12:06:50.219 INFO:tasks.workunit.client.0.smithi044.stdout:./201/.snap/snap-subdir-test 2020-11-20T12:06:50.228 INFO:tasks.workunit.client.0.smithi044.stdout:./282/.snap/snap-subdir-test 2020-11-20T12:06:50.237 INFO:tasks.workunit.client.0.smithi044.stdout:./296/.snap/snap-subdir-test 2020-11-20T12:06:50.246 INFO:tasks.workunit.client.0.smithi044.stdout:./217/.snap/snap-subdir-test 2020-11-20T12:06:50.257 INFO:tasks.workunit.client.0.smithi044.stdout:./153/.snap/snap-subdir-test 2020-11-20T12:06:50.266 INFO:tasks.workunit.client.0.smithi044.stdout:./206/.snap/snap-subdir-test 2020-11-20T12:06:50.275 INFO:tasks.workunit.client.0.smithi044.stdout:./122/.snap/snap-subdir-test 2020-11-20T12:06:50.285 INFO:tasks.workunit.client.0.smithi044.stdout:./162/.snap/snap-subdir-test 2020-11-20T12:06:50.295 INFO:tasks.workunit.client.0.smithi044.stdout:./197/.snap/snap-subdir-test 2020-11-20T12:06:50.305 INFO:tasks.workunit.client.0.smithi044.stdout:./216/.snap/snap-subdir-test 2020-11-20T12:06:51.022 INFO:tasks.workunit.client.0.smithi044.stdout:./254/.snap/snap-subdir-test 2020-11-20T12:06:51.032 INFO:tasks.workunit.client.0.smithi044.stdout:./113/.snap/snap-subdir-test 2020-11-20T12:06:51.041 INFO:tasks.workunit.client.0.smithi044.stdout:./152/.snap/snap-subdir-test 2020-11-20T12:06:51.050 INFO:tasks.workunit.client.0.smithi044.stdout:./121/.snap/snap-subdir-test 2020-11-20T12:06:51.059 INFO:tasks.workunit.client.0.smithi044.stdout:./118/.snap/snap-subdir-test 2020-11-20T12:06:51.069 INFO:tasks.workunit.client.0.smithi044.stdout:./196/.snap/snap-subdir-test 2020-11-20T12:06:51.078 INFO:tasks.workunit.client.0.smithi044.stdout:./124/.snap/snap-subdir-test 2020-11-20T12:06:51.087 INFO:tasks.workunit.client.0.smithi044.stdout:./239/.snap/snap-subdir-test 2020-11-20T12:06:51.097 INFO:tasks.workunit.client.0.smithi044.stdout:./188/.snap/snap-subdir-test 2020-11-20T12:06:51.105 INFO:tasks.workunit.client.0.smithi044.stdout:./287/.snap/snap-subdir-test 2020-11-20T12:06:51.114 INFO:tasks.workunit.client.0.smithi044.stdout:./275/.snap/snap-subdir-test 2020-11-20T12:06:51.122 INFO:tasks.workunit.client.0.smithi044.stdout:./259/.snap/snap-subdir-test 2020-11-20T12:06:51.132 INFO:tasks.workunit.client.0.smithi044.stdout:./224/.snap/snap-subdir-test 2020-11-20T12:06:51.141 INFO:tasks.workunit.client.0.smithi044.stdout:./264/.snap/snap-subdir-test 2020-11-20T12:06:51.151 INFO:tasks.workunit.client.0.smithi044.stdout:./145/.snap/snap-subdir-test 2020-11-20T12:06:51.163 INFO:tasks.workunit.client.0.smithi044.stdout:./279/.snap/snap-subdir-test 2020-11-20T12:06:51.184 INFO:tasks.workunit.client.0.smithi044.stdout:./263/.snap/snap-subdir-test 2020-11-20T12:06:51.241 INFO:tasks.workunit.client.0.smithi044.stdout:./130/.snap/snap-subdir-test 2020-11-20T12:06:51.287 INFO:tasks.workunit.client.0.smithi044.stdout:./193/.snap/snap-subdir-test 2020-11-20T12:06:51.319 INFO:tasks.workunit.client.0.smithi044.stdout:./223/.snap/snap-subdir-test 2020-11-20T12:06:51.328 INFO:tasks.workunit.client.0.smithi044.stdout:./207/.snap/snap-subdir-test 2020-11-20T12:06:51.337 INFO:tasks.workunit.client.0.smithi044.stdout:./276/.snap/snap-subdir-test 2020-11-20T12:06:51.346 INFO:tasks.workunit.client.0.smithi044.stdout:./271/.snap/snap-subdir-test 2020-11-20T12:06:51.355 INFO:tasks.workunit.client.0.smithi044.stdout:./251/.snap/snap-subdir-test 2020-11-20T12:06:51.364 INFO:tasks.workunit.client.0.smithi044.stdout:./175/.snap/snap-subdir-test 2020-11-20T12:06:51.374 INFO:tasks.workunit.client.0.smithi044.stdout:./222/.snap/snap-subdir-test 2020-11-20T12:06:51.385 INFO:tasks.workunit.client.0.smithi044.stdout:./220/.snap/snap-subdir-test 2020-11-20T12:06:51.394 INFO:tasks.workunit.client.0.smithi044.stdout:./203/.snap/snap-subdir-test 2020-11-20T12:06:51.405 INFO:tasks.workunit.client.0.smithi044.stdout:./181/.snap/snap-subdir-test 2020-11-20T12:06:51.415 INFO:tasks.workunit.client.0.smithi044.stdout:./286/.snap/snap-subdir-test 2020-11-20T12:06:51.424 INFO:tasks.workunit.client.0.smithi044.stdout:./127/.snap/snap-subdir-test 2020-11-20T12:06:51.434 INFO:tasks.workunit.client.0.smithi044.stdout:./108/.snap/snap-subdir-test 2020-11-20T12:06:51.443 INFO:tasks.workunit.client.0.smithi044.stdout:./183/.snap/snap-subdir-test 2020-11-20T12:06:51.452 INFO:tasks.workunit.client.0.smithi044.stdout:./294/.snap/snap-subdir-test 2020-11-20T12:06:51.461 INFO:tasks.workunit.client.0.smithi044.stdout:./235/.snap/snap-subdir-test 2020-11-20T12:06:51.470 INFO:tasks.workunit.client.0.smithi044.stdout:./177/.snap/snap-subdir-test 2020-11-20T12:06:51.480 INFO:tasks.workunit.client.0.smithi044.stdout:./169/.snap/snap-subdir-test 2020-11-20T12:06:51.489 INFO:tasks.workunit.client.0.smithi044.stdout:./212/.snap/snap-subdir-test 2020-11-20T12:06:51.500 INFO:tasks.workunit.client.0.smithi044.stdout:./154/.snap/snap-subdir-test 2020-11-20T12:06:51.508 INFO:tasks.workunit.client.0.smithi044.stdout:./171/.snap/snap-subdir-test 2020-11-20T12:06:51.517 INFO:tasks.workunit.client.0.smithi044.stdout:./139/.snap/snap-subdir-test 2020-11-20T12:06:51.527 INFO:tasks.workunit.client.0.smithi044.stdout:./102/.snap/snap-subdir-test 2020-11-20T12:06:51.537 INFO:tasks.workunit.client.0.smithi044.stdout:./129/.snap/snap-subdir-test 2020-11-20T12:06:53.974 INFO:tasks.workunit.client.0.smithi044.stdout:Delete all the files and directories ... 2020-11-20T12:06:54.515 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/19' 2020-11-20T12:06:54.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/15' 2020-11-20T12:06:54.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/31' 2020-11-20T12:06:54.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/36' 2020-11-20T12:06:54.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/20' 2020-11-20T12:06:54.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/12' 2020-11-20T12:06:54.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/39' 2020-11-20T12:06:54.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/10' 2020-11-20T12:06:54.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/34' 2020-11-20T12:06:54.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/38' 2020-11-20T12:06:54.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/11' 2020-11-20T12:06:54.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/16' 2020-11-20T12:06:54.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/32' 2020-11-20T12:06:54.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/30' 2020-11-20T12:06:54.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/14' 2020-11-20T12:06:54.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/18' 2020-11-20T12:06:54.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/13' 2020-11-20T12:06:54.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/33' 2020-11-20T12:06:54.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/17' 2020-11-20T12:06:54.520 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/35' 2020-11-20T12:06:54.520 INFO:tasks.workunit.client.0.smithi044.stdout:removed './100/37' 2020-11-20T12:06:54.520 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './100' 2020-11-20T12:06:54.520 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/19' 2020-11-20T12:06:54.521 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/15' 2020-11-20T12:06:54.521 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/31' 2020-11-20T12:06:54.521 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/36' 2020-11-20T12:06:54.521 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/20' 2020-11-20T12:06:54.521 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/12' 2020-11-20T12:06:54.522 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/39' 2020-11-20T12:06:54.522 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/10' 2020-11-20T12:06:54.522 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/34' 2020-11-20T12:06:54.522 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/38' 2020-11-20T12:06:54.522 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/11' 2020-11-20T12:06:54.523 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/16' 2020-11-20T12:06:54.523 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/32' 2020-11-20T12:06:54.523 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/30' 2020-11-20T12:06:54.523 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/14' 2020-11-20T12:06:54.523 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/18' 2020-11-20T12:06:54.524 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/13' 2020-11-20T12:06:54.524 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/33' 2020-11-20T12:06:54.524 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/17' 2020-11-20T12:06:54.524 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/35' 2020-11-20T12:06:54.524 INFO:tasks.workunit.client.0.smithi044.stdout:removed './101/37' 2020-11-20T12:06:54.525 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './101' 2020-11-20T12:06:54.525 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/19' 2020-11-20T12:06:54.525 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/15' 2020-11-20T12:06:54.525 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/31' 2020-11-20T12:06:54.525 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/36' 2020-11-20T12:06:54.526 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/20' 2020-11-20T12:06:54.526 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/12' 2020-11-20T12:06:54.526 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/39' 2020-11-20T12:06:54.526 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/10' 2020-11-20T12:06:54.526 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/34' 2020-11-20T12:06:54.527 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/38' 2020-11-20T12:06:54.527 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/11' 2020-11-20T12:06:54.527 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/16' 2020-11-20T12:06:54.527 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/32' 2020-11-20T12:06:54.527 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/30' 2020-11-20T12:06:54.528 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/14' 2020-11-20T12:06:54.529 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/18' 2020-11-20T12:06:54.529 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/13' 2020-11-20T12:06:54.529 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/33' 2020-11-20T12:06:54.530 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/17' 2020-11-20T12:06:54.530 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/35' 2020-11-20T12:06:54.531 INFO:tasks.workunit.client.0.smithi044.stdout:removed './102/37' 2020-11-20T12:06:54.531 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './102' 2020-11-20T12:06:54.531 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/19' 2020-11-20T12:06:54.532 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/15' 2020-11-20T12:06:54.532 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/31' 2020-11-20T12:06:54.533 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/36' 2020-11-20T12:06:54.533 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/20' 2020-11-20T12:06:54.533 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/12' 2020-11-20T12:06:54.534 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/39' 2020-11-20T12:06:54.534 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/10' 2020-11-20T12:06:54.535 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/34' 2020-11-20T12:06:54.535 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/38' 2020-11-20T12:06:54.535 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/11' 2020-11-20T12:06:54.536 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/16' 2020-11-20T12:06:54.536 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/32' 2020-11-20T12:06:54.536 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/30' 2020-11-20T12:06:54.537 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/14' 2020-11-20T12:06:54.537 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/18' 2020-11-20T12:06:54.538 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/13' 2020-11-20T12:06:54.538 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/33' 2020-11-20T12:06:54.538 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/17' 2020-11-20T12:06:54.539 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/35' 2020-11-20T12:06:54.539 INFO:tasks.workunit.client.0.smithi044.stdout:removed './103/37' 2020-11-20T12:06:54.540 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './103' 2020-11-20T12:06:54.540 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/19' 2020-11-20T12:06:54.540 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/15' 2020-11-20T12:06:54.541 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/31' 2020-11-20T12:06:54.541 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/36' 2020-11-20T12:06:54.542 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/20' 2020-11-20T12:06:54.542 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/12' 2020-11-20T12:06:54.542 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/39' 2020-11-20T12:06:54.543 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/10' 2020-11-20T12:06:54.543 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/34' 2020-11-20T12:06:54.543 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/38' 2020-11-20T12:06:54.544 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/11' 2020-11-20T12:06:54.544 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/16' 2020-11-20T12:06:54.545 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/32' 2020-11-20T12:06:54.545 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/30' 2020-11-20T12:06:54.545 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/14' 2020-11-20T12:06:54.546 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/18' 2020-11-20T12:06:54.546 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/13' 2020-11-20T12:06:54.547 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/33' 2020-11-20T12:06:54.547 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/17' 2020-11-20T12:06:54.547 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/35' 2020-11-20T12:06:54.548 INFO:tasks.workunit.client.0.smithi044.stdout:removed './104/37' 2020-11-20T12:06:54.548 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './104' 2020-11-20T12:06:54.549 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/19' 2020-11-20T12:06:54.549 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/15' 2020-11-20T12:06:54.549 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/31' 2020-11-20T12:06:54.550 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/36' 2020-11-20T12:06:54.550 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/20' 2020-11-20T12:06:54.551 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/12' 2020-11-20T12:06:54.551 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/39' 2020-11-20T12:06:54.551 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/10' 2020-11-20T12:06:54.552 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/34' 2020-11-20T12:06:54.552 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/38' 2020-11-20T12:06:54.552 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/11' 2020-11-20T12:06:54.553 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/16' 2020-11-20T12:06:54.553 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/32' 2020-11-20T12:06:54.554 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/30' 2020-11-20T12:06:54.554 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/14' 2020-11-20T12:06:54.554 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/18' 2020-11-20T12:06:54.555 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/13' 2020-11-20T12:06:54.555 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/33' 2020-11-20T12:06:54.556 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/17' 2020-11-20T12:06:54.556 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/35' 2020-11-20T12:06:54.556 INFO:tasks.workunit.client.0.smithi044.stdout:removed './105/37' 2020-11-20T12:06:54.557 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './105' 2020-11-20T12:06:54.557 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/19' 2020-11-20T12:06:54.557 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/15' 2020-11-20T12:06:54.558 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/31' 2020-11-20T12:06:54.558 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/36' 2020-11-20T12:06:54.559 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/20' 2020-11-20T12:06:54.559 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/12' 2020-11-20T12:06:54.559 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/39' 2020-11-20T12:06:54.560 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/10' 2020-11-20T12:06:54.560 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/34' 2020-11-20T12:06:54.561 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/38' 2020-11-20T12:06:54.561 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/11' 2020-11-20T12:06:54.561 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/16' 2020-11-20T12:06:54.562 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/32' 2020-11-20T12:06:54.562 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/30' 2020-11-20T12:06:54.563 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/14' 2020-11-20T12:06:54.563 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/18' 2020-11-20T12:06:54.563 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/13' 2020-11-20T12:06:54.564 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/33' 2020-11-20T12:06:54.564 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/17' 2020-11-20T12:06:54.565 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/35' 2020-11-20T12:06:54.565 INFO:tasks.workunit.client.0.smithi044.stdout:removed './106/37' 2020-11-20T12:06:54.565 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './106' 2020-11-20T12:06:54.566 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/19' 2020-11-20T12:06:54.566 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/15' 2020-11-20T12:06:54.566 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/31' 2020-11-20T12:06:54.567 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/36' 2020-11-20T12:06:54.567 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/20' 2020-11-20T12:06:54.568 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/12' 2020-11-20T12:06:54.568 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/39' 2020-11-20T12:06:54.568 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/10' 2020-11-20T12:06:54.569 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/34' 2020-11-20T12:06:54.569 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/38' 2020-11-20T12:06:54.570 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/11' 2020-11-20T12:06:54.570 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/16' 2020-11-20T12:06:54.570 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/32' 2020-11-20T12:06:54.571 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/30' 2020-11-20T12:06:54.571 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/14' 2020-11-20T12:06:54.571 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/18' 2020-11-20T12:06:54.572 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/13' 2020-11-20T12:06:54.572 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/33' 2020-11-20T12:06:54.573 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/17' 2020-11-20T12:06:54.573 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/35' 2020-11-20T12:06:54.573 INFO:tasks.workunit.client.0.smithi044.stdout:removed './107/37' 2020-11-20T12:06:54.574 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './107' 2020-11-20T12:06:54.574 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/19' 2020-11-20T12:06:54.575 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/15' 2020-11-20T12:06:54.575 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/31' 2020-11-20T12:06:54.575 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/36' 2020-11-20T12:06:54.576 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/20' 2020-11-20T12:06:54.576 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/12' 2020-11-20T12:06:54.577 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/39' 2020-11-20T12:06:54.577 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/10' 2020-11-20T12:06:54.577 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/34' 2020-11-20T12:06:54.578 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/38' 2020-11-20T12:06:54.578 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/11' 2020-11-20T12:06:54.578 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/16' 2020-11-20T12:06:54.579 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/32' 2020-11-20T12:06:54.579 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/30' 2020-11-20T12:06:54.580 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/14' 2020-11-20T12:06:54.580 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/18' 2020-11-20T12:06:54.580 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/13' 2020-11-20T12:06:54.580 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/33' 2020-11-20T12:06:54.580 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/17' 2020-11-20T12:06:54.581 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/35' 2020-11-20T12:06:54.581 INFO:tasks.workunit.client.0.smithi044.stdout:removed './108/37' 2020-11-20T12:06:54.581 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './108' 2020-11-20T12:06:54.581 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/19' 2020-11-20T12:06:54.581 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/15' 2020-11-20T12:06:54.582 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/31' 2020-11-20T12:06:54.582 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/36' 2020-11-20T12:06:54.582 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/20' 2020-11-20T12:06:54.582 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/12' 2020-11-20T12:06:54.582 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/39' 2020-11-20T12:06:54.583 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/10' 2020-11-20T12:06:54.583 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/34' 2020-11-20T12:06:54.583 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/38' 2020-11-20T12:06:54.583 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/11' 2020-11-20T12:06:54.583 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/16' 2020-11-20T12:06:54.584 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/32' 2020-11-20T12:06:54.584 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/30' 2020-11-20T12:06:55.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/14' 2020-11-20T12:06:55.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/18' 2020-11-20T12:06:55.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/13' 2020-11-20T12:06:55.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/33' 2020-11-20T12:06:55.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/17' 2020-11-20T12:06:55.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/35' 2020-11-20T12:06:55.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './109/37' 2020-11-20T12:06:55.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './109' 2020-11-20T12:06:55.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/19' 2020-11-20T12:06:55.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/15' 2020-11-20T12:06:55.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/31' 2020-11-20T12:06:55.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/36' 2020-11-20T12:06:55.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/20' 2020-11-20T12:06:55.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/12' 2020-11-20T12:06:55.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/39' 2020-11-20T12:06:55.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/10' 2020-11-20T12:06:55.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/34' 2020-11-20T12:06:55.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/38' 2020-11-20T12:06:55.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/11' 2020-11-20T12:06:55.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/16' 2020-11-20T12:06:55.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/32' 2020-11-20T12:06:55.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/30' 2020-11-20T12:06:55.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/14' 2020-11-20T12:06:55.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/18' 2020-11-20T12:06:55.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/13' 2020-11-20T12:06:55.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/33' 2020-11-20T12:06:55.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/17' 2020-11-20T12:06:55.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/35' 2020-11-20T12:06:55.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './110/37' 2020-11-20T12:06:55.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './110' 2020-11-20T12:06:55.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/19' 2020-11-20T12:06:55.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/15' 2020-11-20T12:06:55.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/31' 2020-11-20T12:06:55.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/36' 2020-11-20T12:06:55.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/20' 2020-11-20T12:06:55.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/12' 2020-11-20T12:06:55.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/39' 2020-11-20T12:06:55.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/10' 2020-11-20T12:06:55.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/34' 2020-11-20T12:06:55.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/38' 2020-11-20T12:06:55.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/11' 2020-11-20T12:06:55.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/16' 2020-11-20T12:06:55.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/32' 2020-11-20T12:06:55.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/30' 2020-11-20T12:06:55.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/14' 2020-11-20T12:06:55.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/18' 2020-11-20T12:06:55.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/13' 2020-11-20T12:06:55.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/33' 2020-11-20T12:06:55.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/17' 2020-11-20T12:06:55.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/35' 2020-11-20T12:06:55.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './111/37' 2020-11-20T12:06:55.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './111' 2020-11-20T12:06:55.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/19' 2020-11-20T12:06:55.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/15' 2020-11-20T12:06:55.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/31' 2020-11-20T12:06:55.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/36' 2020-11-20T12:06:55.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/20' 2020-11-20T12:06:55.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/12' 2020-11-20T12:06:55.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/39' 2020-11-20T12:06:55.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/10' 2020-11-20T12:06:55.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/34' 2020-11-20T12:06:55.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/38' 2020-11-20T12:06:55.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/11' 2020-11-20T12:06:55.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/16' 2020-11-20T12:06:55.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/32' 2020-11-20T12:06:55.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/30' 2020-11-20T12:06:55.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/14' 2020-11-20T12:06:55.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/18' 2020-11-20T12:06:55.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/13' 2020-11-20T12:06:55.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/33' 2020-11-20T12:06:55.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/17' 2020-11-20T12:06:55.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/35' 2020-11-20T12:06:55.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './112/37' 2020-11-20T12:06:55.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './112' 2020-11-20T12:06:55.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/19' 2020-11-20T12:06:55.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/15' 2020-11-20T12:06:55.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/31' 2020-11-20T12:06:55.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/36' 2020-11-20T12:06:55.463 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/20' 2020-11-20T12:06:55.463 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/12' 2020-11-20T12:06:55.463 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/39' 2020-11-20T12:06:55.463 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/10' 2020-11-20T12:06:55.463 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/34' 2020-11-20T12:06:55.464 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/38' 2020-11-20T12:06:55.464 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/11' 2020-11-20T12:06:55.464 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/16' 2020-11-20T12:06:55.464 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/32' 2020-11-20T12:06:55.464 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/30' 2020-11-20T12:06:55.465 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/14' 2020-11-20T12:06:55.465 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/18' 2020-11-20T12:06:55.465 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/13' 2020-11-20T12:06:55.465 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/33' 2020-11-20T12:06:55.465 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/17' 2020-11-20T12:06:55.466 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/35' 2020-11-20T12:06:55.466 INFO:tasks.workunit.client.0.smithi044.stdout:removed './113/37' 2020-11-20T12:06:55.466 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './113' 2020-11-20T12:06:55.466 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/19' 2020-11-20T12:06:55.466 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/15' 2020-11-20T12:06:55.467 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/31' 2020-11-20T12:06:55.467 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/36' 2020-11-20T12:06:55.467 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/20' 2020-11-20T12:06:55.467 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/12' 2020-11-20T12:06:55.468 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/39' 2020-11-20T12:06:55.468 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/10' 2020-11-20T12:06:55.468 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/34' 2020-11-20T12:06:55.468 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/38' 2020-11-20T12:06:55.468 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/11' 2020-11-20T12:06:55.468 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/16' 2020-11-20T12:06:55.469 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/32' 2020-11-20T12:06:55.469 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/30' 2020-11-20T12:06:55.469 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/14' 2020-11-20T12:06:55.469 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/18' 2020-11-20T12:06:55.470 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/13' 2020-11-20T12:06:55.470 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/33' 2020-11-20T12:06:55.470 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/17' 2020-11-20T12:06:55.470 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/35' 2020-11-20T12:06:55.470 INFO:tasks.workunit.client.0.smithi044.stdout:removed './114/37' 2020-11-20T12:06:55.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './114' 2020-11-20T12:06:55.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/19' 2020-11-20T12:06:55.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/15' 2020-11-20T12:06:55.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/31' 2020-11-20T12:06:55.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/36' 2020-11-20T12:06:55.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/20' 2020-11-20T12:06:55.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/12' 2020-11-20T12:06:55.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/39' 2020-11-20T12:06:55.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/10' 2020-11-20T12:06:55.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/34' 2020-11-20T12:06:55.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/38' 2020-11-20T12:06:55.473 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/11' 2020-11-20T12:06:55.473 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/16' 2020-11-20T12:06:55.473 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/32' 2020-11-20T12:06:55.473 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/30' 2020-11-20T12:06:55.474 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/14' 2020-11-20T12:06:55.474 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/18' 2020-11-20T12:06:55.474 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/13' 2020-11-20T12:06:55.474 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/33' 2020-11-20T12:06:55.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/17' 2020-11-20T12:06:55.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/35' 2020-11-20T12:06:55.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './115/37' 2020-11-20T12:06:55.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './115' 2020-11-20T12:06:55.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/19' 2020-11-20T12:06:55.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/15' 2020-11-20T12:06:55.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/31' 2020-11-20T12:06:55.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/36' 2020-11-20T12:06:55.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/20' 2020-11-20T12:06:55.477 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/12' 2020-11-20T12:06:55.477 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/39' 2020-11-20T12:06:55.477 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/10' 2020-11-20T12:06:55.477 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/34' 2020-11-20T12:06:55.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/38' 2020-11-20T12:06:55.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/11' 2020-11-20T12:06:55.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/16' 2020-11-20T12:06:55.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/32' 2020-11-20T12:06:55.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/30' 2020-11-20T12:06:55.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/14' 2020-11-20T12:06:55.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/18' 2020-11-20T12:06:55.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/13' 2020-11-20T12:06:55.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/33' 2020-11-20T12:06:55.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/17' 2020-11-20T12:06:55.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/35' 2020-11-20T12:06:55.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './116/37' 2020-11-20T12:06:55.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './116' 2020-11-20T12:06:55.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/19' 2020-11-20T12:06:55.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/15' 2020-11-20T12:06:55.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/31' 2020-11-20T12:06:55.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/36' 2020-11-20T12:06:55.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/20' 2020-11-20T12:06:55.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/12' 2020-11-20T12:06:55.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/39' 2020-11-20T12:06:55.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/10' 2020-11-20T12:06:55.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/34' 2020-11-20T12:06:55.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/38' 2020-11-20T12:06:55.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/11' 2020-11-20T12:06:55.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/16' 2020-11-20T12:06:55.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/32' 2020-11-20T12:06:55.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/30' 2020-11-20T12:06:55.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/14' 2020-11-20T12:06:55.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/18' 2020-11-20T12:06:55.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/13' 2020-11-20T12:06:55.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/33' 2020-11-20T12:06:55.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/17' 2020-11-20T12:06:55.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/35' 2020-11-20T12:06:55.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './117/37' 2020-11-20T12:06:55.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './117' 2020-11-20T12:06:55.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/19' 2020-11-20T12:06:55.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/15' 2020-11-20T12:06:55.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/31' 2020-11-20T12:06:55.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/36' 2020-11-20T12:06:55.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/20' 2020-11-20T12:06:55.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/12' 2020-11-20T12:06:55.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/39' 2020-11-20T12:06:55.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/10' 2020-11-20T12:06:55.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/34' 2020-11-20T12:06:55.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/38' 2020-11-20T12:06:55.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/11' 2020-11-20T12:06:55.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/16' 2020-11-20T12:06:55.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/32' 2020-11-20T12:06:55.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/30' 2020-11-20T12:06:55.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/14' 2020-11-20T12:06:55.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/18' 2020-11-20T12:06:55.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/13' 2020-11-20T12:06:55.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/33' 2020-11-20T12:06:55.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/17' 2020-11-20T12:06:55.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/35' 2020-11-20T12:06:55.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './118/37' 2020-11-20T12:06:55.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './118' 2020-11-20T12:06:55.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/19' 2020-11-20T12:06:55.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/15' 2020-11-20T12:06:55.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/31' 2020-11-20T12:06:55.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/36' 2020-11-20T12:06:55.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/20' 2020-11-20T12:06:55.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/12' 2020-11-20T12:06:56.205 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/39' 2020-11-20T12:06:56.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/10' 2020-11-20T12:06:56.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/34' 2020-11-20T12:06:56.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/38' 2020-11-20T12:06:56.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/11' 2020-11-20T12:06:56.207 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/16' 2020-11-20T12:06:56.207 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/32' 2020-11-20T12:06:56.207 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/30' 2020-11-20T12:06:56.207 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/14' 2020-11-20T12:06:56.208 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/18' 2020-11-20T12:06:56.208 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/13' 2020-11-20T12:06:56.208 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/33' 2020-11-20T12:06:56.208 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/17' 2020-11-20T12:06:56.208 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/35' 2020-11-20T12:06:56.209 INFO:tasks.workunit.client.0.smithi044.stdout:removed './119/37' 2020-11-20T12:06:56.209 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './119' 2020-11-20T12:06:56.209 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/19' 2020-11-20T12:06:56.209 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/15' 2020-11-20T12:06:56.210 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/31' 2020-11-20T12:06:56.210 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/36' 2020-11-20T12:06:56.210 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/20' 2020-11-20T12:06:56.210 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/12' 2020-11-20T12:06:56.211 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/39' 2020-11-20T12:06:56.211 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/10' 2020-11-20T12:06:56.211 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/34' 2020-11-20T12:06:56.211 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/38' 2020-11-20T12:06:56.211 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/11' 2020-11-20T12:06:56.212 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/16' 2020-11-20T12:06:56.212 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/32' 2020-11-20T12:06:56.212 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/30' 2020-11-20T12:06:56.212 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/14' 2020-11-20T12:06:56.212 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/18' 2020-11-20T12:06:56.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/13' 2020-11-20T12:06:56.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/33' 2020-11-20T12:06:56.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/17' 2020-11-20T12:06:56.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/35' 2020-11-20T12:06:56.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './120/37' 2020-11-20T12:06:56.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './120' 2020-11-20T12:06:56.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/19' 2020-11-20T12:06:56.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/15' 2020-11-20T12:06:56.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/31' 2020-11-20T12:06:56.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/36' 2020-11-20T12:06:56.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/20' 2020-11-20T12:06:56.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/12' 2020-11-20T12:06:56.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/39' 2020-11-20T12:06:56.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/10' 2020-11-20T12:06:56.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/34' 2020-11-20T12:06:56.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/38' 2020-11-20T12:06:56.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/11' 2020-11-20T12:06:56.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/16' 2020-11-20T12:06:56.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/32' 2020-11-20T12:06:56.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/30' 2020-11-20T12:06:56.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/14' 2020-11-20T12:06:56.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/18' 2020-11-20T12:06:56.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/13' 2020-11-20T12:06:56.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/33' 2020-11-20T12:06:56.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/17' 2020-11-20T12:06:56.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/35' 2020-11-20T12:06:56.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './121/37' 2020-11-20T12:06:56.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './121' 2020-11-20T12:06:56.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/19' 2020-11-20T12:06:56.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/15' 2020-11-20T12:06:56.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/31' 2020-11-20T12:06:56.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/36' 2020-11-20T12:06:56.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/20' 2020-11-20T12:06:56.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/12' 2020-11-20T12:06:56.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/39' 2020-11-20T12:06:56.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/10' 2020-11-20T12:06:56.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/34' 2020-11-20T12:06:56.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/38' 2020-11-20T12:06:56.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/11' 2020-11-20T12:06:56.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/16' 2020-11-20T12:06:56.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/32' 2020-11-20T12:06:56.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/30' 2020-11-20T12:06:56.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/14' 2020-11-20T12:06:56.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/18' 2020-11-20T12:06:56.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/13' 2020-11-20T12:06:56.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/33' 2020-11-20T12:06:56.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/17' 2020-11-20T12:06:56.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/35' 2020-11-20T12:06:56.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './122/37' 2020-11-20T12:06:56.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './122' 2020-11-20T12:06:56.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/19' 2020-11-20T12:06:56.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/15' 2020-11-20T12:06:56.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/31' 2020-11-20T12:06:56.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/36' 2020-11-20T12:06:56.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/20' 2020-11-20T12:06:56.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/12' 2020-11-20T12:06:56.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/39' 2020-11-20T12:06:56.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/10' 2020-11-20T12:06:56.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/34' 2020-11-20T12:06:56.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/38' 2020-11-20T12:06:56.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/11' 2020-11-20T12:06:56.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/16' 2020-11-20T12:06:56.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/32' 2020-11-20T12:06:56.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/30' 2020-11-20T12:06:56.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/14' 2020-11-20T12:06:56.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/18' 2020-11-20T12:06:56.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/13' 2020-11-20T12:06:56.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/33' 2020-11-20T12:06:56.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/17' 2020-11-20T12:06:56.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/35' 2020-11-20T12:06:56.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './123/37' 2020-11-20T12:06:56.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './123' 2020-11-20T12:06:56.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/19' 2020-11-20T12:06:56.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/15' 2020-11-20T12:06:56.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/31' 2020-11-20T12:06:56.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/36' 2020-11-20T12:06:56.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/20' 2020-11-20T12:06:56.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/12' 2020-11-20T12:06:56.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/39' 2020-11-20T12:06:56.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/10' 2020-11-20T12:06:56.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/34' 2020-11-20T12:06:56.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/38' 2020-11-20T12:06:56.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/11' 2020-11-20T12:06:56.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/16' 2020-11-20T12:06:56.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/32' 2020-11-20T12:06:56.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/30' 2020-11-20T12:06:56.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/14' 2020-11-20T12:06:56.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/18' 2020-11-20T12:06:56.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/13' 2020-11-20T12:06:56.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/33' 2020-11-20T12:06:56.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/17' 2020-11-20T12:06:56.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/35' 2020-11-20T12:06:56.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed './124/37' 2020-11-20T12:06:56.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './124' 2020-11-20T12:06:56.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/19' 2020-11-20T12:06:56.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/15' 2020-11-20T12:06:56.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/31' 2020-11-20T12:06:56.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/36' 2020-11-20T12:06:56.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/20' 2020-11-20T12:06:56.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/12' 2020-11-20T12:06:56.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/39' 2020-11-20T12:06:56.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/10' 2020-11-20T12:06:56.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/34' 2020-11-20T12:06:56.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/38' 2020-11-20T12:06:56.235 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/11' 2020-11-20T12:06:56.235 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/16' 2020-11-20T12:06:56.235 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/32' 2020-11-20T12:06:56.235 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/30' 2020-11-20T12:06:56.235 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/14' 2020-11-20T12:06:56.236 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/18' 2020-11-20T12:06:56.236 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/13' 2020-11-20T12:06:56.236 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/33' 2020-11-20T12:06:56.236 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/17' 2020-11-20T12:06:56.236 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/35' 2020-11-20T12:06:56.237 INFO:tasks.workunit.client.0.smithi044.stdout:removed './125/37' 2020-11-20T12:06:56.237 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './125' 2020-11-20T12:06:56.237 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/19' 2020-11-20T12:06:56.237 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/15' 2020-11-20T12:06:56.237 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/31' 2020-11-20T12:06:56.238 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/36' 2020-11-20T12:06:56.238 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/20' 2020-11-20T12:06:56.238 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/12' 2020-11-20T12:06:56.238 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/39' 2020-11-20T12:06:56.238 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/10' 2020-11-20T12:06:56.239 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/34' 2020-11-20T12:06:56.239 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/38' 2020-11-20T12:06:56.239 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/11' 2020-11-20T12:06:56.239 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/16' 2020-11-20T12:06:56.239 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/32' 2020-11-20T12:06:56.240 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/30' 2020-11-20T12:06:56.240 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/14' 2020-11-20T12:06:56.240 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/18' 2020-11-20T12:06:56.240 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/13' 2020-11-20T12:06:56.241 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/33' 2020-11-20T12:06:56.241 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/17' 2020-11-20T12:06:56.241 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/35' 2020-11-20T12:06:56.241 INFO:tasks.workunit.client.0.smithi044.stdout:removed './126/37' 2020-11-20T12:06:56.242 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './126' 2020-11-20T12:06:56.242 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/19' 2020-11-20T12:06:56.242 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/15' 2020-11-20T12:06:56.242 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/31' 2020-11-20T12:06:56.242 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/36' 2020-11-20T12:06:56.243 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/20' 2020-11-20T12:06:56.243 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/12' 2020-11-20T12:06:56.243 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/39' 2020-11-20T12:06:56.243 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/10' 2020-11-20T12:06:56.243 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/34' 2020-11-20T12:06:56.244 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/38' 2020-11-20T12:06:56.244 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/11' 2020-11-20T12:06:56.244 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/16' 2020-11-20T12:06:56.244 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/32' 2020-11-20T12:06:56.244 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/30' 2020-11-20T12:06:56.245 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/14' 2020-11-20T12:06:56.245 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/18' 2020-11-20T12:06:56.245 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/13' 2020-11-20T12:06:56.245 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/33' 2020-11-20T12:06:56.245 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/17' 2020-11-20T12:06:56.246 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/35' 2020-11-20T12:06:56.246 INFO:tasks.workunit.client.0.smithi044.stdout:removed './127/37' 2020-11-20T12:06:56.246 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './127' 2020-11-20T12:06:56.246 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/19' 2020-11-20T12:06:56.246 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/15' 2020-11-20T12:06:56.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/31' 2020-11-20T12:06:56.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/36' 2020-11-20T12:06:56.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/20' 2020-11-20T12:06:56.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/12' 2020-11-20T12:06:56.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/39' 2020-11-20T12:06:56.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/10' 2020-11-20T12:06:56.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/34' 2020-11-20T12:06:56.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/38' 2020-11-20T12:06:56.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/11' 2020-11-20T12:06:56.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/16' 2020-11-20T12:06:56.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/32' 2020-11-20T12:06:56.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/30' 2020-11-20T12:06:56.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/14' 2020-11-20T12:06:56.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/18' 2020-11-20T12:06:56.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/13' 2020-11-20T12:06:56.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/33' 2020-11-20T12:06:56.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/17' 2020-11-20T12:06:56.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/35' 2020-11-20T12:06:56.775 INFO:tasks.workunit.client.0.smithi044.stdout:removed './128/37' 2020-11-20T12:06:56.775 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './128' 2020-11-20T12:06:56.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/19' 2020-11-20T12:06:56.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/15' 2020-11-20T12:06:56.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/31' 2020-11-20T12:06:56.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/36' 2020-11-20T12:06:56.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/20' 2020-11-20T12:06:56.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/12' 2020-11-20T12:06:56.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/39' 2020-11-20T12:06:56.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/10' 2020-11-20T12:06:56.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/34' 2020-11-20T12:06:56.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/38' 2020-11-20T12:06:56.778 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/11' 2020-11-20T12:06:56.778 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/16' 2020-11-20T12:06:56.779 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/32' 2020-11-20T12:06:56.779 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/30' 2020-11-20T12:06:56.779 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/14' 2020-11-20T12:06:56.780 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/18' 2020-11-20T12:06:56.780 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/13' 2020-11-20T12:06:56.781 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/33' 2020-11-20T12:06:56.781 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/17' 2020-11-20T12:06:56.781 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/35' 2020-11-20T12:06:56.782 INFO:tasks.workunit.client.0.smithi044.stdout:removed './129/37' 2020-11-20T12:06:56.782 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './129' 2020-11-20T12:06:56.782 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/19' 2020-11-20T12:06:56.782 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/15' 2020-11-20T12:06:56.783 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/31' 2020-11-20T12:06:56.783 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/36' 2020-11-20T12:06:56.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/20' 2020-11-20T12:06:56.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/12' 2020-11-20T12:06:56.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/39' 2020-11-20T12:06:56.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/10' 2020-11-20T12:06:56.785 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/34' 2020-11-20T12:06:56.785 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/38' 2020-11-20T12:06:56.785 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/11' 2020-11-20T12:06:56.785 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/16' 2020-11-20T12:06:56.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/32' 2020-11-20T12:06:56.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/30' 2020-11-20T12:06:56.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/14' 2020-11-20T12:06:56.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/18' 2020-11-20T12:06:56.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/13' 2020-11-20T12:06:56.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/33' 2020-11-20T12:06:56.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/17' 2020-11-20T12:06:56.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/35' 2020-11-20T12:06:56.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './130/37' 2020-11-20T12:06:56.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './130' 2020-11-20T12:06:56.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/19' 2020-11-20T12:06:56.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/15' 2020-11-20T12:06:56.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/31' 2020-11-20T12:06:56.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/36' 2020-11-20T12:06:56.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/20' 2020-11-20T12:06:56.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/12' 2020-11-20T12:06:56.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/39' 2020-11-20T12:06:56.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/10' 2020-11-20T12:06:56.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/34' 2020-11-20T12:06:56.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/38' 2020-11-20T12:06:56.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/11' 2020-11-20T12:06:56.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/16' 2020-11-20T12:06:56.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/32' 2020-11-20T12:06:56.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/30' 2020-11-20T12:06:56.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/14' 2020-11-20T12:06:56.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/18' 2020-11-20T12:06:56.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/13' 2020-11-20T12:06:56.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/33' 2020-11-20T12:06:56.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/17' 2020-11-20T12:06:56.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/35' 2020-11-20T12:06:56.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './131/37' 2020-11-20T12:06:56.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './131' 2020-11-20T12:06:56.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/19' 2020-11-20T12:06:56.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/15' 2020-11-20T12:06:56.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/31' 2020-11-20T12:06:56.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/36' 2020-11-20T12:06:56.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/20' 2020-11-20T12:06:56.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/12' 2020-11-20T12:06:56.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/39' 2020-11-20T12:06:56.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/10' 2020-11-20T12:06:56.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/34' 2020-11-20T12:06:56.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/38' 2020-11-20T12:06:56.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/11' 2020-11-20T12:06:56.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/16' 2020-11-20T12:06:56.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/32' 2020-11-20T12:06:56.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/30' 2020-11-20T12:06:56.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/14' 2020-11-20T12:06:56.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/18' 2020-11-20T12:06:56.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/13' 2020-11-20T12:06:56.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/33' 2020-11-20T12:06:56.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/17' 2020-11-20T12:06:56.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/35' 2020-11-20T12:06:56.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './132/37' 2020-11-20T12:06:56.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './132' 2020-11-20T12:06:56.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/19' 2020-11-20T12:06:56.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/15' 2020-11-20T12:06:56.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/31' 2020-11-20T12:06:56.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/36' 2020-11-20T12:06:56.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/20' 2020-11-20T12:06:56.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/12' 2020-11-20T12:06:56.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/39' 2020-11-20T12:06:56.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/10' 2020-11-20T12:06:56.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/34' 2020-11-20T12:06:56.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/38' 2020-11-20T12:06:56.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/11' 2020-11-20T12:06:56.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/16' 2020-11-20T12:06:56.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/32' 2020-11-20T12:06:56.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/30' 2020-11-20T12:06:56.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/14' 2020-11-20T12:06:56.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/18' 2020-11-20T12:06:56.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/13' 2020-11-20T12:06:56.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/33' 2020-11-20T12:06:56.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/17' 2020-11-20T12:06:56.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/35' 2020-11-20T12:06:56.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './133/37' 2020-11-20T12:06:56.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './133' 2020-11-20T12:06:56.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/19' 2020-11-20T12:06:56.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/15' 2020-11-20T12:06:56.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/31' 2020-11-20T12:06:56.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/36' 2020-11-20T12:06:56.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/20' 2020-11-20T12:06:56.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/12' 2020-11-20T12:06:56.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/39' 2020-11-20T12:06:56.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/10' 2020-11-20T12:06:56.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/34' 2020-11-20T12:06:56.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/38' 2020-11-20T12:06:56.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/11' 2020-11-20T12:06:56.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/16' 2020-11-20T12:06:56.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/32' 2020-11-20T12:06:56.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/30' 2020-11-20T12:06:56.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/14' 2020-11-20T12:06:56.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/18' 2020-11-20T12:06:56.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/13' 2020-11-20T12:06:56.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/33' 2020-11-20T12:06:56.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/17' 2020-11-20T12:06:56.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/35' 2020-11-20T12:06:56.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './134/37' 2020-11-20T12:06:56.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './134' 2020-11-20T12:06:56.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/19' 2020-11-20T12:06:56.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/15' 2020-11-20T12:06:56.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/31' 2020-11-20T12:06:56.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/36' 2020-11-20T12:06:56.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/20' 2020-11-20T12:06:56.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/12' 2020-11-20T12:06:56.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/39' 2020-11-20T12:06:56.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/10' 2020-11-20T12:06:56.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/34' 2020-11-20T12:06:56.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/38' 2020-11-20T12:06:56.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/11' 2020-11-20T12:06:56.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/16' 2020-11-20T12:06:56.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/32' 2020-11-20T12:06:56.809 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/30' 2020-11-20T12:06:56.809 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/14' 2020-11-20T12:06:56.809 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/18' 2020-11-20T12:06:56.809 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/13' 2020-11-20T12:06:56.809 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/33' 2020-11-20T12:06:56.810 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/17' 2020-11-20T12:06:56.810 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/35' 2020-11-20T12:06:56.810 INFO:tasks.workunit.client.0.smithi044.stdout:removed './135/37' 2020-11-20T12:06:56.810 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './135' 2020-11-20T12:06:56.810 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/19' 2020-11-20T12:06:56.811 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/15' 2020-11-20T12:06:56.811 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/31' 2020-11-20T12:06:56.811 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/36' 2020-11-20T12:06:56.811 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/20' 2020-11-20T12:06:56.811 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/12' 2020-11-20T12:06:56.812 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/39' 2020-11-20T12:06:56.812 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/10' 2020-11-20T12:06:56.812 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/34' 2020-11-20T12:06:56.812 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/38' 2020-11-20T12:06:56.812 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/11' 2020-11-20T12:06:56.813 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/16' 2020-11-20T12:06:56.813 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/32' 2020-11-20T12:06:56.813 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/30' 2020-11-20T12:06:56.813 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/14' 2020-11-20T12:06:56.813 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/18' 2020-11-20T12:06:56.814 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/13' 2020-11-20T12:06:56.814 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/33' 2020-11-20T12:06:56.814 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/17' 2020-11-20T12:06:56.814 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/35' 2020-11-20T12:06:56.814 INFO:tasks.workunit.client.0.smithi044.stdout:removed './136/37' 2020-11-20T12:06:56.815 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './136' 2020-11-20T12:06:56.815 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/19' 2020-11-20T12:06:56.815 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/15' 2020-11-20T12:06:56.815 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/31' 2020-11-20T12:06:56.815 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/36' 2020-11-20T12:06:56.816 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/20' 2020-11-20T12:06:56.816 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/12' 2020-11-20T12:06:56.816 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/39' 2020-11-20T12:06:56.816 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/10' 2020-11-20T12:06:56.816 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/34' 2020-11-20T12:06:56.817 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/38' 2020-11-20T12:06:56.817 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/11' 2020-11-20T12:06:56.817 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/16' 2020-11-20T12:06:56.817 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/32' 2020-11-20T12:06:56.817 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/30' 2020-11-20T12:06:56.818 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/14' 2020-11-20T12:06:56.818 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/18' 2020-11-20T12:06:56.818 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/13' 2020-11-20T12:06:56.818 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/33' 2020-11-20T12:06:56.818 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/17' 2020-11-20T12:06:56.819 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/35' 2020-11-20T12:06:56.819 INFO:tasks.workunit.client.0.smithi044.stdout:removed './137/37' 2020-11-20T12:06:56.819 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './137' 2020-11-20T12:06:56.819 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/19' 2020-11-20T12:06:56.819 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/15' 2020-11-20T12:06:56.820 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/31' 2020-11-20T12:06:56.820 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/36' 2020-11-20T12:06:56.820 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/20' 2020-11-20T12:06:56.820 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/12' 2020-11-20T12:06:56.820 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/39' 2020-11-20T12:06:56.821 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/10' 2020-11-20T12:06:56.821 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/34' 2020-11-20T12:06:56.821 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/38' 2020-11-20T12:06:56.821 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/11' 2020-11-20T12:06:56.821 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/16' 2020-11-20T12:06:57.418 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/32' 2020-11-20T12:06:57.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/30' 2020-11-20T12:06:57.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/14' 2020-11-20T12:06:57.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/18' 2020-11-20T12:06:57.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/13' 2020-11-20T12:06:57.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/33' 2020-11-20T12:06:57.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/17' 2020-11-20T12:06:57.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/35' 2020-11-20T12:06:57.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './138/37' 2020-11-20T12:06:57.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './138' 2020-11-20T12:06:57.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/19' 2020-11-20T12:06:57.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/15' 2020-11-20T12:06:57.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/31' 2020-11-20T12:06:57.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/36' 2020-11-20T12:06:57.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/20' 2020-11-20T12:06:57.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/12' 2020-11-20T12:06:57.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/39' 2020-11-20T12:06:57.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/10' 2020-11-20T12:06:57.423 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/34' 2020-11-20T12:06:57.423 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/38' 2020-11-20T12:06:57.423 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/11' 2020-11-20T12:06:57.423 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/16' 2020-11-20T12:06:57.424 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/32' 2020-11-20T12:06:57.424 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/30' 2020-11-20T12:06:57.424 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/14' 2020-11-20T12:06:57.424 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/18' 2020-11-20T12:06:57.424 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/13' 2020-11-20T12:06:57.425 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/33' 2020-11-20T12:06:57.425 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/17' 2020-11-20T12:06:57.425 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/35' 2020-11-20T12:06:57.425 INFO:tasks.workunit.client.0.smithi044.stdout:removed './139/37' 2020-11-20T12:06:57.426 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './139' 2020-11-20T12:06:57.426 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/19' 2020-11-20T12:06:57.426 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/15' 2020-11-20T12:06:57.426 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/31' 2020-11-20T12:06:57.426 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/36' 2020-11-20T12:06:57.427 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/20' 2020-11-20T12:06:57.427 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/12' 2020-11-20T12:06:57.427 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/39' 2020-11-20T12:06:57.427 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/10' 2020-11-20T12:06:57.428 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/34' 2020-11-20T12:06:57.428 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/38' 2020-11-20T12:06:57.428 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/11' 2020-11-20T12:06:57.428 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/16' 2020-11-20T12:06:57.428 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/32' 2020-11-20T12:06:57.429 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/30' 2020-11-20T12:06:57.429 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/14' 2020-11-20T12:06:57.429 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/18' 2020-11-20T12:06:57.429 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/13' 2020-11-20T12:06:57.429 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/33' 2020-11-20T12:06:57.430 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/17' 2020-11-20T12:06:57.430 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/35' 2020-11-20T12:06:57.430 INFO:tasks.workunit.client.0.smithi044.stdout:removed './140/37' 2020-11-20T12:06:57.430 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './140' 2020-11-20T12:06:57.430 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/19' 2020-11-20T12:06:57.431 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/15' 2020-11-20T12:06:57.431 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/31' 2020-11-20T12:06:57.431 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/36' 2020-11-20T12:06:57.431 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/20' 2020-11-20T12:06:57.431 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/12' 2020-11-20T12:06:57.432 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/39' 2020-11-20T12:06:57.432 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/10' 2020-11-20T12:06:57.432 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/34' 2020-11-20T12:06:57.432 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/38' 2020-11-20T12:06:57.432 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/11' 2020-11-20T12:06:57.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/16' 2020-11-20T12:06:57.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/32' 2020-11-20T12:06:57.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/30' 2020-11-20T12:06:57.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/14' 2020-11-20T12:06:57.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/18' 2020-11-20T12:06:57.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/13' 2020-11-20T12:06:57.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/33' 2020-11-20T12:06:57.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/17' 2020-11-20T12:06:57.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/35' 2020-11-20T12:06:57.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './141/37' 2020-11-20T12:06:57.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './141' 2020-11-20T12:06:57.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/19' 2020-11-20T12:06:57.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/15' 2020-11-20T12:06:57.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/31' 2020-11-20T12:06:57.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/36' 2020-11-20T12:06:57.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/20' 2020-11-20T12:06:57.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/12' 2020-11-20T12:06:57.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/39' 2020-11-20T12:06:57.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/10' 2020-11-20T12:06:57.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/34' 2020-11-20T12:06:57.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/38' 2020-11-20T12:06:57.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/11' 2020-11-20T12:06:57.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/16' 2020-11-20T12:06:57.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/32' 2020-11-20T12:06:57.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/30' 2020-11-20T12:06:57.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/14' 2020-11-20T12:06:57.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/18' 2020-11-20T12:06:57.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/13' 2020-11-20T12:06:57.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/33' 2020-11-20T12:06:57.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/17' 2020-11-20T12:06:57.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/35' 2020-11-20T12:06:57.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './142/37' 2020-11-20T12:06:57.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './142' 2020-11-20T12:06:57.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/19' 2020-11-20T12:06:57.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/15' 2020-11-20T12:06:57.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/31' 2020-11-20T12:06:57.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/36' 2020-11-20T12:06:57.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/20' 2020-11-20T12:06:57.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/12' 2020-11-20T12:06:57.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/39' 2020-11-20T12:06:57.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/10' 2020-11-20T12:06:57.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/34' 2020-11-20T12:06:57.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/38' 2020-11-20T12:06:57.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/11' 2020-11-20T12:06:57.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/16' 2020-11-20T12:06:57.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/32' 2020-11-20T12:06:57.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/30' 2020-11-20T12:06:57.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/14' 2020-11-20T12:06:57.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/18' 2020-11-20T12:06:57.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/13' 2020-11-20T12:06:57.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/33' 2020-11-20T12:06:57.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/17' 2020-11-20T12:06:57.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/35' 2020-11-20T12:06:57.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './143/37' 2020-11-20T12:06:57.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './143' 2020-11-20T12:06:57.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/19' 2020-11-20T12:06:57.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/15' 2020-11-20T12:06:57.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/31' 2020-11-20T12:06:57.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/36' 2020-11-20T12:06:57.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/20' 2020-11-20T12:06:57.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/12' 2020-11-20T12:06:57.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/39' 2020-11-20T12:06:57.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/10' 2020-11-20T12:06:57.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/34' 2020-11-20T12:06:57.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/38' 2020-11-20T12:06:57.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/11' 2020-11-20T12:06:57.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/16' 2020-11-20T12:06:57.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/32' 2020-11-20T12:06:57.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/30' 2020-11-20T12:06:57.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/14' 2020-11-20T12:06:57.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/18' 2020-11-20T12:06:57.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/13' 2020-11-20T12:06:57.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/33' 2020-11-20T12:06:57.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/17' 2020-11-20T12:06:57.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/35' 2020-11-20T12:06:57.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './144/37' 2020-11-20T12:06:57.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './144' 2020-11-20T12:06:57.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/19' 2020-11-20T12:06:57.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/15' 2020-11-20T12:06:57.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/31' 2020-11-20T12:06:57.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/36' 2020-11-20T12:06:57.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/20' 2020-11-20T12:06:57.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/12' 2020-11-20T12:06:57.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/39' 2020-11-20T12:06:57.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/10' 2020-11-20T12:06:57.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/34' 2020-11-20T12:06:57.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/38' 2020-11-20T12:06:57.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/11' 2020-11-20T12:06:57.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/16' 2020-11-20T12:06:57.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/32' 2020-11-20T12:06:57.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/30' 2020-11-20T12:06:57.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/14' 2020-11-20T12:06:57.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/18' 2020-11-20T12:06:57.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/13' 2020-11-20T12:06:57.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/33' 2020-11-20T12:06:57.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/17' 2020-11-20T12:06:57.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/35' 2020-11-20T12:06:57.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './145/37' 2020-11-20T12:06:57.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './145' 2020-11-20T12:06:57.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/19' 2020-11-20T12:06:57.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/15' 2020-11-20T12:06:57.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/31' 2020-11-20T12:06:57.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/36' 2020-11-20T12:06:57.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/20' 2020-11-20T12:06:57.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/12' 2020-11-20T12:06:57.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/39' 2020-11-20T12:06:57.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/10' 2020-11-20T12:06:57.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/34' 2020-11-20T12:06:57.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/38' 2020-11-20T12:06:57.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/11' 2020-11-20T12:06:57.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/16' 2020-11-20T12:06:57.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/32' 2020-11-20T12:06:57.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/30' 2020-11-20T12:06:57.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/14' 2020-11-20T12:06:57.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/18' 2020-11-20T12:06:57.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/13' 2020-11-20T12:06:57.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/33' 2020-11-20T12:06:57.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/17' 2020-11-20T12:06:57.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/35' 2020-11-20T12:06:57.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './146/37' 2020-11-20T12:06:57.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './146' 2020-11-20T12:06:57.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/19' 2020-11-20T12:06:57.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/15' 2020-11-20T12:06:57.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/31' 2020-11-20T12:06:57.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/36' 2020-11-20T12:06:57.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/20' 2020-11-20T12:06:57.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/12' 2020-11-20T12:06:57.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/39' 2020-11-20T12:06:57.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/10' 2020-11-20T12:06:57.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/34' 2020-11-20T12:06:57.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/38' 2020-11-20T12:06:57.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/11' 2020-11-20T12:06:57.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/16' 2020-11-20T12:06:57.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/32' 2020-11-20T12:06:57.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/30' 2020-11-20T12:06:57.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/14' 2020-11-20T12:06:57.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/18' 2020-11-20T12:06:57.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/13' 2020-11-20T12:06:57.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/33' 2020-11-20T12:06:57.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/17' 2020-11-20T12:06:57.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/35' 2020-11-20T12:06:57.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './147/37' 2020-11-20T12:06:57.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './147' 2020-11-20T12:06:57.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/19' 2020-11-20T12:06:57.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/15' 2020-11-20T12:06:57.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/31' 2020-11-20T12:06:57.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/36' 2020-11-20T12:06:58.133 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/20' 2020-11-20T12:06:58.133 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/12' 2020-11-20T12:06:58.133 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/39' 2020-11-20T12:06:58.134 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/10' 2020-11-20T12:06:58.134 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/34' 2020-11-20T12:06:58.134 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/38' 2020-11-20T12:06:58.134 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/11' 2020-11-20T12:06:58.135 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/16' 2020-11-20T12:06:58.135 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/32' 2020-11-20T12:06:58.135 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/30' 2020-11-20T12:06:58.136 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/14' 2020-11-20T12:06:58.136 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/18' 2020-11-20T12:06:58.137 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/13' 2020-11-20T12:06:58.137 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/33' 2020-11-20T12:06:58.137 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/17' 2020-11-20T12:06:58.138 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/35' 2020-11-20T12:06:58.138 INFO:tasks.workunit.client.0.smithi044.stdout:removed './148/37' 2020-11-20T12:06:58.139 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './148' 2020-11-20T12:06:58.139 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/19' 2020-11-20T12:06:58.140 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/15' 2020-11-20T12:06:58.140 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/31' 2020-11-20T12:06:58.140 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/36' 2020-11-20T12:06:58.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/20' 2020-11-20T12:06:58.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/12' 2020-11-20T12:06:58.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/39' 2020-11-20T12:06:58.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/10' 2020-11-20T12:06:58.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/34' 2020-11-20T12:06:58.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/38' 2020-11-20T12:06:58.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/11' 2020-11-20T12:06:58.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/16' 2020-11-20T12:06:58.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/32' 2020-11-20T12:06:58.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/30' 2020-11-20T12:06:58.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/14' 2020-11-20T12:06:58.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/18' 2020-11-20T12:06:58.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/13' 2020-11-20T12:06:58.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/33' 2020-11-20T12:06:58.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/17' 2020-11-20T12:06:58.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/35' 2020-11-20T12:06:58.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './149/37' 2020-11-20T12:06:58.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './149' 2020-11-20T12:06:58.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/19' 2020-11-20T12:06:58.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/15' 2020-11-20T12:06:58.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/31' 2020-11-20T12:06:58.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/36' 2020-11-20T12:06:58.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/20' 2020-11-20T12:06:58.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/12' 2020-11-20T12:06:58.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/39' 2020-11-20T12:06:58.147 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/10' 2020-11-20T12:06:58.147 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/34' 2020-11-20T12:06:58.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/38' 2020-11-20T12:06:58.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/11' 2020-11-20T12:06:58.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/16' 2020-11-20T12:06:58.149 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/32' 2020-11-20T12:06:58.149 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/30' 2020-11-20T12:06:58.150 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/14' 2020-11-20T12:06:58.151 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/18' 2020-11-20T12:06:58.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/13' 2020-11-20T12:06:58.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/33' 2020-11-20T12:06:58.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/17' 2020-11-20T12:06:58.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/35' 2020-11-20T12:06:58.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './150/37' 2020-11-20T12:06:58.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './150' 2020-11-20T12:06:58.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/19' 2020-11-20T12:06:58.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/15' 2020-11-20T12:06:58.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/31' 2020-11-20T12:06:58.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/36' 2020-11-20T12:06:58.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/20' 2020-11-20T12:06:58.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/12' 2020-11-20T12:06:58.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/39' 2020-11-20T12:06:58.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/10' 2020-11-20T12:06:58.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/34' 2020-11-20T12:06:58.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/38' 2020-11-20T12:06:58.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/11' 2020-11-20T12:06:58.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/16' 2020-11-20T12:06:58.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/32' 2020-11-20T12:06:58.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/30' 2020-11-20T12:06:58.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/14' 2020-11-20T12:06:58.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/18' 2020-11-20T12:06:58.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/13' 2020-11-20T12:06:58.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/33' 2020-11-20T12:06:58.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/17' 2020-11-20T12:06:58.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/35' 2020-11-20T12:06:58.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './151/37' 2020-11-20T12:06:58.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './151' 2020-11-20T12:06:58.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/19' 2020-11-20T12:06:58.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/15' 2020-11-20T12:06:58.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/31' 2020-11-20T12:06:58.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/36' 2020-11-20T12:06:58.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/20' 2020-11-20T12:06:58.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/12' 2020-11-20T12:06:58.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/39' 2020-11-20T12:06:58.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/10' 2020-11-20T12:06:58.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/34' 2020-11-20T12:06:58.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/38' 2020-11-20T12:06:58.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/11' 2020-11-20T12:06:58.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/16' 2020-11-20T12:06:58.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/32' 2020-11-20T12:06:58.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/30' 2020-11-20T12:06:58.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/14' 2020-11-20T12:06:58.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/18' 2020-11-20T12:06:58.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/13' 2020-11-20T12:06:58.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/33' 2020-11-20T12:06:58.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/17' 2020-11-20T12:06:58.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/35' 2020-11-20T12:06:58.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './152/37' 2020-11-20T12:06:58.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './152' 2020-11-20T12:06:58.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/19' 2020-11-20T12:06:58.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/15' 2020-11-20T12:06:58.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/31' 2020-11-20T12:06:58.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/36' 2020-11-20T12:06:58.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/20' 2020-11-20T12:06:58.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/12' 2020-11-20T12:06:58.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/39' 2020-11-20T12:06:58.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/10' 2020-11-20T12:06:58.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/34' 2020-11-20T12:06:58.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/38' 2020-11-20T12:06:58.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/11' 2020-11-20T12:06:58.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/16' 2020-11-20T12:06:58.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/32' 2020-11-20T12:06:58.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/30' 2020-11-20T12:06:58.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/14' 2020-11-20T12:06:58.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/18' 2020-11-20T12:06:58.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/13' 2020-11-20T12:06:58.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/33' 2020-11-20T12:06:58.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/17' 2020-11-20T12:06:58.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/35' 2020-11-20T12:06:58.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './153/37' 2020-11-20T12:06:58.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './153' 2020-11-20T12:06:58.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/19' 2020-11-20T12:06:58.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/15' 2020-11-20T12:06:58.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/31' 2020-11-20T12:06:58.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/36' 2020-11-20T12:06:58.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/20' 2020-11-20T12:06:58.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/12' 2020-11-20T12:06:58.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/39' 2020-11-20T12:06:58.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/10' 2020-11-20T12:06:58.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/34' 2020-11-20T12:06:58.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/38' 2020-11-20T12:06:58.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/11' 2020-11-20T12:06:58.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/16' 2020-11-20T12:06:58.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/32' 2020-11-20T12:06:58.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/30' 2020-11-20T12:06:58.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/14' 2020-11-20T12:06:58.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/18' 2020-11-20T12:06:58.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/13' 2020-11-20T12:06:58.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/33' 2020-11-20T12:06:58.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/17' 2020-11-20T12:06:58.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/35' 2020-11-20T12:06:58.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './154/37' 2020-11-20T12:06:58.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './154' 2020-11-20T12:06:58.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/19' 2020-11-20T12:06:58.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/15' 2020-11-20T12:06:58.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/31' 2020-11-20T12:06:58.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/36' 2020-11-20T12:06:58.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/20' 2020-11-20T12:06:58.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/12' 2020-11-20T12:06:58.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/39' 2020-11-20T12:06:58.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/10' 2020-11-20T12:06:58.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/34' 2020-11-20T12:06:58.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/38' 2020-11-20T12:06:58.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/11' 2020-11-20T12:06:58.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/16' 2020-11-20T12:06:58.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/32' 2020-11-20T12:06:58.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/30' 2020-11-20T12:06:58.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/14' 2020-11-20T12:06:58.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/18' 2020-11-20T12:06:58.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/13' 2020-11-20T12:06:58.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/33' 2020-11-20T12:06:58.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/17' 2020-11-20T12:06:58.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/35' 2020-11-20T12:06:58.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './155/37' 2020-11-20T12:06:58.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './155' 2020-11-20T12:06:58.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/19' 2020-11-20T12:06:58.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/15' 2020-11-20T12:06:58.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/31' 2020-11-20T12:06:58.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/36' 2020-11-20T12:06:58.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/20' 2020-11-20T12:06:58.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/12' 2020-11-20T12:06:58.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/39' 2020-11-20T12:06:58.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/10' 2020-11-20T12:06:58.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/34' 2020-11-20T12:06:58.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/38' 2020-11-20T12:06:58.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/11' 2020-11-20T12:06:58.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/16' 2020-11-20T12:06:58.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/32' 2020-11-20T12:06:58.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/30' 2020-11-20T12:06:58.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/14' 2020-11-20T12:06:58.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/18' 2020-11-20T12:06:58.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/13' 2020-11-20T12:06:58.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/33' 2020-11-20T12:06:58.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/17' 2020-11-20T12:06:58.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/35' 2020-11-20T12:06:58.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './156/37' 2020-11-20T12:06:58.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './156' 2020-11-20T12:06:58.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/19' 2020-11-20T12:06:58.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/15' 2020-11-20T12:06:58.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/31' 2020-11-20T12:06:58.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/36' 2020-11-20T12:06:58.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/20' 2020-11-20T12:06:58.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/12' 2020-11-20T12:06:58.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/39' 2020-11-20T12:06:58.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/10' 2020-11-20T12:06:58.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/34' 2020-11-20T12:06:58.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/38' 2020-11-20T12:06:58.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/11' 2020-11-20T12:06:58.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/16' 2020-11-20T12:06:58.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/32' 2020-11-20T12:06:58.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/30' 2020-11-20T12:06:58.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/14' 2020-11-20T12:06:58.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/18' 2020-11-20T12:06:58.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/13' 2020-11-20T12:06:58.186 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/33' 2020-11-20T12:06:58.918 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/17' 2020-11-20T12:06:58.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/35' 2020-11-20T12:06:58.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './157/37' 2020-11-20T12:06:58.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './157' 2020-11-20T12:06:58.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/19' 2020-11-20T12:06:58.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/15' 2020-11-20T12:06:58.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/31' 2020-11-20T12:06:58.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/36' 2020-11-20T12:06:58.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/20' 2020-11-20T12:06:58.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/12' 2020-11-20T12:06:58.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/39' 2020-11-20T12:06:58.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/10' 2020-11-20T12:06:58.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/34' 2020-11-20T12:06:58.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/38' 2020-11-20T12:06:58.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/11' 2020-11-20T12:06:58.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/16' 2020-11-20T12:06:58.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/32' 2020-11-20T12:06:58.923 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/30' 2020-11-20T12:06:58.923 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/14' 2020-11-20T12:06:58.923 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/18' 2020-11-20T12:06:58.923 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/13' 2020-11-20T12:06:58.924 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/33' 2020-11-20T12:06:58.924 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/17' 2020-11-20T12:06:58.924 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/35' 2020-11-20T12:06:58.924 INFO:tasks.workunit.client.0.smithi044.stdout:removed './158/37' 2020-11-20T12:06:58.924 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './158' 2020-11-20T12:06:58.925 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/19' 2020-11-20T12:06:58.925 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/15' 2020-11-20T12:06:58.925 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/31' 2020-11-20T12:06:58.925 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/36' 2020-11-20T12:06:58.926 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/20' 2020-11-20T12:06:58.926 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/12' 2020-11-20T12:06:58.926 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/39' 2020-11-20T12:06:58.926 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/10' 2020-11-20T12:06:58.926 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/34' 2020-11-20T12:06:58.927 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/38' 2020-11-20T12:06:58.927 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/11' 2020-11-20T12:06:58.927 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/16' 2020-11-20T12:06:58.927 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/32' 2020-11-20T12:06:58.927 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/30' 2020-11-20T12:06:58.928 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/14' 2020-11-20T12:06:58.928 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/18' 2020-11-20T12:06:58.928 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/13' 2020-11-20T12:06:58.928 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/33' 2020-11-20T12:06:58.928 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/17' 2020-11-20T12:06:58.929 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/35' 2020-11-20T12:06:58.929 INFO:tasks.workunit.client.0.smithi044.stdout:removed './159/37' 2020-11-20T12:06:58.929 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './159' 2020-11-20T12:06:58.929 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/19' 2020-11-20T12:06:58.929 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/15' 2020-11-20T12:06:58.930 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/31' 2020-11-20T12:06:58.930 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/36' 2020-11-20T12:06:58.930 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/20' 2020-11-20T12:06:58.930 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/12' 2020-11-20T12:06:58.930 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/39' 2020-11-20T12:06:58.931 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/10' 2020-11-20T12:06:58.931 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/34' 2020-11-20T12:06:58.931 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/38' 2020-11-20T12:06:58.931 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/11' 2020-11-20T12:06:58.931 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/16' 2020-11-20T12:06:58.932 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/32' 2020-11-20T12:06:58.932 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/30' 2020-11-20T12:06:58.932 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/14' 2020-11-20T12:06:58.932 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/18' 2020-11-20T12:06:58.932 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/13' 2020-11-20T12:06:58.933 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/33' 2020-11-20T12:06:58.933 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/17' 2020-11-20T12:06:58.933 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/35' 2020-11-20T12:06:58.933 INFO:tasks.workunit.client.0.smithi044.stdout:removed './160/37' 2020-11-20T12:06:58.933 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './160' 2020-11-20T12:06:58.934 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/19' 2020-11-20T12:06:58.934 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/15' 2020-11-20T12:06:58.934 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/31' 2020-11-20T12:06:58.934 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/36' 2020-11-20T12:06:58.934 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/20' 2020-11-20T12:06:58.935 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/12' 2020-11-20T12:06:58.935 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/39' 2020-11-20T12:06:58.935 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/10' 2020-11-20T12:06:58.935 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/34' 2020-11-20T12:06:58.935 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/38' 2020-11-20T12:06:58.936 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/11' 2020-11-20T12:06:58.936 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/16' 2020-11-20T12:06:58.936 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/32' 2020-11-20T12:06:58.936 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/30' 2020-11-20T12:06:58.936 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/14' 2020-11-20T12:06:58.937 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/18' 2020-11-20T12:06:58.937 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/13' 2020-11-20T12:06:58.937 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/33' 2020-11-20T12:06:58.937 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/17' 2020-11-20T12:06:58.937 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/35' 2020-11-20T12:06:58.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './161/37' 2020-11-20T12:06:58.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './161' 2020-11-20T12:06:58.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/19' 2020-11-20T12:06:58.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/15' 2020-11-20T12:06:58.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/31' 2020-11-20T12:06:58.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/36' 2020-11-20T12:06:58.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/20' 2020-11-20T12:06:58.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/12' 2020-11-20T12:06:58.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/39' 2020-11-20T12:06:58.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/10' 2020-11-20T12:06:58.940 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/34' 2020-11-20T12:06:58.940 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/38' 2020-11-20T12:06:58.940 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/11' 2020-11-20T12:06:58.940 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/16' 2020-11-20T12:06:58.940 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/32' 2020-11-20T12:06:58.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/30' 2020-11-20T12:06:58.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/14' 2020-11-20T12:06:58.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/18' 2020-11-20T12:06:58.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/13' 2020-11-20T12:06:58.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/33' 2020-11-20T12:06:58.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/17' 2020-11-20T12:06:58.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/35' 2020-11-20T12:06:58.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './162/37' 2020-11-20T12:06:58.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './162' 2020-11-20T12:06:58.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/19' 2020-11-20T12:06:58.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/15' 2020-11-20T12:06:58.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/31' 2020-11-20T12:06:58.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/36' 2020-11-20T12:06:58.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/20' 2020-11-20T12:06:58.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/12' 2020-11-20T12:06:58.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/39' 2020-11-20T12:06:58.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/10' 2020-11-20T12:06:58.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/34' 2020-11-20T12:06:58.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/38' 2020-11-20T12:06:58.945 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/11' 2020-11-20T12:06:58.945 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/16' 2020-11-20T12:06:58.945 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/32' 2020-11-20T12:06:58.945 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/30' 2020-11-20T12:06:58.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/14' 2020-11-20T12:06:58.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/18' 2020-11-20T12:06:58.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/13' 2020-11-20T12:06:58.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/33' 2020-11-20T12:06:58.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/17' 2020-11-20T12:06:58.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/35' 2020-11-20T12:06:58.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './163/37' 2020-11-20T12:06:58.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './163' 2020-11-20T12:06:58.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/19' 2020-11-20T12:06:58.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/15' 2020-11-20T12:06:58.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/31' 2020-11-20T12:06:58.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/36' 2020-11-20T12:06:58.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/20' 2020-11-20T12:06:58.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/12' 2020-11-20T12:06:58.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/39' 2020-11-20T12:06:58.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/10' 2020-11-20T12:06:58.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/34' 2020-11-20T12:06:58.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/38' 2020-11-20T12:06:58.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/11' 2020-11-20T12:06:58.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/16' 2020-11-20T12:06:58.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/32' 2020-11-20T12:06:58.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/30' 2020-11-20T12:06:58.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/14' 2020-11-20T12:06:58.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/18' 2020-11-20T12:06:58.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/13' 2020-11-20T12:06:58.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/33' 2020-11-20T12:06:58.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/17' 2020-11-20T12:06:58.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/35' 2020-11-20T12:06:58.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './164/37' 2020-11-20T12:06:58.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './164' 2020-11-20T12:06:58.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/19' 2020-11-20T12:06:58.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/15' 2020-11-20T12:06:58.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/31' 2020-11-20T12:06:58.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/36' 2020-11-20T12:06:58.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/20' 2020-11-20T12:06:58.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/12' 2020-11-20T12:06:58.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/39' 2020-11-20T12:06:58.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/10' 2020-11-20T12:06:58.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/34' 2020-11-20T12:06:58.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/38' 2020-11-20T12:06:58.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/11' 2020-11-20T12:06:58.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/16' 2020-11-20T12:06:58.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/32' 2020-11-20T12:06:58.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/30' 2020-11-20T12:06:58.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/14' 2020-11-20T12:06:58.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/18' 2020-11-20T12:06:58.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/13' 2020-11-20T12:06:58.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/33' 2020-11-20T12:06:58.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/17' 2020-11-20T12:06:58.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/35' 2020-11-20T12:06:58.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './165/37' 2020-11-20T12:06:58.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './165' 2020-11-20T12:06:58.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/19' 2020-11-20T12:06:58.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/15' 2020-11-20T12:06:58.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/31' 2020-11-20T12:06:58.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/36' 2020-11-20T12:06:58.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/20' 2020-11-20T12:06:58.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/12' 2020-11-20T12:06:58.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/39' 2020-11-20T12:06:58.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/10' 2020-11-20T12:06:58.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/34' 2020-11-20T12:06:58.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/38' 2020-11-20T12:06:58.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/11' 2020-11-20T12:06:58.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/16' 2020-11-20T12:06:58.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/32' 2020-11-20T12:06:58.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/30' 2020-11-20T12:06:58.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/14' 2020-11-20T12:06:58.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/18' 2020-11-20T12:06:58.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/13' 2020-11-20T12:06:58.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/33' 2020-11-20T12:06:58.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/17' 2020-11-20T12:06:58.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/35' 2020-11-20T12:06:58.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './166/37' 2020-11-20T12:06:58.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './166' 2020-11-20T12:06:58.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/19' 2020-11-20T12:06:58.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/15' 2020-11-20T12:06:58.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/31' 2020-11-20T12:06:58.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/36' 2020-11-20T12:06:58.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/20' 2020-11-20T12:06:58.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/12' 2020-11-20T12:06:58.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/39' 2020-11-20T12:06:58.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/10' 2020-11-20T12:06:58.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/34' 2020-11-20T12:06:58.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/38' 2020-11-20T12:06:59.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/11' 2020-11-20T12:06:59.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/16' 2020-11-20T12:06:59.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/32' 2020-11-20T12:06:59.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/30' 2020-11-20T12:06:59.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/14' 2020-11-20T12:06:59.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/18' 2020-11-20T12:06:59.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/13' 2020-11-20T12:06:59.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/33' 2020-11-20T12:06:59.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/17' 2020-11-20T12:06:59.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/35' 2020-11-20T12:06:59.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './167/37' 2020-11-20T12:06:59.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './167' 2020-11-20T12:06:59.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/19' 2020-11-20T12:06:59.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/15' 2020-11-20T12:06:59.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/31' 2020-11-20T12:06:59.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/36' 2020-11-20T12:06:59.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/20' 2020-11-20T12:06:59.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/12' 2020-11-20T12:06:59.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/39' 2020-11-20T12:06:59.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/10' 2020-11-20T12:06:59.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/34' 2020-11-20T12:06:59.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/38' 2020-11-20T12:06:59.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/11' 2020-11-20T12:06:59.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/16' 2020-11-20T12:06:59.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/32' 2020-11-20T12:06:59.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/30' 2020-11-20T12:06:59.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/14' 2020-11-20T12:06:59.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/18' 2020-11-20T12:06:59.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/13' 2020-11-20T12:06:59.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/33' 2020-11-20T12:06:59.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/17' 2020-11-20T12:06:59.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/35' 2020-11-20T12:06:59.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './168/37' 2020-11-20T12:06:59.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './168' 2020-11-20T12:06:59.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/19' 2020-11-20T12:06:59.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/15' 2020-11-20T12:06:59.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/31' 2020-11-20T12:06:59.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/36' 2020-11-20T12:06:59.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/20' 2020-11-20T12:06:59.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/12' 2020-11-20T12:06:59.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/39' 2020-11-20T12:06:59.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/10' 2020-11-20T12:06:59.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/34' 2020-11-20T12:06:59.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/38' 2020-11-20T12:06:59.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/11' 2020-11-20T12:06:59.770 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/16' 2020-11-20T12:06:59.770 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/32' 2020-11-20T12:06:59.770 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/30' 2020-11-20T12:06:59.770 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/14' 2020-11-20T12:06:59.770 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/18' 2020-11-20T12:06:59.771 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/13' 2020-11-20T12:06:59.771 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/33' 2020-11-20T12:06:59.771 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/17' 2020-11-20T12:06:59.771 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/35' 2020-11-20T12:06:59.771 INFO:tasks.workunit.client.0.smithi044.stdout:removed './169/37' 2020-11-20T12:06:59.772 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './169' 2020-11-20T12:06:59.772 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/19' 2020-11-20T12:06:59.772 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/15' 2020-11-20T12:06:59.772 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/31' 2020-11-20T12:06:59.773 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/36' 2020-11-20T12:06:59.773 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/20' 2020-11-20T12:06:59.773 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/12' 2020-11-20T12:06:59.773 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/39' 2020-11-20T12:06:59.774 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/10' 2020-11-20T12:06:59.774 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/34' 2020-11-20T12:06:59.774 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/38' 2020-11-20T12:06:59.774 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/11' 2020-11-20T12:06:59.774 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/16' 2020-11-20T12:06:59.775 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/32' 2020-11-20T12:06:59.775 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/30' 2020-11-20T12:06:59.775 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/14' 2020-11-20T12:06:59.775 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/18' 2020-11-20T12:06:59.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/13' 2020-11-20T12:06:59.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/33' 2020-11-20T12:06:59.776 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/17' 2020-11-20T12:06:59.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/35' 2020-11-20T12:06:59.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed './170/37' 2020-11-20T12:06:59.777 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './170' 2020-11-20T12:06:59.778 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/19' 2020-11-20T12:06:59.778 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/15' 2020-11-20T12:06:59.778 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/31' 2020-11-20T12:06:59.778 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/36' 2020-11-20T12:06:59.778 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/20' 2020-11-20T12:06:59.779 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/12' 2020-11-20T12:06:59.779 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/39' 2020-11-20T12:06:59.779 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/10' 2020-11-20T12:06:59.779 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/34' 2020-11-20T12:06:59.780 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/38' 2020-11-20T12:06:59.780 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/11' 2020-11-20T12:06:59.780 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/16' 2020-11-20T12:06:59.780 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/32' 2020-11-20T12:06:59.780 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/30' 2020-11-20T12:06:59.781 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/14' 2020-11-20T12:06:59.781 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/18' 2020-11-20T12:06:59.782 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/13' 2020-11-20T12:06:59.782 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/33' 2020-11-20T12:06:59.782 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/17' 2020-11-20T12:06:59.783 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/35' 2020-11-20T12:06:59.783 INFO:tasks.workunit.client.0.smithi044.stdout:removed './171/37' 2020-11-20T12:06:59.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './171' 2020-11-20T12:06:59.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/19' 2020-11-20T12:06:59.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/15' 2020-11-20T12:06:59.784 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/31' 2020-11-20T12:06:59.785 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/36' 2020-11-20T12:06:59.785 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/20' 2020-11-20T12:06:59.785 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/12' 2020-11-20T12:06:59.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/39' 2020-11-20T12:06:59.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/10' 2020-11-20T12:06:59.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/34' 2020-11-20T12:06:59.786 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/38' 2020-11-20T12:06:59.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/11' 2020-11-20T12:06:59.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/16' 2020-11-20T12:06:59.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/32' 2020-11-20T12:06:59.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/30' 2020-11-20T12:06:59.787 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/14' 2020-11-20T12:06:59.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/18' 2020-11-20T12:06:59.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/13' 2020-11-20T12:06:59.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/33' 2020-11-20T12:06:59.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/17' 2020-11-20T12:06:59.788 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/35' 2020-11-20T12:06:59.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './172/37' 2020-11-20T12:06:59.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './172' 2020-11-20T12:06:59.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/19' 2020-11-20T12:06:59.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/15' 2020-11-20T12:06:59.789 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/31' 2020-11-20T12:06:59.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/36' 2020-11-20T12:06:59.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/20' 2020-11-20T12:06:59.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/12' 2020-11-20T12:06:59.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/39' 2020-11-20T12:06:59.790 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/10' 2020-11-20T12:06:59.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/34' 2020-11-20T12:06:59.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/38' 2020-11-20T12:06:59.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/11' 2020-11-20T12:06:59.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/16' 2020-11-20T12:06:59.791 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/32' 2020-11-20T12:06:59.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/30' 2020-11-20T12:06:59.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/14' 2020-11-20T12:06:59.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/18' 2020-11-20T12:06:59.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/13' 2020-11-20T12:06:59.792 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/33' 2020-11-20T12:06:59.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/17' 2020-11-20T12:06:59.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/35' 2020-11-20T12:06:59.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed './173/37' 2020-11-20T12:06:59.793 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './173' 2020-11-20T12:06:59.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/19' 2020-11-20T12:06:59.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/15' 2020-11-20T12:06:59.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/31' 2020-11-20T12:06:59.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/36' 2020-11-20T12:06:59.794 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/20' 2020-11-20T12:06:59.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/12' 2020-11-20T12:06:59.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/39' 2020-11-20T12:06:59.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/10' 2020-11-20T12:06:59.795 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/34' 2020-11-20T12:06:59.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/38' 2020-11-20T12:06:59.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/11' 2020-11-20T12:06:59.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/16' 2020-11-20T12:06:59.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/32' 2020-11-20T12:06:59.796 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/30' 2020-11-20T12:06:59.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/14' 2020-11-20T12:06:59.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/18' 2020-11-20T12:06:59.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/13' 2020-11-20T12:06:59.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/33' 2020-11-20T12:06:59.797 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/17' 2020-11-20T12:06:59.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/35' 2020-11-20T12:06:59.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './174/37' 2020-11-20T12:06:59.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './174' 2020-11-20T12:06:59.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/19' 2020-11-20T12:06:59.798 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/15' 2020-11-20T12:06:59.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/31' 2020-11-20T12:06:59.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/36' 2020-11-20T12:06:59.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/20' 2020-11-20T12:06:59.799 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/12' 2020-11-20T12:06:59.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/39' 2020-11-20T12:06:59.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/10' 2020-11-20T12:06:59.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/34' 2020-11-20T12:06:59.800 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/38' 2020-11-20T12:06:59.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/11' 2020-11-20T12:06:59.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/16' 2020-11-20T12:06:59.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/32' 2020-11-20T12:06:59.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/30' 2020-11-20T12:06:59.801 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/14' 2020-11-20T12:06:59.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/18' 2020-11-20T12:06:59.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/13' 2020-11-20T12:06:59.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/33' 2020-11-20T12:06:59.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/17' 2020-11-20T12:06:59.802 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/35' 2020-11-20T12:06:59.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './175/37' 2020-11-20T12:06:59.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './175' 2020-11-20T12:06:59.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/19' 2020-11-20T12:06:59.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/15' 2020-11-20T12:06:59.803 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/31' 2020-11-20T12:06:59.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/36' 2020-11-20T12:06:59.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/20' 2020-11-20T12:06:59.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/12' 2020-11-20T12:06:59.804 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/39' 2020-11-20T12:06:59.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/10' 2020-11-20T12:06:59.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/34' 2020-11-20T12:06:59.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/38' 2020-11-20T12:06:59.805 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/11' 2020-11-20T12:06:59.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/16' 2020-11-20T12:06:59.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/32' 2020-11-20T12:06:59.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/30' 2020-11-20T12:06:59.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/14' 2020-11-20T12:06:59.806 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/18' 2020-11-20T12:06:59.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/13' 2020-11-20T12:06:59.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/33' 2020-11-20T12:06:59.807 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/17' 2020-11-20T12:06:59.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/35' 2020-11-20T12:06:59.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed './176/37' 2020-11-20T12:06:59.808 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './176' 2020-11-20T12:06:59.809 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/19' 2020-11-20T12:06:59.809 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/15' 2020-11-20T12:07:00.637 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/31' 2020-11-20T12:07:00.637 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/36' 2020-11-20T12:07:00.638 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/20' 2020-11-20T12:07:00.638 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/12' 2020-11-20T12:07:00.638 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/39' 2020-11-20T12:07:00.638 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/10' 2020-11-20T12:07:00.638 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/34' 2020-11-20T12:07:00.639 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/38' 2020-11-20T12:07:00.639 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/11' 2020-11-20T12:07:00.639 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/16' 2020-11-20T12:07:00.639 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/32' 2020-11-20T12:07:00.639 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/30' 2020-11-20T12:07:00.640 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/14' 2020-11-20T12:07:00.640 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/18' 2020-11-20T12:07:00.640 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/13' 2020-11-20T12:07:00.640 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/33' 2020-11-20T12:07:00.640 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/17' 2020-11-20T12:07:00.641 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/35' 2020-11-20T12:07:00.641 INFO:tasks.workunit.client.0.smithi044.stdout:removed './177/37' 2020-11-20T12:07:00.641 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './177' 2020-11-20T12:07:00.641 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/19' 2020-11-20T12:07:00.641 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/15' 2020-11-20T12:07:00.642 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/31' 2020-11-20T12:07:00.642 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/36' 2020-11-20T12:07:00.642 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/20' 2020-11-20T12:07:00.642 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/12' 2020-11-20T12:07:00.642 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/39' 2020-11-20T12:07:00.643 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/10' 2020-11-20T12:07:00.643 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/34' 2020-11-20T12:07:00.643 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/38' 2020-11-20T12:07:00.643 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/11' 2020-11-20T12:07:00.643 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/16' 2020-11-20T12:07:00.644 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/32' 2020-11-20T12:07:00.644 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/30' 2020-11-20T12:07:00.644 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/14' 2020-11-20T12:07:00.644 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/18' 2020-11-20T12:07:00.644 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/13' 2020-11-20T12:07:00.645 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/33' 2020-11-20T12:07:00.645 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/17' 2020-11-20T12:07:00.645 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/35' 2020-11-20T12:07:00.645 INFO:tasks.workunit.client.0.smithi044.stdout:removed './178/37' 2020-11-20T12:07:00.645 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './178' 2020-11-20T12:07:00.646 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/19' 2020-11-20T12:07:00.646 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/15' 2020-11-20T12:07:00.646 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/31' 2020-11-20T12:07:00.646 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/36' 2020-11-20T12:07:00.646 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/20' 2020-11-20T12:07:00.647 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/12' 2020-11-20T12:07:00.647 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/39' 2020-11-20T12:07:00.647 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/10' 2020-11-20T12:07:00.647 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/34' 2020-11-20T12:07:00.648 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/38' 2020-11-20T12:07:00.648 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/11' 2020-11-20T12:07:00.648 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/16' 2020-11-20T12:07:00.648 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/32' 2020-11-20T12:07:00.648 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/30' 2020-11-20T12:07:00.648 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/14' 2020-11-20T12:07:00.649 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/18' 2020-11-20T12:07:00.649 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/13' 2020-11-20T12:07:00.649 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/33' 2020-11-20T12:07:00.649 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/17' 2020-11-20T12:07:00.650 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/35' 2020-11-20T12:07:00.650 INFO:tasks.workunit.client.0.smithi044.stdout:removed './179/37' 2020-11-20T12:07:00.650 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './179' 2020-11-20T12:07:00.650 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/19' 2020-11-20T12:07:00.650 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/15' 2020-11-20T12:07:00.651 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/31' 2020-11-20T12:07:00.651 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/36' 2020-11-20T12:07:00.651 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/20' 2020-11-20T12:07:00.651 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/12' 2020-11-20T12:07:00.651 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/39' 2020-11-20T12:07:00.652 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/10' 2020-11-20T12:07:00.652 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/34' 2020-11-20T12:07:00.652 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/38' 2020-11-20T12:07:00.652 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/11' 2020-11-20T12:07:00.652 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/16' 2020-11-20T12:07:00.653 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/32' 2020-11-20T12:07:00.653 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/30' 2020-11-20T12:07:00.653 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/14' 2020-11-20T12:07:00.653 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/18' 2020-11-20T12:07:00.654 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/13' 2020-11-20T12:07:00.654 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/33' 2020-11-20T12:07:00.654 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/17' 2020-11-20T12:07:00.654 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/35' 2020-11-20T12:07:00.654 INFO:tasks.workunit.client.0.smithi044.stdout:removed './180/37' 2020-11-20T12:07:00.655 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './180' 2020-11-20T12:07:00.655 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/19' 2020-11-20T12:07:00.655 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/15' 2020-11-20T12:07:00.655 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/31' 2020-11-20T12:07:00.655 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/36' 2020-11-20T12:07:00.656 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/20' 2020-11-20T12:07:00.656 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/12' 2020-11-20T12:07:00.656 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/39' 2020-11-20T12:07:00.656 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/10' 2020-11-20T12:07:00.656 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/34' 2020-11-20T12:07:00.657 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/38' 2020-11-20T12:07:00.657 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/11' 2020-11-20T12:07:00.657 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/16' 2020-11-20T12:07:00.657 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/32' 2020-11-20T12:07:00.657 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/30' 2020-11-20T12:07:00.658 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/14' 2020-11-20T12:07:00.658 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/18' 2020-11-20T12:07:00.658 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/13' 2020-11-20T12:07:00.658 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/33' 2020-11-20T12:07:00.658 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/17' 2020-11-20T12:07:00.659 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/35' 2020-11-20T12:07:00.659 INFO:tasks.workunit.client.0.smithi044.stdout:removed './181/37' 2020-11-20T12:07:00.659 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './181' 2020-11-20T12:07:00.659 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/19' 2020-11-20T12:07:00.659 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/15' 2020-11-20T12:07:00.660 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/31' 2020-11-20T12:07:00.660 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/36' 2020-11-20T12:07:00.660 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/20' 2020-11-20T12:07:00.660 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/12' 2020-11-20T12:07:00.660 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/39' 2020-11-20T12:07:00.661 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/10' 2020-11-20T12:07:00.661 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/34' 2020-11-20T12:07:00.661 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/38' 2020-11-20T12:07:00.661 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/11' 2020-11-20T12:07:00.661 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/16' 2020-11-20T12:07:00.662 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/32' 2020-11-20T12:07:00.662 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/30' 2020-11-20T12:07:00.662 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/14' 2020-11-20T12:07:00.662 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/18' 2020-11-20T12:07:00.662 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/13' 2020-11-20T12:07:00.663 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/33' 2020-11-20T12:07:00.663 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/17' 2020-11-20T12:07:00.663 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/35' 2020-11-20T12:07:00.663 INFO:tasks.workunit.client.0.smithi044.stdout:removed './182/37' 2020-11-20T12:07:00.663 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './182' 2020-11-20T12:07:00.664 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/19' 2020-11-20T12:07:00.664 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/15' 2020-11-20T12:07:00.664 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/31' 2020-11-20T12:07:00.664 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/36' 2020-11-20T12:07:00.664 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/20' 2020-11-20T12:07:00.665 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/12' 2020-11-20T12:07:00.665 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/39' 2020-11-20T12:07:00.665 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/10' 2020-11-20T12:07:00.665 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/34' 2020-11-20T12:07:00.665 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/38' 2020-11-20T12:07:00.666 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/11' 2020-11-20T12:07:00.666 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/16' 2020-11-20T12:07:00.666 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/32' 2020-11-20T12:07:00.666 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/30' 2020-11-20T12:07:00.666 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/14' 2020-11-20T12:07:00.667 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/18' 2020-11-20T12:07:00.667 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/13' 2020-11-20T12:07:00.667 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/33' 2020-11-20T12:07:00.667 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/17' 2020-11-20T12:07:00.667 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/35' 2020-11-20T12:07:00.668 INFO:tasks.workunit.client.0.smithi044.stdout:removed './183/37' 2020-11-20T12:07:00.668 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './183' 2020-11-20T12:07:00.668 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/19' 2020-11-20T12:07:00.668 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/15' 2020-11-20T12:07:00.668 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/31' 2020-11-20T12:07:00.669 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/36' 2020-11-20T12:07:00.669 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/20' 2020-11-20T12:07:00.669 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/12' 2020-11-20T12:07:00.669 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/39' 2020-11-20T12:07:00.669 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/10' 2020-11-20T12:07:00.670 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/34' 2020-11-20T12:07:00.670 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/38' 2020-11-20T12:07:00.670 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/11' 2020-11-20T12:07:00.670 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/16' 2020-11-20T12:07:00.670 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/32' 2020-11-20T12:07:00.671 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/30' 2020-11-20T12:07:00.671 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/14' 2020-11-20T12:07:00.671 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/18' 2020-11-20T12:07:00.671 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/13' 2020-11-20T12:07:00.671 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/33' 2020-11-20T12:07:00.671 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/17' 2020-11-20T12:07:00.672 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/35' 2020-11-20T12:07:00.672 INFO:tasks.workunit.client.0.smithi044.stdout:removed './184/37' 2020-11-20T12:07:00.672 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './184' 2020-11-20T12:07:00.672 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/19' 2020-11-20T12:07:00.672 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/15' 2020-11-20T12:07:00.673 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/31' 2020-11-20T12:07:00.673 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/36' 2020-11-20T12:07:00.673 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/20' 2020-11-20T12:07:00.673 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/12' 2020-11-20T12:07:00.673 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/39' 2020-11-20T12:07:00.674 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/10' 2020-11-20T12:07:00.674 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/34' 2020-11-20T12:07:00.675 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/38' 2020-11-20T12:07:00.675 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/11' 2020-11-20T12:07:00.675 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/16' 2020-11-20T12:07:00.675 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/32' 2020-11-20T12:07:00.675 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/30' 2020-11-20T12:07:00.676 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/14' 2020-11-20T12:07:00.676 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/18' 2020-11-20T12:07:00.676 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/13' 2020-11-20T12:07:00.676 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/33' 2020-11-20T12:07:00.676 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/17' 2020-11-20T12:07:00.677 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/35' 2020-11-20T12:07:00.677 INFO:tasks.workunit.client.0.smithi044.stdout:removed './185/37' 2020-11-20T12:07:00.677 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './185' 2020-11-20T12:07:00.677 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/19' 2020-11-20T12:07:00.678 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/15' 2020-11-20T12:07:00.678 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/31' 2020-11-20T12:07:00.678 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/36' 2020-11-20T12:07:00.678 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/20' 2020-11-20T12:07:00.678 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/12' 2020-11-20T12:07:00.679 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/39' 2020-11-20T12:07:00.679 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/10' 2020-11-20T12:07:00.679 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/34' 2020-11-20T12:07:00.679 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/38' 2020-11-20T12:07:00.679 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/11' 2020-11-20T12:07:00.680 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/16' 2020-11-20T12:07:00.680 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/32' 2020-11-20T12:07:00.680 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/30' 2020-11-20T12:07:00.680 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/14' 2020-11-20T12:07:00.680 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/18' 2020-11-20T12:07:01.246 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/13' 2020-11-20T12:07:01.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/33' 2020-11-20T12:07:01.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/17' 2020-11-20T12:07:01.247 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/35' 2020-11-20T12:07:01.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './186/37' 2020-11-20T12:07:01.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './186' 2020-11-20T12:07:01.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/19' 2020-11-20T12:07:01.248 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/15' 2020-11-20T12:07:01.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/31' 2020-11-20T12:07:01.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/36' 2020-11-20T12:07:01.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/20' 2020-11-20T12:07:01.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/12' 2020-11-20T12:07:01.249 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/39' 2020-11-20T12:07:01.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/10' 2020-11-20T12:07:01.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/34' 2020-11-20T12:07:01.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/38' 2020-11-20T12:07:01.250 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/11' 2020-11-20T12:07:01.251 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/16' 2020-11-20T12:07:01.251 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/32' 2020-11-20T12:07:01.251 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/30' 2020-11-20T12:07:01.251 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/14' 2020-11-20T12:07:01.251 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/18' 2020-11-20T12:07:01.252 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/13' 2020-11-20T12:07:01.252 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/33' 2020-11-20T12:07:01.252 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/17' 2020-11-20T12:07:01.252 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/35' 2020-11-20T12:07:01.252 INFO:tasks.workunit.client.0.smithi044.stdout:removed './187/37' 2020-11-20T12:07:01.253 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './187' 2020-11-20T12:07:01.253 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/19' 2020-11-20T12:07:01.253 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/15' 2020-11-20T12:07:01.253 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/31' 2020-11-20T12:07:01.253 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/36' 2020-11-20T12:07:01.254 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/20' 2020-11-20T12:07:01.254 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/12' 2020-11-20T12:07:01.254 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/39' 2020-11-20T12:07:01.254 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/10' 2020-11-20T12:07:01.254 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/34' 2020-11-20T12:07:01.255 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/38' 2020-11-20T12:07:01.255 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/11' 2020-11-20T12:07:01.255 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/16' 2020-11-20T12:07:01.255 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/32' 2020-11-20T12:07:01.255 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/30' 2020-11-20T12:07:01.256 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/14' 2020-11-20T12:07:01.256 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/18' 2020-11-20T12:07:01.256 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/13' 2020-11-20T12:07:01.256 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/33' 2020-11-20T12:07:01.257 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/17' 2020-11-20T12:07:01.257 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/35' 2020-11-20T12:07:01.257 INFO:tasks.workunit.client.0.smithi044.stdout:removed './188/37' 2020-11-20T12:07:01.258 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './188' 2020-11-20T12:07:01.258 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/19' 2020-11-20T12:07:01.258 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/15' 2020-11-20T12:07:01.258 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/31' 2020-11-20T12:07:01.258 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/36' 2020-11-20T12:07:01.259 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/20' 2020-11-20T12:07:01.259 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/12' 2020-11-20T12:07:01.259 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/39' 2020-11-20T12:07:01.259 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/10' 2020-11-20T12:07:01.260 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/34' 2020-11-20T12:07:01.260 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/38' 2020-11-20T12:07:01.260 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/11' 2020-11-20T12:07:01.260 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/16' 2020-11-20T12:07:01.260 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/32' 2020-11-20T12:07:01.261 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/30' 2020-11-20T12:07:01.261 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/14' 2020-11-20T12:07:01.261 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/18' 2020-11-20T12:07:01.262 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/13' 2020-11-20T12:07:01.262 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/33' 2020-11-20T12:07:01.263 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/17' 2020-11-20T12:07:01.263 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/35' 2020-11-20T12:07:01.263 INFO:tasks.workunit.client.0.smithi044.stdout:removed './189/37' 2020-11-20T12:07:01.264 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './189' 2020-11-20T12:07:01.264 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/19' 2020-11-20T12:07:01.264 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/15' 2020-11-20T12:07:01.264 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/31' 2020-11-20T12:07:01.265 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/36' 2020-11-20T12:07:01.265 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/20' 2020-11-20T12:07:01.265 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/12' 2020-11-20T12:07:01.265 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/39' 2020-11-20T12:07:01.266 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/10' 2020-11-20T12:07:01.266 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/34' 2020-11-20T12:07:01.267 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/38' 2020-11-20T12:07:01.267 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/11' 2020-11-20T12:07:01.267 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/16' 2020-11-20T12:07:01.268 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/32' 2020-11-20T12:07:01.268 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/30' 2020-11-20T12:07:01.268 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/14' 2020-11-20T12:07:01.269 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/18' 2020-11-20T12:07:01.269 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/13' 2020-11-20T12:07:01.269 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/33' 2020-11-20T12:07:01.269 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/17' 2020-11-20T12:07:01.270 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/35' 2020-11-20T12:07:01.270 INFO:tasks.workunit.client.0.smithi044.stdout:removed './190/37' 2020-11-20T12:07:01.270 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './190' 2020-11-20T12:07:01.270 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/19' 2020-11-20T12:07:01.270 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/15' 2020-11-20T12:07:01.271 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/31' 2020-11-20T12:07:01.271 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/36' 2020-11-20T12:07:01.271 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/20' 2020-11-20T12:07:01.271 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/12' 2020-11-20T12:07:01.272 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/39' 2020-11-20T12:07:01.272 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/10' 2020-11-20T12:07:01.273 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/34' 2020-11-20T12:07:01.273 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/38' 2020-11-20T12:07:01.273 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/11' 2020-11-20T12:07:01.274 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/16' 2020-11-20T12:07:01.274 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/32' 2020-11-20T12:07:01.275 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/30' 2020-11-20T12:07:01.275 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/14' 2020-11-20T12:07:01.275 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/18' 2020-11-20T12:07:01.276 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/13' 2020-11-20T12:07:01.276 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/33' 2020-11-20T12:07:01.277 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/17' 2020-11-20T12:07:01.277 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/35' 2020-11-20T12:07:01.277 INFO:tasks.workunit.client.0.smithi044.stdout:removed './191/37' 2020-11-20T12:07:01.278 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './191' 2020-11-20T12:07:01.278 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/19' 2020-11-20T12:07:01.278 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/15' 2020-11-20T12:07:01.279 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/31' 2020-11-20T12:07:01.279 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/36' 2020-11-20T12:07:01.279 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/20' 2020-11-20T12:07:01.280 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/12' 2020-11-20T12:07:01.280 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/39' 2020-11-20T12:07:01.281 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/10' 2020-11-20T12:07:01.281 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/34' 2020-11-20T12:07:01.281 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/38' 2020-11-20T12:07:01.282 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/11' 2020-11-20T12:07:01.282 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/16' 2020-11-20T12:07:01.283 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/32' 2020-11-20T12:07:01.283 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/30' 2020-11-20T12:07:01.283 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/14' 2020-11-20T12:07:01.284 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/18' 2020-11-20T12:07:01.284 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/13' 2020-11-20T12:07:01.285 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/33' 2020-11-20T12:07:01.285 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/17' 2020-11-20T12:07:01.285 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/35' 2020-11-20T12:07:01.286 INFO:tasks.workunit.client.0.smithi044.stdout:removed './192/37' 2020-11-20T12:07:01.286 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './192' 2020-11-20T12:07:01.286 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/19' 2020-11-20T12:07:01.286 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/15' 2020-11-20T12:07:01.287 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/31' 2020-11-20T12:07:01.287 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/36' 2020-11-20T12:07:01.287 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/20' 2020-11-20T12:07:01.287 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/12' 2020-11-20T12:07:01.287 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/39' 2020-11-20T12:07:01.288 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/10' 2020-11-20T12:07:01.288 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/34' 2020-11-20T12:07:01.288 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/38' 2020-11-20T12:07:01.288 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/11' 2020-11-20T12:07:01.288 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/16' 2020-11-20T12:07:01.289 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/32' 2020-11-20T12:07:01.289 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/30' 2020-11-20T12:07:01.289 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/14' 2020-11-20T12:07:01.289 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/18' 2020-11-20T12:07:01.289 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/13' 2020-11-20T12:07:01.290 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/33' 2020-11-20T12:07:01.290 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/17' 2020-11-20T12:07:01.290 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/35' 2020-11-20T12:07:01.290 INFO:tasks.workunit.client.0.smithi044.stdout:removed './193/37' 2020-11-20T12:07:01.290 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './193' 2020-11-20T12:07:01.291 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/19' 2020-11-20T12:07:01.291 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/15' 2020-11-20T12:07:01.291 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/31' 2020-11-20T12:07:01.291 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/36' 2020-11-20T12:07:01.292 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/20' 2020-11-20T12:07:01.292 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/12' 2020-11-20T12:07:01.292 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/39' 2020-11-20T12:07:01.292 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/10' 2020-11-20T12:07:01.292 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/34' 2020-11-20T12:07:01.293 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/38' 2020-11-20T12:07:01.293 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/11' 2020-11-20T12:07:01.293 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/16' 2020-11-20T12:07:01.293 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/32' 2020-11-20T12:07:01.293 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/30' 2020-11-20T12:07:01.294 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/14' 2020-11-20T12:07:01.294 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/18' 2020-11-20T12:07:01.294 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/13' 2020-11-20T12:07:01.294 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/33' 2020-11-20T12:07:01.294 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/17' 2020-11-20T12:07:01.295 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/35' 2020-11-20T12:07:01.295 INFO:tasks.workunit.client.0.smithi044.stdout:removed './194/37' 2020-11-20T12:07:01.295 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './194' 2020-11-20T12:07:01.295 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/19' 2020-11-20T12:07:01.295 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/15' 2020-11-20T12:07:01.296 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/31' 2020-11-20T12:07:01.296 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/36' 2020-11-20T12:07:01.297 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/20' 2020-11-20T12:07:01.297 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/12' 2020-11-20T12:07:01.297 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/39' 2020-11-20T12:07:01.297 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/10' 2020-11-20T12:07:01.298 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/34' 2020-11-20T12:07:01.298 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/38' 2020-11-20T12:07:01.298 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/11' 2020-11-20T12:07:01.298 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/16' 2020-11-20T12:07:01.299 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/32' 2020-11-20T12:07:01.299 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/30' 2020-11-20T12:07:01.299 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/14' 2020-11-20T12:07:01.299 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/18' 2020-11-20T12:07:01.300 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/13' 2020-11-20T12:07:01.300 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/33' 2020-11-20T12:07:01.300 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/17' 2020-11-20T12:07:01.300 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/35' 2020-11-20T12:07:01.300 INFO:tasks.workunit.client.0.smithi044.stdout:removed './195/37' 2020-11-20T12:07:01.301 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './195' 2020-11-20T12:07:01.301 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/19' 2020-11-20T12:07:01.301 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/15' 2020-11-20T12:07:01.301 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/31' 2020-11-20T12:07:01.301 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/36' 2020-11-20T12:07:01.302 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/20' 2020-11-20T12:07:01.302 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/12' 2020-11-20T12:07:01.302 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/39' 2020-11-20T12:07:01.302 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/10' 2020-11-20T12:07:02.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/34' 2020-11-20T12:07:02.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/38' 2020-11-20T12:07:02.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/11' 2020-11-20T12:07:02.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/16' 2020-11-20T12:07:02.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/32' 2020-11-20T12:07:02.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/30' 2020-11-20T12:07:02.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/14' 2020-11-20T12:07:02.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/18' 2020-11-20T12:07:02.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/13' 2020-11-20T12:07:02.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/33' 2020-11-20T12:07:02.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/17' 2020-11-20T12:07:02.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/35' 2020-11-20T12:07:02.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './196/37' 2020-11-20T12:07:02.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './196' 2020-11-20T12:07:02.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/19' 2020-11-20T12:07:02.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/15' 2020-11-20T12:07:02.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/31' 2020-11-20T12:07:02.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/36' 2020-11-20T12:07:02.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/20' 2020-11-20T12:07:02.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/12' 2020-11-20T12:07:02.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/39' 2020-11-20T12:07:02.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/10' 2020-11-20T12:07:02.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/34' 2020-11-20T12:07:02.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/38' 2020-11-20T12:07:02.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/11' 2020-11-20T12:07:02.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/16' 2020-11-20T12:07:02.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/32' 2020-11-20T12:07:02.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/30' 2020-11-20T12:07:02.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/14' 2020-11-20T12:07:02.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/18' 2020-11-20T12:07:02.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/13' 2020-11-20T12:07:02.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/33' 2020-11-20T12:07:02.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/17' 2020-11-20T12:07:02.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/35' 2020-11-20T12:07:02.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './197/37' 2020-11-20T12:07:02.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './197' 2020-11-20T12:07:02.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/19' 2020-11-20T12:07:02.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/15' 2020-11-20T12:07:02.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/31' 2020-11-20T12:07:02.076 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/36' 2020-11-20T12:07:02.076 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/20' 2020-11-20T12:07:02.076 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/12' 2020-11-20T12:07:02.076 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/39' 2020-11-20T12:07:02.077 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/10' 2020-11-20T12:07:02.077 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/34' 2020-11-20T12:07:02.077 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/38' 2020-11-20T12:07:02.077 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/11' 2020-11-20T12:07:02.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/16' 2020-11-20T12:07:02.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/32' 2020-11-20T12:07:02.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/30' 2020-11-20T12:07:02.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/14' 2020-11-20T12:07:02.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/18' 2020-11-20T12:07:02.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/13' 2020-11-20T12:07:02.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/33' 2020-11-20T12:07:02.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/17' 2020-11-20T12:07:02.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/35' 2020-11-20T12:07:02.080 INFO:tasks.workunit.client.0.smithi044.stdout:removed './198/37' 2020-11-20T12:07:02.080 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './198' 2020-11-20T12:07:02.080 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/19' 2020-11-20T12:07:02.081 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/15' 2020-11-20T12:07:02.081 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/31' 2020-11-20T12:07:02.081 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/36' 2020-11-20T12:07:02.081 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/20' 2020-11-20T12:07:02.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/12' 2020-11-20T12:07:02.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/39' 2020-11-20T12:07:02.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/10' 2020-11-20T12:07:02.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/34' 2020-11-20T12:07:02.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/38' 2020-11-20T12:07:02.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/11' 2020-11-20T12:07:02.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/16' 2020-11-20T12:07:02.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/32' 2020-11-20T12:07:02.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/30' 2020-11-20T12:07:02.084 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/14' 2020-11-20T12:07:02.084 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/18' 2020-11-20T12:07:02.084 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/13' 2020-11-20T12:07:02.084 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/33' 2020-11-20T12:07:02.085 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/17' 2020-11-20T12:07:02.085 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/35' 2020-11-20T12:07:02.085 INFO:tasks.workunit.client.0.smithi044.stdout:removed './199/37' 2020-11-20T12:07:02.085 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './199' 2020-11-20T12:07:02.086 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/21' 2020-11-20T12:07:02.086 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/27' 2020-11-20T12:07:02.087 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/31' 2020-11-20T12:07:02.087 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/36' 2020-11-20T12:07:02.087 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/20' 2020-11-20T12:07:02.087 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/39' 2020-11-20T12:07:02.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/34' 2020-11-20T12:07:02.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/38' 2020-11-20T12:07:02.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/28' 2020-11-20T12:07:02.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/24' 2020-11-20T12:07:02.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/29' 2020-11-20T12:07:02.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/32' 2020-11-20T12:07:02.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/22' 2020-11-20T12:07:02.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/25' 2020-11-20T12:07:02.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/30' 2020-11-20T12:07:02.090 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/33' 2020-11-20T12:07:02.090 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/23' 2020-11-20T12:07:02.090 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/35' 2020-11-20T12:07:02.090 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/26' 2020-11-20T12:07:02.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './200/37' 2020-11-20T12:07:02.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './200' 2020-11-20T12:07:02.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/21' 2020-11-20T12:07:02.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/27' 2020-11-20T12:07:02.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/31' 2020-11-20T12:07:02.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/36' 2020-11-20T12:07:02.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/20' 2020-11-20T12:07:02.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/39' 2020-11-20T12:07:02.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/34' 2020-11-20T12:07:02.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/38' 2020-11-20T12:07:02.093 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/28' 2020-11-20T12:07:02.093 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/24' 2020-11-20T12:07:02.093 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/29' 2020-11-20T12:07:02.093 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/32' 2020-11-20T12:07:02.094 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/22' 2020-11-20T12:07:02.094 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/25' 2020-11-20T12:07:02.094 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/30' 2020-11-20T12:07:02.094 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/33' 2020-11-20T12:07:02.094 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/23' 2020-11-20T12:07:02.095 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/35' 2020-11-20T12:07:02.095 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/26' 2020-11-20T12:07:02.095 INFO:tasks.workunit.client.0.smithi044.stdout:removed './201/37' 2020-11-20T12:07:02.095 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './201' 2020-11-20T12:07:02.095 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/21' 2020-11-20T12:07:02.095 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/27' 2020-11-20T12:07:02.096 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/31' 2020-11-20T12:07:02.096 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/36' 2020-11-20T12:07:02.096 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/20' 2020-11-20T12:07:02.096 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/39' 2020-11-20T12:07:02.096 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/34' 2020-11-20T12:07:02.097 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/38' 2020-11-20T12:07:02.097 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/28' 2020-11-20T12:07:02.097 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/24' 2020-11-20T12:07:02.097 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/29' 2020-11-20T12:07:02.097 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/32' 2020-11-20T12:07:02.098 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/22' 2020-11-20T12:07:02.098 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/25' 2020-11-20T12:07:02.098 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/30' 2020-11-20T12:07:02.098 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/33' 2020-11-20T12:07:02.099 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/23' 2020-11-20T12:07:02.099 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/35' 2020-11-20T12:07:02.099 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/26' 2020-11-20T12:07:02.099 INFO:tasks.workunit.client.0.smithi044.stdout:removed './202/37' 2020-11-20T12:07:02.100 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './202' 2020-11-20T12:07:02.100 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/21' 2020-11-20T12:07:02.100 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/27' 2020-11-20T12:07:02.100 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/31' 2020-11-20T12:07:02.101 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/36' 2020-11-20T12:07:02.101 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/20' 2020-11-20T12:07:02.101 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/39' 2020-11-20T12:07:02.101 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/34' 2020-11-20T12:07:02.101 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/38' 2020-11-20T12:07:02.102 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/28' 2020-11-20T12:07:02.102 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/24' 2020-11-20T12:07:02.102 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/29' 2020-11-20T12:07:02.102 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/32' 2020-11-20T12:07:02.102 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/22' 2020-11-20T12:07:02.103 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/25' 2020-11-20T12:07:02.103 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/30' 2020-11-20T12:07:02.103 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/33' 2020-11-20T12:07:02.103 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/23' 2020-11-20T12:07:02.103 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/35' 2020-11-20T12:07:02.104 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/26' 2020-11-20T12:07:02.104 INFO:tasks.workunit.client.0.smithi044.stdout:removed './203/37' 2020-11-20T12:07:02.104 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './203' 2020-11-20T12:07:02.105 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/21' 2020-11-20T12:07:02.105 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/27' 2020-11-20T12:07:02.105 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/31' 2020-11-20T12:07:02.106 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/36' 2020-11-20T12:07:02.106 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/20' 2020-11-20T12:07:02.106 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/39' 2020-11-20T12:07:02.107 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/34' 2020-11-20T12:07:02.107 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/38' 2020-11-20T12:07:02.107 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/28' 2020-11-20T12:07:02.108 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/24' 2020-11-20T12:07:02.108 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/29' 2020-11-20T12:07:02.108 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/32' 2020-11-20T12:07:02.109 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/22' 2020-11-20T12:07:02.109 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/25' 2020-11-20T12:07:02.109 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/30' 2020-11-20T12:07:02.109 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/33' 2020-11-20T12:07:02.110 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/23' 2020-11-20T12:07:02.110 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/35' 2020-11-20T12:07:02.110 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/26' 2020-11-20T12:07:02.111 INFO:tasks.workunit.client.0.smithi044.stdout:removed './204/37' 2020-11-20T12:07:02.111 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './204' 2020-11-20T12:07:02.111 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/21' 2020-11-20T12:07:02.112 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/27' 2020-11-20T12:07:02.112 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/31' 2020-11-20T12:07:02.113 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/36' 2020-11-20T12:07:02.113 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/20' 2020-11-20T12:07:02.114 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/39' 2020-11-20T12:07:02.114 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/34' 2020-11-20T12:07:02.114 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/38' 2020-11-20T12:07:02.115 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/28' 2020-11-20T12:07:02.115 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/24' 2020-11-20T12:07:02.116 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/29' 2020-11-20T12:07:02.116 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/32' 2020-11-20T12:07:02.116 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/22' 2020-11-20T12:07:02.117 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/25' 2020-11-20T12:07:02.117 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/30' 2020-11-20T12:07:02.117 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/33' 2020-11-20T12:07:02.117 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/23' 2020-11-20T12:07:02.118 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/35' 2020-11-20T12:07:02.118 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/26' 2020-11-20T12:07:02.118 INFO:tasks.workunit.client.0.smithi044.stdout:removed './205/37' 2020-11-20T12:07:02.118 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './205' 2020-11-20T12:07:02.119 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/21' 2020-11-20T12:07:02.119 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/27' 2020-11-20T12:07:02.119 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/31' 2020-11-20T12:07:02.120 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/36' 2020-11-20T12:07:02.120 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/20' 2020-11-20T12:07:02.120 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/39' 2020-11-20T12:07:02.445 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:07:02.445 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:07:02.446 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:07:02.447 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:07:02.450 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:07:02.451 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:07:02.451 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.658+0000 7fc8b9e69700 1 Processor -- start 2020-11-20T12:07:02.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.658+0000 7fc8b9e69700 1 -- start start 2020-11-20T12:07:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b9e69700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc8b4060330 0x7fc8b40a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b9e69700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 0x7fc8b413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b9e69700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b413ae80 0x7fc8b4136f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:02.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b9e69700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40a52e0 con 0x7fc8b40a4c30 2020-11-20T12:07:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b9e69700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40a6260 con 0x7fc8b4060330 2020-11-20T12:07:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b9e69700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40a6490 con 0x7fc8b413ae80 2020-11-20T12:07:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc8b4060330 0x7fc8b40a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc8b4060330 0x7fc8b40a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58478/0 (socket says 172.21.15.44:58478) 2020-11-20T12:07:02.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b37fe700 1 -- 172.21.15.44:0/3139479059 learned_addr learned my addr 172.21.15.44:0/3139479059 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b3fff700 1 --2- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b413ae80 0x7fc8b4136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.659+0000 7fc8b2ffd700 1 --2- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 0x7fc8b413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b2ffd700 1 -- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 msgr2=0x7fc8b413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:07:02.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b2ffd700 1 -- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 msgr2=0x7fc8b413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b2ffd700 1 --2- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 0x7fc8b413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b37fe700 1 -- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b413ae80 msgr2=0x7fc8b4136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b37fe700 1 --2- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b413ae80 0x7fc8b4136f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b37fe700 1 -- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 msgr2=0x7fc8b413a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:02.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b2ffd700 1 --2- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 0x7fc8b413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b37fe700 1 --2- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 0x7fc8b413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b37fe700 1 -- 172.21.15.44:0/3139479059 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40a6730 con 0x7fc8b4060330 2020-11-20T12:07:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b37fe700 1 --2- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b4060330 0x7fc8b40a4740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc8b40adec0 tx=0x7fc8a4006eb0).ready entity=mon.1 client_cookie=e4130692a9e099ee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b27fc700 1 -- 172.21.15.44:0/3139479059 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4009810 con 0x7fc8b4060330 2020-11-20T12:07:02.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b27fc700 1 -- 172.21.15.44:0/3139479059 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a4009970 con 0x7fc8b4060330 2020-11-20T12:07:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.660+0000 7fc8b27fc700 1 -- 172.21.15.44:0/3139479059 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4009cf0 con 0x7fc8b4060330 2020-11-20T12:07:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 -- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b4060330 msgr2=0x7fc8b40a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b4060330 0x7fc8b40a4740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc8b40adec0 tx=0x7fc8a4006eb0).stop 2020-11-20T12:07:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 -- 172.21.15.44:0/3139479059 shutdown_connections 2020-11-20T12:07:02.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b413ae80 0x7fc8b4136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/3139479059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b4060330 0x7fc8b40a4740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/3139479059 >> v2:172.21.15.44:3300/0 conn(0x7fc8b40a4c30 0x7fc8b413a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 -- 172.21.15.44:0/3139479059 >> 172.21.15.44:0/3139479059 conn(0x7fc8b4062690 msgr2=0x7fc8b40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 -- 172.21.15.44:0/3139479059 shutdown_connections 2020-11-20T12:07:02.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.661+0000 7fc8b9e69700 1 -- 172.21.15.44:0/3139479059 wait complete. 2020-11-20T12:07:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.662+0000 7fc8b9e69700 1 Processor -- start 2020-11-20T12:07:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.662+0000 7fc8b9e69700 1 -- start start 2020-11-20T12:07:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b9e69700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 0x7fc8b4134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b9e69700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b40a4c30 0x7fc8b413d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:02.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b9e69700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b413ae80 0x7fc8b413c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b9e69700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4137680 con 0x7fc8b4060330 2020-11-20T12:07:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b9e69700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4137920 con 0x7fc8b413ae80 2020-11-20T12:07:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b9e69700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b413d540 con 0x7fc8b40a4c30 2020-11-20T12:07:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 0x7fc8b4134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:02.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 0x7fc8b4134ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38202/0 (socket says 172.21.15.44:38202) 2020-11-20T12:07:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b37fe700 1 -- 172.21.15.44:0/1360299394 learned_addr learned my addr 172.21.15.44:0/1360299394 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b3fff700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b413ae80 0x7fc8b413c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b37fe700 1 -- 172.21.15.44:0/1360299394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 msgr2=0x7fc8b4134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:07:02.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b37fe700 1 -- 172.21.15.44:0/1360299394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 msgr2=0x7fc8b4134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:07:02.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b2ffd700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b40a4c30 0x7fc8b413d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:02.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b37fe700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 0x7fc8b4134ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:02.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b37fe700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 0x7fc8b4134ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:02.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b2ffd700 1 -- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b413ae80 msgr2=0x7fc8b413c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:02.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b2ffd700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b413ae80 0x7fc8b413c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:02.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b2ffd700 1 -- 172.21.15.44:0/1360299394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 msgr2=0x7fc8b4134ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:02.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b2ffd700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 0x7fc8b4134ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:02.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b2ffd700 1 -- 172.21.15.44:0/1360299394 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4135570 con 0x7fc8b40a4c30 2020-11-20T12:07:02.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b3fff700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b413ae80 0x7fc8b413c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:02.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b2ffd700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b40a4c30 0x7fc8b413d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc8a8006c00 tx=0x7fc8a8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:02.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8937fe700 1 -- 172.21.15.44:0/1360299394 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800f5e0 con 0x7fc8b40a4c30 2020-11-20T12:07:02.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8937fe700 1 -- 172.21.15.44:0/1360299394 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a800f740 con 0x7fc8b40a4c30 2020-11-20T12:07:02.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4145b90 con 0x7fc8b40a4c30 2020-11-20T12:07:02.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.663+0000 7fc8937fe700 1 -- 172.21.15.44:0/1360299394 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800fae0 con 0x7fc8b40a4c30 2020-11-20T12:07:02.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.664+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b413e110 con 0x7fc8b40a4c30 2020-11-20T12:07:02.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.665+0000 7fc8937fe700 1 -- 172.21.15.44:0/1360299394 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc8a801c460 con 0x7fc8b40a4c30 2020-11-20T12:07:02.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.665+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b413d700 con 0x7fc8b40a4c30 2020-11-20T12:07:02.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.665+0000 7fc8937fe700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc88c056d20 0x7fc88c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:02.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.665+0000 7fc8937fe700 1 -- 172.21.15.44:0/1360299394 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc8a8070020 con 0x7fc8b40a4c30 2020-11-20T12:07:02.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.665+0000 7fc8b37fe700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc88c056d20 0x7fc88c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:02.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.666+0000 7fc8b37fe700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc88c056d20 0x7fc88c0591e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fc8a400b1b0 tx=0x7fc8a4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:02.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.669+0000 7fc8937fe700 1 -- 172.21.15.44:0/1360299394 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc8a8045290 con 0x7fc8b40a4c30 2020-11-20T12:07:02.876 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/34' 2020-11-20T12:07:02.877 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/38' 2020-11-20T12:07:02.877 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/28' 2020-11-20T12:07:02.877 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/24' 2020-11-20T12:07:02.877 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/29' 2020-11-20T12:07:02.878 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/32' 2020-11-20T12:07:02.878 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/22' 2020-11-20T12:07:02.878 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/25' 2020-11-20T12:07:02.878 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/30' 2020-11-20T12:07:02.878 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/33' 2020-11-20T12:07:02.879 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/23' 2020-11-20T12:07:02.879 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/35' 2020-11-20T12:07:02.879 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/26' 2020-11-20T12:07:02.879 INFO:tasks.workunit.client.0.smithi044.stdout:removed './206/37' 2020-11-20T12:07:02.879 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './206' 2020-11-20T12:07:02.880 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/21' 2020-11-20T12:07:02.880 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/27' 2020-11-20T12:07:02.880 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/31' 2020-11-20T12:07:02.880 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/36' 2020-11-20T12:07:02.881 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/20' 2020-11-20T12:07:02.881 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/39' 2020-11-20T12:07:02.881 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/34' 2020-11-20T12:07:02.881 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/38' 2020-11-20T12:07:02.881 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/28' 2020-11-20T12:07:02.882 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/24' 2020-11-20T12:07:02.882 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/29' 2020-11-20T12:07:02.882 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/32' 2020-11-20T12:07:02.882 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/22' 2020-11-20T12:07:02.883 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/25' 2020-11-20T12:07:02.883 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/30' 2020-11-20T12:07:02.883 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/33' 2020-11-20T12:07:02.883 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/23' 2020-11-20T12:07:02.884 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/35' 2020-11-20T12:07:02.884 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/26' 2020-11-20T12:07:02.884 INFO:tasks.workunit.client.0.smithi044.stdout:removed './207/37' 2020-11-20T12:07:02.885 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './207' 2020-11-20T12:07:02.885 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/21' 2020-11-20T12:07:02.885 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/27' 2020-11-20T12:07:02.885 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/31' 2020-11-20T12:07:02.886 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/36' 2020-11-20T12:07:02.886 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/20' 2020-11-20T12:07:02.886 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/39' 2020-11-20T12:07:02.886 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/34' 2020-11-20T12:07:02.886 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/38' 2020-11-20T12:07:02.887 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/28' 2020-11-20T12:07:02.887 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/24' 2020-11-20T12:07:02.887 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/29' 2020-11-20T12:07:02.887 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/32' 2020-11-20T12:07:02.887 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/22' 2020-11-20T12:07:02.888 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/25' 2020-11-20T12:07:02.888 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/30' 2020-11-20T12:07:02.888 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/33' 2020-11-20T12:07:02.888 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/23' 2020-11-20T12:07:02.888 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/35' 2020-11-20T12:07:02.889 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/26' 2020-11-20T12:07:02.889 INFO:tasks.workunit.client.0.smithi044.stdout:removed './208/37' 2020-11-20T12:07:02.889 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './208' 2020-11-20T12:07:02.889 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/21' 2020-11-20T12:07:02.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/27' 2020-11-20T12:07:02.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/31' 2020-11-20T12:07:02.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/36' 2020-11-20T12:07:02.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/20' 2020-11-20T12:07:02.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/39' 2020-11-20T12:07:02.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/34' 2020-11-20T12:07:02.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/38' 2020-11-20T12:07:02.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/28' 2020-11-20T12:07:02.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/24' 2020-11-20T12:07:02.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/29' 2020-11-20T12:07:02.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/32' 2020-11-20T12:07:02.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/22' 2020-11-20T12:07:02.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/25' 2020-11-20T12:07:02.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/30' 2020-11-20T12:07:02.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/33' 2020-11-20T12:07:02.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/23' 2020-11-20T12:07:02.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/35' 2020-11-20T12:07:02.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/26' 2020-11-20T12:07:02.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './209/37' 2020-11-20T12:07:02.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './209' 2020-11-20T12:07:02.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/21' 2020-11-20T12:07:02.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/27' 2020-11-20T12:07:02.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/31' 2020-11-20T12:07:02.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/36' 2020-11-20T12:07:02.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/20' 2020-11-20T12:07:02.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/39' 2020-11-20T12:07:02.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/34' 2020-11-20T12:07:02.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/38' 2020-11-20T12:07:02.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/28' 2020-11-20T12:07:02.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/24' 2020-11-20T12:07:02.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/29' 2020-11-20T12:07:02.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/32' 2020-11-20T12:07:02.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/22' 2020-11-20T12:07:02.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/25' 2020-11-20T12:07:02.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/30' 2020-11-20T12:07:02.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/33' 2020-11-20T12:07:02.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/23' 2020-11-20T12:07:02.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/35' 2020-11-20T12:07:02.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/26' 2020-11-20T12:07:02.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './210/37' 2020-11-20T12:07:02.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './210' 2020-11-20T12:07:02.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/21' 2020-11-20T12:07:02.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/27' 2020-11-20T12:07:02.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/31' 2020-11-20T12:07:02.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/36' 2020-11-20T12:07:02.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/20' 2020-11-20T12:07:02.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/39' 2020-11-20T12:07:02.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/34' 2020-11-20T12:07:02.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/38' 2020-11-20T12:07:02.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/28' 2020-11-20T12:07:02.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/24' 2020-11-20T12:07:02.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/29' 2020-11-20T12:07:02.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/32' 2020-11-20T12:07:02.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/22' 2020-11-20T12:07:02.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/25' 2020-11-20T12:07:02.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/30' 2020-11-20T12:07:02.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/33' 2020-11-20T12:07:02.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/23' 2020-11-20T12:07:02.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/35' 2020-11-20T12:07:02.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/26' 2020-11-20T12:07:02.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './211/37' 2020-11-20T12:07:02.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './211' 2020-11-20T12:07:02.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/21' 2020-11-20T12:07:02.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/27' 2020-11-20T12:07:02.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/31' 2020-11-20T12:07:02.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/36' 2020-11-20T12:07:02.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/20' 2020-11-20T12:07:02.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/39' 2020-11-20T12:07:02.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/34' 2020-11-20T12:07:02.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/38' 2020-11-20T12:07:02.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/28' 2020-11-20T12:07:02.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/24' 2020-11-20T12:07:02.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/29' 2020-11-20T12:07:02.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/32' 2020-11-20T12:07:02.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/22' 2020-11-20T12:07:02.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/25' 2020-11-20T12:07:02.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/30' 2020-11-20T12:07:02.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/33' 2020-11-20T12:07:02.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/23' 2020-11-20T12:07:02.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/35' 2020-11-20T12:07:02.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/26' 2020-11-20T12:07:02.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './212/37' 2020-11-20T12:07:02.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './212' 2020-11-20T12:07:02.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/21' 2020-11-20T12:07:02.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/27' 2020-11-20T12:07:02.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/31' 2020-11-20T12:07:02.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/36' 2020-11-20T12:07:02.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/20' 2020-11-20T12:07:02.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/39' 2020-11-20T12:07:02.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/34' 2020-11-20T12:07:02.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/38' 2020-11-20T12:07:02.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/28' 2020-11-20T12:07:02.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/24' 2020-11-20T12:07:02.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/29' 2020-11-20T12:07:02.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/32' 2020-11-20T12:07:02.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/22' 2020-11-20T12:07:02.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/25' 2020-11-20T12:07:02.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/30' 2020-11-20T12:07:02.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/33' 2020-11-20T12:07:02.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/23' 2020-11-20T12:07:02.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/35' 2020-11-20T12:07:02.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/26' 2020-11-20T12:07:02.913 INFO:tasks.workunit.client.0.smithi044.stdout:removed './213/37' 2020-11-20T12:07:02.913 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './213' 2020-11-20T12:07:02.913 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/21' 2020-11-20T12:07:02.913 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/27' 2020-11-20T12:07:02.913 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/31' 2020-11-20T12:07:02.914 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/36' 2020-11-20T12:07:02.914 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/20' 2020-11-20T12:07:02.914 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/39' 2020-11-20T12:07:02.914 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/34' 2020-11-20T12:07:02.914 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/38' 2020-11-20T12:07:02.915 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/28' 2020-11-20T12:07:02.915 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/24' 2020-11-20T12:07:02.915 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/29' 2020-11-20T12:07:02.915 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/32' 2020-11-20T12:07:02.915 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/22' 2020-11-20T12:07:02.916 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/25' 2020-11-20T12:07:02.916 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/30' 2020-11-20T12:07:02.916 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/33' 2020-11-20T12:07:02.916 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/23' 2020-11-20T12:07:02.916 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/35' 2020-11-20T12:07:02.917 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/26' 2020-11-20T12:07:02.917 INFO:tasks.workunit.client.0.smithi044.stdout:removed './214/37' 2020-11-20T12:07:02.917 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './214' 2020-11-20T12:07:02.917 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/21' 2020-11-20T12:07:02.917 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/27' 2020-11-20T12:07:02.918 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/31' 2020-11-20T12:07:02.918 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/36' 2020-11-20T12:07:02.918 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/20' 2020-11-20T12:07:02.918 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/39' 2020-11-20T12:07:02.918 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/34' 2020-11-20T12:07:02.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/38' 2020-11-20T12:07:02.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/28' 2020-11-20T12:07:02.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/24' 2020-11-20T12:07:02.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/29' 2020-11-20T12:07:02.919 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/32' 2020-11-20T12:07:02.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/22' 2020-11-20T12:07:02.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/25' 2020-11-20T12:07:02.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/30' 2020-11-20T12:07:02.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/33' 2020-11-20T12:07:02.920 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/23' 2020-11-20T12:07:02.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/35' 2020-11-20T12:07:02.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/26' 2020-11-20T12:07:02.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed './215/37' 2020-11-20T12:07:02.921 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './215' 2020-11-20T12:07:02.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/21' 2020-11-20T12:07:02.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/27' 2020-11-20T12:07:02.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/31' 2020-11-20T12:07:02.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/36' 2020-11-20T12:07:02.922 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/20' 2020-11-20T12:07:02.923 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/39' 2020-11-20T12:07:02.923 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/34' 2020-11-20T12:07:02.923 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/38' 2020-11-20T12:07:02.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:02.928+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc8b4061650 con 0x7fc8b40a4c30 2020-11-20T12:07:03.693 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/28' 2020-11-20T12:07:03.693 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/24' 2020-11-20T12:07:03.694 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/29' 2020-11-20T12:07:03.694 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/32' 2020-11-20T12:07:03.694 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/22' 2020-11-20T12:07:03.694 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/25' 2020-11-20T12:07:03.695 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/30' 2020-11-20T12:07:03.695 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/33' 2020-11-20T12:07:03.695 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/23' 2020-11-20T12:07:03.695 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/35' 2020-11-20T12:07:03.695 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/26' 2020-11-20T12:07:03.696 INFO:tasks.workunit.client.0.smithi044.stdout:removed './216/37' 2020-11-20T12:07:03.696 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './216' 2020-11-20T12:07:03.696 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/21' 2020-11-20T12:07:03.696 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/27' 2020-11-20T12:07:03.696 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/31' 2020-11-20T12:07:03.697 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/36' 2020-11-20T12:07:03.697 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/20' 2020-11-20T12:07:03.697 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/39' 2020-11-20T12:07:03.697 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/34' 2020-11-20T12:07:03.697 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/38' 2020-11-20T12:07:03.698 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/28' 2020-11-20T12:07:03.698 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/24' 2020-11-20T12:07:03.698 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/29' 2020-11-20T12:07:03.698 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/32' 2020-11-20T12:07:03.698 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/22' 2020-11-20T12:07:03.699 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/25' 2020-11-20T12:07:03.699 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/30' 2020-11-20T12:07:03.699 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/33' 2020-11-20T12:07:03.699 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/23' 2020-11-20T12:07:03.699 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/35' 2020-11-20T12:07:03.700 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/26' 2020-11-20T12:07:03.700 INFO:tasks.workunit.client.0.smithi044.stdout:removed './217/37' 2020-11-20T12:07:03.700 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './217' 2020-11-20T12:07:03.700 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/21' 2020-11-20T12:07:03.700 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/27' 2020-11-20T12:07:03.701 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/31' 2020-11-20T12:07:03.701 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/36' 2020-11-20T12:07:03.701 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/20' 2020-11-20T12:07:03.701 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/39' 2020-11-20T12:07:03.701 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/34' 2020-11-20T12:07:03.702 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/38' 2020-11-20T12:07:03.702 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/28' 2020-11-20T12:07:03.702 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/24' 2020-11-20T12:07:03.702 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/29' 2020-11-20T12:07:03.702 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/32' 2020-11-20T12:07:03.703 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/22' 2020-11-20T12:07:03.703 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/25' 2020-11-20T12:07:03.703 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/30' 2020-11-20T12:07:03.703 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/33' 2020-11-20T12:07:03.703 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/23' 2020-11-20T12:07:03.704 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/35' 2020-11-20T12:07:03.704 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/26' 2020-11-20T12:07:03.704 INFO:tasks.workunit.client.0.smithi044.stdout:removed './218/37' 2020-11-20T12:07:03.704 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './218' 2020-11-20T12:07:03.704 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/21' 2020-11-20T12:07:03.705 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/27' 2020-11-20T12:07:03.705 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/31' 2020-11-20T12:07:03.705 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/36' 2020-11-20T12:07:03.705 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/20' 2020-11-20T12:07:03.705 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/39' 2020-11-20T12:07:03.706 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/34' 2020-11-20T12:07:03.706 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/38' 2020-11-20T12:07:03.706 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/28' 2020-11-20T12:07:03.706 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/24' 2020-11-20T12:07:03.706 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/29' 2020-11-20T12:07:03.707 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/32' 2020-11-20T12:07:03.707 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/22' 2020-11-20T12:07:03.707 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/25' 2020-11-20T12:07:03.707 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/30' 2020-11-20T12:07:03.707 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/33' 2020-11-20T12:07:03.708 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/23' 2020-11-20T12:07:03.708 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/35' 2020-11-20T12:07:03.708 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/26' 2020-11-20T12:07:03.708 INFO:tasks.workunit.client.0.smithi044.stdout:removed './219/37' 2020-11-20T12:07:03.708 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './219' 2020-11-20T12:07:03.709 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/21' 2020-11-20T12:07:03.709 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/27' 2020-11-20T12:07:03.709 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/31' 2020-11-20T12:07:03.709 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/36' 2020-11-20T12:07:03.709 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/20' 2020-11-20T12:07:03.710 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/39' 2020-11-20T12:07:03.710 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/34' 2020-11-20T12:07:03.710 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/38' 2020-11-20T12:07:03.710 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/28' 2020-11-20T12:07:03.710 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/24' 2020-11-20T12:07:03.711 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/29' 2020-11-20T12:07:03.711 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/32' 2020-11-20T12:07:03.711 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/22' 2020-11-20T12:07:03.711 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/25' 2020-11-20T12:07:03.712 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/30' 2020-11-20T12:07:03.712 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/33' 2020-11-20T12:07:03.712 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/23' 2020-11-20T12:07:03.712 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/35' 2020-11-20T12:07:03.712 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/26' 2020-11-20T12:07:03.713 INFO:tasks.workunit.client.0.smithi044.stdout:removed './220/37' 2020-11-20T12:07:03.713 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './220' 2020-11-20T12:07:03.713 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/21' 2020-11-20T12:07:03.713 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/27' 2020-11-20T12:07:03.713 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/31' 2020-11-20T12:07:03.714 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/36' 2020-11-20T12:07:03.714 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/20' 2020-11-20T12:07:03.714 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/39' 2020-11-20T12:07:03.714 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/34' 2020-11-20T12:07:03.714 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/38' 2020-11-20T12:07:03.715 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/28' 2020-11-20T12:07:03.715 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/24' 2020-11-20T12:07:03.715 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/29' 2020-11-20T12:07:03.715 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/32' 2020-11-20T12:07:03.715 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/22' 2020-11-20T12:07:03.716 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/25' 2020-11-20T12:07:03.716 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/30' 2020-11-20T12:07:03.716 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/33' 2020-11-20T12:07:03.716 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/23' 2020-11-20T12:07:03.716 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/35' 2020-11-20T12:07:03.716 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/26' 2020-11-20T12:07:03.717 INFO:tasks.workunit.client.0.smithi044.stdout:removed './221/37' 2020-11-20T12:07:03.717 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './221' 2020-11-20T12:07:03.717 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/21' 2020-11-20T12:07:03.717 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/27' 2020-11-20T12:07:03.717 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/31' 2020-11-20T12:07:03.718 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/36' 2020-11-20T12:07:03.718 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/20' 2020-11-20T12:07:03.718 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/39' 2020-11-20T12:07:03.718 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/34' 2020-11-20T12:07:03.719 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/38' 2020-11-20T12:07:03.719 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/28' 2020-11-20T12:07:03.719 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/24' 2020-11-20T12:07:03.719 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/29' 2020-11-20T12:07:03.719 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/32' 2020-11-20T12:07:03.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/22' 2020-11-20T12:07:03.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/25' 2020-11-20T12:07:03.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/30' 2020-11-20T12:07:03.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/33' 2020-11-20T12:07:03.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/23' 2020-11-20T12:07:03.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/35' 2020-11-20T12:07:03.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/26' 2020-11-20T12:07:03.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './222/37' 2020-11-20T12:07:03.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './222' 2020-11-20T12:07:03.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/21' 2020-11-20T12:07:03.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/27' 2020-11-20T12:07:03.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/31' 2020-11-20T12:07:03.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/36' 2020-11-20T12:07:03.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/20' 2020-11-20T12:07:03.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/39' 2020-11-20T12:07:03.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/34' 2020-11-20T12:07:03.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/38' 2020-11-20T12:07:03.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/28' 2020-11-20T12:07:03.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/24' 2020-11-20T12:07:03.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/29' 2020-11-20T12:07:03.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/32' 2020-11-20T12:07:03.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/22' 2020-11-20T12:07:03.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/25' 2020-11-20T12:07:03.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/30' 2020-11-20T12:07:03.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/33' 2020-11-20T12:07:03.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/23' 2020-11-20T12:07:03.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/35' 2020-11-20T12:07:03.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/26' 2020-11-20T12:07:03.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './223/37' 2020-11-20T12:07:03.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './223' 2020-11-20T12:07:03.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/21' 2020-11-20T12:07:03.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/27' 2020-11-20T12:07:03.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/31' 2020-11-20T12:07:03.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/36' 2020-11-20T12:07:03.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/20' 2020-11-20T12:07:03.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/39' 2020-11-20T12:07:03.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/34' 2020-11-20T12:07:03.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/38' 2020-11-20T12:07:03.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/28' 2020-11-20T12:07:03.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/24' 2020-11-20T12:07:03.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/29' 2020-11-20T12:07:03.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/32' 2020-11-20T12:07:03.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/22' 2020-11-20T12:07:03.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/25' 2020-11-20T12:07:03.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/30' 2020-11-20T12:07:03.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/33' 2020-11-20T12:07:03.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/23' 2020-11-20T12:07:03.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/35' 2020-11-20T12:07:03.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/26' 2020-11-20T12:07:03.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './224/37' 2020-11-20T12:07:03.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './224' 2020-11-20T12:07:03.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/21' 2020-11-20T12:07:03.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/27' 2020-11-20T12:07:03.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/31' 2020-11-20T12:07:03.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/36' 2020-11-20T12:07:03.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/20' 2020-11-20T12:07:03.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/39' 2020-11-20T12:07:03.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/34' 2020-11-20T12:07:03.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/38' 2020-11-20T12:07:03.732 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/28' 2020-11-20T12:07:03.732 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/24' 2020-11-20T12:07:03.732 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/29' 2020-11-20T12:07:03.732 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/32' 2020-11-20T12:07:03.732 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/22' 2020-11-20T12:07:03.733 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/25' 2020-11-20T12:07:03.733 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/30' 2020-11-20T12:07:03.733 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/33' 2020-11-20T12:07:03.733 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/23' 2020-11-20T12:07:03.733 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/35' 2020-11-20T12:07:03.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/26' 2020-11-20T12:07:03.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed './225/37' 2020-11-20T12:07:03.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './225' 2020-11-20T12:07:03.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/21' 2020-11-20T12:07:03.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/27' 2020-11-20T12:07:03.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/31' 2020-11-20T12:07:03.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/36' 2020-11-20T12:07:03.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/20' 2020-11-20T12:07:03.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/39' 2020-11-20T12:07:03.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/34' 2020-11-20T12:07:03.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/38' 2020-11-20T12:07:03.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/28' 2020-11-20T12:07:03.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/24' 2020-11-20T12:07:04.432 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/29' 2020-11-20T12:07:04.432 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/32' 2020-11-20T12:07:04.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/22' 2020-11-20T12:07:04.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/25' 2020-11-20T12:07:04.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/30' 2020-11-20T12:07:04.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/33' 2020-11-20T12:07:04.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/23' 2020-11-20T12:07:04.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/35' 2020-11-20T12:07:04.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/26' 2020-11-20T12:07:04.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './226/37' 2020-11-20T12:07:04.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './226' 2020-11-20T12:07:04.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/21' 2020-11-20T12:07:04.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/27' 2020-11-20T12:07:04.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/31' 2020-11-20T12:07:04.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/36' 2020-11-20T12:07:04.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/20' 2020-11-20T12:07:04.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/39' 2020-11-20T12:07:04.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/34' 2020-11-20T12:07:04.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/38' 2020-11-20T12:07:04.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/28' 2020-11-20T12:07:04.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/24' 2020-11-20T12:07:04.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/29' 2020-11-20T12:07:04.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/32' 2020-11-20T12:07:04.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/22' 2020-11-20T12:07:04.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/25' 2020-11-20T12:07:04.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/30' 2020-11-20T12:07:04.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/33' 2020-11-20T12:07:04.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/23' 2020-11-20T12:07:04.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/35' 2020-11-20T12:07:04.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/26' 2020-11-20T12:07:04.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './227/37' 2020-11-20T12:07:04.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './227' 2020-11-20T12:07:04.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/21' 2020-11-20T12:07:04.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/27' 2020-11-20T12:07:04.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/31' 2020-11-20T12:07:04.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/36' 2020-11-20T12:07:04.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/20' 2020-11-20T12:07:04.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/39' 2020-11-20T12:07:04.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/34' 2020-11-20T12:07:04.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/38' 2020-11-20T12:07:04.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/28' 2020-11-20T12:07:04.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/24' 2020-11-20T12:07:04.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/29' 2020-11-20T12:07:04.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/32' 2020-11-20T12:07:04.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/22' 2020-11-20T12:07:04.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/25' 2020-11-20T12:07:04.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/30' 2020-11-20T12:07:04.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/33' 2020-11-20T12:07:04.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/23' 2020-11-20T12:07:04.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/35' 2020-11-20T12:07:04.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/26' 2020-11-20T12:07:04.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './228/37' 2020-11-20T12:07:04.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './228' 2020-11-20T12:07:04.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/21' 2020-11-20T12:07:04.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/27' 2020-11-20T12:07:04.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/31' 2020-11-20T12:07:04.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/36' 2020-11-20T12:07:04.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/20' 2020-11-20T12:07:04.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/39' 2020-11-20T12:07:04.470 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/34' 2020-11-20T12:07:04.470 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/38' 2020-11-20T12:07:04.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/28' 2020-11-20T12:07:04.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/24' 2020-11-20T12:07:04.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/29' 2020-11-20T12:07:04.471 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/32' 2020-11-20T12:07:04.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/22' 2020-11-20T12:07:04.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/25' 2020-11-20T12:07:04.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/30' 2020-11-20T12:07:04.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/33' 2020-11-20T12:07:04.472 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/23' 2020-11-20T12:07:04.473 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/35' 2020-11-20T12:07:04.473 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/26' 2020-11-20T12:07:04.473 INFO:tasks.workunit.client.0.smithi044.stdout:removed './229/37' 2020-11-20T12:07:04.474 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './229' 2020-11-20T12:07:04.474 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/21' 2020-11-20T12:07:04.474 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/27' 2020-11-20T12:07:04.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/31' 2020-11-20T12:07:04.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/36' 2020-11-20T12:07:04.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/20' 2020-11-20T12:07:04.475 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/39' 2020-11-20T12:07:04.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/34' 2020-11-20T12:07:04.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/38' 2020-11-20T12:07:04.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/28' 2020-11-20T12:07:04.476 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/24' 2020-11-20T12:07:04.477 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/29' 2020-11-20T12:07:04.477 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/32' 2020-11-20T12:07:04.477 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/22' 2020-11-20T12:07:04.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/25' 2020-11-20T12:07:04.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/30' 2020-11-20T12:07:04.478 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/33' 2020-11-20T12:07:04.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/23' 2020-11-20T12:07:04.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/35' 2020-11-20T12:07:04.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/26' 2020-11-20T12:07:04.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './230/37' 2020-11-20T12:07:04.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './230' 2020-11-20T12:07:04.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/21' 2020-11-20T12:07:04.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/27' 2020-11-20T12:07:04.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/31' 2020-11-20T12:07:04.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/36' 2020-11-20T12:07:04.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/20' 2020-11-20T12:07:04.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/39' 2020-11-20T12:07:04.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/34' 2020-11-20T12:07:04.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/38' 2020-11-20T12:07:04.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/28' 2020-11-20T12:07:04.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/24' 2020-11-20T12:07:04.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/29' 2020-11-20T12:07:04.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/32' 2020-11-20T12:07:04.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/22' 2020-11-20T12:07:04.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/25' 2020-11-20T12:07:04.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/30' 2020-11-20T12:07:04.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/33' 2020-11-20T12:07:04.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/23' 2020-11-20T12:07:04.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/35' 2020-11-20T12:07:04.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/26' 2020-11-20T12:07:04.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './231/37' 2020-11-20T12:07:04.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './231' 2020-11-20T12:07:04.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/21' 2020-11-20T12:07:04.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/27' 2020-11-20T12:07:04.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/31' 2020-11-20T12:07:04.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/36' 2020-11-20T12:07:04.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/20' 2020-11-20T12:07:04.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/39' 2020-11-20T12:07:04.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/34' 2020-11-20T12:07:04.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/38' 2020-11-20T12:07:04.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/28' 2020-11-20T12:07:04.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/24' 2020-11-20T12:07:04.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/29' 2020-11-20T12:07:04.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/32' 2020-11-20T12:07:04.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/22' 2020-11-20T12:07:04.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/25' 2020-11-20T12:07:04.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/30' 2020-11-20T12:07:04.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/33' 2020-11-20T12:07:04.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/23' 2020-11-20T12:07:04.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/35' 2020-11-20T12:07:04.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/26' 2020-11-20T12:07:04.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './232/37' 2020-11-20T12:07:04.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './232' 2020-11-20T12:07:04.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/21' 2020-11-20T12:07:04.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/27' 2020-11-20T12:07:04.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/31' 2020-11-20T12:07:04.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/36' 2020-11-20T12:07:04.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/20' 2020-11-20T12:07:04.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/39' 2020-11-20T12:07:04.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/34' 2020-11-20T12:07:04.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/38' 2020-11-20T12:07:04.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/28' 2020-11-20T12:07:04.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/24' 2020-11-20T12:07:04.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/29' 2020-11-20T12:07:04.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/32' 2020-11-20T12:07:04.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/22' 2020-11-20T12:07:04.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/25' 2020-11-20T12:07:04.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/30' 2020-11-20T12:07:04.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/33' 2020-11-20T12:07:04.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/23' 2020-11-20T12:07:04.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/35' 2020-11-20T12:07:04.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/26' 2020-11-20T12:07:04.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './233/37' 2020-11-20T12:07:04.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './233' 2020-11-20T12:07:04.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/21' 2020-11-20T12:07:04.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/27' 2020-11-20T12:07:04.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/31' 2020-11-20T12:07:04.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/36' 2020-11-20T12:07:04.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/20' 2020-11-20T12:07:04.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/39' 2020-11-20T12:07:04.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/34' 2020-11-20T12:07:04.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/38' 2020-11-20T12:07:04.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/28' 2020-11-20T12:07:04.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/24' 2020-11-20T12:07:04.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/29' 2020-11-20T12:07:04.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/32' 2020-11-20T12:07:04.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/22' 2020-11-20T12:07:04.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/25' 2020-11-20T12:07:04.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/30' 2020-11-20T12:07:04.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/33' 2020-11-20T12:07:04.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/23' 2020-11-20T12:07:04.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/35' 2020-11-20T12:07:04.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/26' 2020-11-20T12:07:04.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './234/37' 2020-11-20T12:07:04.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './234' 2020-11-20T12:07:04.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/21' 2020-11-20T12:07:04.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/27' 2020-11-20T12:07:04.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/31' 2020-11-20T12:07:04.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/36' 2020-11-20T12:07:04.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/20' 2020-11-20T12:07:04.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/39' 2020-11-20T12:07:04.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/34' 2020-11-20T12:07:04.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/38' 2020-11-20T12:07:04.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/28' 2020-11-20T12:07:04.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/24' 2020-11-20T12:07:04.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/29' 2020-11-20T12:07:04.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/32' 2020-11-20T12:07:04.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/22' 2020-11-20T12:07:04.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/25' 2020-11-20T12:07:04.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/30' 2020-11-20T12:07:04.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/33' 2020-11-20T12:07:04.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/23' 2020-11-20T12:07:04.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/35' 2020-11-20T12:07:04.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/26' 2020-11-20T12:07:04.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './235/37' 2020-11-20T12:07:04.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './235' 2020-11-20T12:07:04.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/21' 2020-11-20T12:07:04.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/27' 2020-11-20T12:07:04.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/31' 2020-11-20T12:07:04.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/36' 2020-11-20T12:07:04.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/20' 2020-11-20T12:07:04.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/39' 2020-11-20T12:07:04.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/34' 2020-11-20T12:07:04.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/38' 2020-11-20T12:07:04.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/28' 2020-11-20T12:07:04.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/24' 2020-11-20T12:07:04.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/29' 2020-11-20T12:07:04.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/32' 2020-11-20T12:07:05.122 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/22' 2020-11-20T12:07:05.123 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/25' 2020-11-20T12:07:05.123 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/30' 2020-11-20T12:07:05.124 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/33' 2020-11-20T12:07:05.124 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/23' 2020-11-20T12:07:05.124 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/35' 2020-11-20T12:07:05.125 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/26' 2020-11-20T12:07:05.125 INFO:tasks.workunit.client.0.smithi044.stdout:removed './236/37' 2020-11-20T12:07:05.125 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './236' 2020-11-20T12:07:05.126 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/21' 2020-11-20T12:07:05.126 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/27' 2020-11-20T12:07:05.127 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/31' 2020-11-20T12:07:05.127 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/36' 2020-11-20T12:07:05.127 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/20' 2020-11-20T12:07:05.128 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/39' 2020-11-20T12:07:05.128 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/34' 2020-11-20T12:07:05.128 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/38' 2020-11-20T12:07:05.130 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/28' 2020-11-20T12:07:05.130 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/24' 2020-11-20T12:07:05.131 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/29' 2020-11-20T12:07:05.131 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/32' 2020-11-20T12:07:05.132 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/22' 2020-11-20T12:07:05.132 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/25' 2020-11-20T12:07:05.132 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/30' 2020-11-20T12:07:05.133 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/33' 2020-11-20T12:07:05.133 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/23' 2020-11-20T12:07:05.134 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/35' 2020-11-20T12:07:05.134 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/26' 2020-11-20T12:07:05.134 INFO:tasks.workunit.client.0.smithi044.stdout:removed './237/37' 2020-11-20T12:07:05.135 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './237' 2020-11-20T12:07:05.135 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/21' 2020-11-20T12:07:05.135 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/27' 2020-11-20T12:07:05.136 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/31' 2020-11-20T12:07:05.136 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/36' 2020-11-20T12:07:05.136 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/20' 2020-11-20T12:07:05.136 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/39' 2020-11-20T12:07:05.137 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/34' 2020-11-20T12:07:05.137 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/38' 2020-11-20T12:07:05.137 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/28' 2020-11-20T12:07:05.138 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/24' 2020-11-20T12:07:05.138 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/29' 2020-11-20T12:07:05.138 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/32' 2020-11-20T12:07:05.138 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/22' 2020-11-20T12:07:05.138 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/25' 2020-11-20T12:07:05.139 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/30' 2020-11-20T12:07:05.139 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/33' 2020-11-20T12:07:05.139 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/23' 2020-11-20T12:07:05.139 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/35' 2020-11-20T12:07:05.139 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/26' 2020-11-20T12:07:05.140 INFO:tasks.workunit.client.0.smithi044.stdout:removed './238/37' 2020-11-20T12:07:05.140 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './238' 2020-11-20T12:07:05.140 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/21' 2020-11-20T12:07:05.140 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/27' 2020-11-20T12:07:05.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/31' 2020-11-20T12:07:05.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/36' 2020-11-20T12:07:05.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/20' 2020-11-20T12:07:05.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/39' 2020-11-20T12:07:05.141 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/34' 2020-11-20T12:07:05.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/38' 2020-11-20T12:07:05.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/28' 2020-11-20T12:07:05.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/24' 2020-11-20T12:07:05.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/29' 2020-11-20T12:07:05.142 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/32' 2020-11-20T12:07:05.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/22' 2020-11-20T12:07:05.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/25' 2020-11-20T12:07:05.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/30' 2020-11-20T12:07:05.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/33' 2020-11-20T12:07:05.143 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/23' 2020-11-20T12:07:05.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/35' 2020-11-20T12:07:05.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/26' 2020-11-20T12:07:05.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './239/37' 2020-11-20T12:07:05.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './239' 2020-11-20T12:07:05.144 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/21' 2020-11-20T12:07:05.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/27' 2020-11-20T12:07:05.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/31' 2020-11-20T12:07:05.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/36' 2020-11-20T12:07:05.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/20' 2020-11-20T12:07:05.145 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/39' 2020-11-20T12:07:05.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/34' 2020-11-20T12:07:05.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/38' 2020-11-20T12:07:05.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/28' 2020-11-20T12:07:05.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/24' 2020-11-20T12:07:05.146 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/29' 2020-11-20T12:07:05.147 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/32' 2020-11-20T12:07:05.147 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/22' 2020-11-20T12:07:05.147 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/25' 2020-11-20T12:07:05.147 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/30' 2020-11-20T12:07:05.147 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/33' 2020-11-20T12:07:05.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/23' 2020-11-20T12:07:05.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/35' 2020-11-20T12:07:05.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/26' 2020-11-20T12:07:05.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './240/37' 2020-11-20T12:07:05.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './240' 2020-11-20T12:07:05.148 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/21' 2020-11-20T12:07:05.149 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/27' 2020-11-20T12:07:05.149 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/31' 2020-11-20T12:07:05.149 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/36' 2020-11-20T12:07:05.149 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/20' 2020-11-20T12:07:05.150 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/39' 2020-11-20T12:07:05.150 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/34' 2020-11-20T12:07:05.150 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/38' 2020-11-20T12:07:05.150 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/28' 2020-11-20T12:07:05.150 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/24' 2020-11-20T12:07:05.150 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/29' 2020-11-20T12:07:05.151 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/32' 2020-11-20T12:07:05.151 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/22' 2020-11-20T12:07:05.151 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/25' 2020-11-20T12:07:05.151 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/30' 2020-11-20T12:07:05.151 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/33' 2020-11-20T12:07:05.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/23' 2020-11-20T12:07:05.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/35' 2020-11-20T12:07:05.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/26' 2020-11-20T12:07:05.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed './241/37' 2020-11-20T12:07:05.152 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './241' 2020-11-20T12:07:05.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/21' 2020-11-20T12:07:05.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/27' 2020-11-20T12:07:05.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/31' 2020-11-20T12:07:05.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/36' 2020-11-20T12:07:05.153 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/20' 2020-11-20T12:07:05.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/39' 2020-11-20T12:07:05.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/34' 2020-11-20T12:07:05.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/38' 2020-11-20T12:07:05.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/28' 2020-11-20T12:07:05.154 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/24' 2020-11-20T12:07:05.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/29' 2020-11-20T12:07:05.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/32' 2020-11-20T12:07:05.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/22' 2020-11-20T12:07:05.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/25' 2020-11-20T12:07:05.155 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/30' 2020-11-20T12:07:05.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/33' 2020-11-20T12:07:05.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/23' 2020-11-20T12:07:05.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/35' 2020-11-20T12:07:05.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/26' 2020-11-20T12:07:05.156 INFO:tasks.workunit.client.0.smithi044.stdout:removed './242/37' 2020-11-20T12:07:05.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './242' 2020-11-20T12:07:05.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/21' 2020-11-20T12:07:05.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/27' 2020-11-20T12:07:05.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/31' 2020-11-20T12:07:05.157 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/36' 2020-11-20T12:07:05.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/20' 2020-11-20T12:07:05.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/39' 2020-11-20T12:07:05.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/34' 2020-11-20T12:07:05.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/38' 2020-11-20T12:07:05.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/28' 2020-11-20T12:07:05.158 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/24' 2020-11-20T12:07:05.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/29' 2020-11-20T12:07:05.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/32' 2020-11-20T12:07:05.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/22' 2020-11-20T12:07:05.159 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/25' 2020-11-20T12:07:05.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/30' 2020-11-20T12:07:05.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/33' 2020-11-20T12:07:05.160 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/23' 2020-11-20T12:07:05.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/35' 2020-11-20T12:07:05.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/26' 2020-11-20T12:07:05.161 INFO:tasks.workunit.client.0.smithi044.stdout:removed './243/37' 2020-11-20T12:07:05.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './243' 2020-11-20T12:07:05.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/21' 2020-11-20T12:07:05.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/27' 2020-11-20T12:07:05.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/31' 2020-11-20T12:07:05.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/36' 2020-11-20T12:07:05.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/20' 2020-11-20T12:07:05.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/39' 2020-11-20T12:07:05.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/34' 2020-11-20T12:07:05.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/38' 2020-11-20T12:07:05.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/28' 2020-11-20T12:07:05.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/24' 2020-11-20T12:07:05.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/29' 2020-11-20T12:07:05.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/32' 2020-11-20T12:07:05.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/22' 2020-11-20T12:07:05.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/25' 2020-11-20T12:07:05.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/30' 2020-11-20T12:07:05.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/33' 2020-11-20T12:07:05.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/23' 2020-11-20T12:07:05.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/35' 2020-11-20T12:07:05.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/26' 2020-11-20T12:07:05.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './244/37' 2020-11-20T12:07:05.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './244' 2020-11-20T12:07:05.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/21' 2020-11-20T12:07:05.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/27' 2020-11-20T12:07:05.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/31' 2020-11-20T12:07:05.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/36' 2020-11-20T12:07:05.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/20' 2020-11-20T12:07:05.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/39' 2020-11-20T12:07:05.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/34' 2020-11-20T12:07:05.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/38' 2020-11-20T12:07:05.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/28' 2020-11-20T12:07:05.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/24' 2020-11-20T12:07:05.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/29' 2020-11-20T12:07:05.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/32' 2020-11-20T12:07:05.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/22' 2020-11-20T12:07:05.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/25' 2020-11-20T12:07:05.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/30' 2020-11-20T12:07:05.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/33' 2020-11-20T12:07:05.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/23' 2020-11-20T12:07:05.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/35' 2020-11-20T12:07:05.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/26' 2020-11-20T12:07:05.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './245/37' 2020-11-20T12:07:05.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './245' 2020-11-20T12:07:05.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/21' 2020-11-20T12:07:05.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/27' 2020-11-20T12:07:05.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/31' 2020-11-20T12:07:05.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/36' 2020-11-20T12:07:05.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/20' 2020-11-20T12:07:05.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/39' 2020-11-20T12:07:05.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/34' 2020-11-20T12:07:05.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/38' 2020-11-20T12:07:05.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/28' 2020-11-20T12:07:05.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/24' 2020-11-20T12:07:05.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/29' 2020-11-20T12:07:05.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/32' 2020-11-20T12:07:05.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/22' 2020-11-20T12:07:05.937 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/25' 2020-11-20T12:07:05.937 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/30' 2020-11-20T12:07:05.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/33' 2020-11-20T12:07:05.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/23' 2020-11-20T12:07:05.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/35' 2020-11-20T12:07:05.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/26' 2020-11-20T12:07:05.938 INFO:tasks.workunit.client.0.smithi044.stdout:removed './246/37' 2020-11-20T12:07:05.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './246' 2020-11-20T12:07:05.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/21' 2020-11-20T12:07:05.939 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/27' 2020-11-20T12:07:05.940 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/31' 2020-11-20T12:07:05.940 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/36' 2020-11-20T12:07:05.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/20' 2020-11-20T12:07:05.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/39' 2020-11-20T12:07:05.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/34' 2020-11-20T12:07:05.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/38' 2020-11-20T12:07:05.941 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/28' 2020-11-20T12:07:05.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/24' 2020-11-20T12:07:05.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/29' 2020-11-20T12:07:05.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/32' 2020-11-20T12:07:05.942 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/22' 2020-11-20T12:07:05.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/25' 2020-11-20T12:07:05.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/30' 2020-11-20T12:07:05.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/33' 2020-11-20T12:07:05.943 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/23' 2020-11-20T12:07:05.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/35' 2020-11-20T12:07:05.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/26' 2020-11-20T12:07:05.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed './247/37' 2020-11-20T12:07:05.944 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './247' 2020-11-20T12:07:05.945 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/21' 2020-11-20T12:07:05.945 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/27' 2020-11-20T12:07:05.945 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/31' 2020-11-20T12:07:05.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/36' 2020-11-20T12:07:05.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/20' 2020-11-20T12:07:05.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/39' 2020-11-20T12:07:05.946 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/34' 2020-11-20T12:07:05.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/38' 2020-11-20T12:07:05.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/28' 2020-11-20T12:07:05.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/24' 2020-11-20T12:07:05.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/29' 2020-11-20T12:07:05.947 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/32' 2020-11-20T12:07:05.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/22' 2020-11-20T12:07:05.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/25' 2020-11-20T12:07:05.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/30' 2020-11-20T12:07:05.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/33' 2020-11-20T12:07:05.948 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/23' 2020-11-20T12:07:05.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/35' 2020-11-20T12:07:05.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/26' 2020-11-20T12:07:05.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './248/37' 2020-11-20T12:07:05.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './248' 2020-11-20T12:07:05.949 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/21' 2020-11-20T12:07:05.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/27' 2020-11-20T12:07:05.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/31' 2020-11-20T12:07:05.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/36' 2020-11-20T12:07:05.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/20' 2020-11-20T12:07:05.950 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/39' 2020-11-20T12:07:05.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/34' 2020-11-20T12:07:05.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/38' 2020-11-20T12:07:05.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/28' 2020-11-20T12:07:05.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/24' 2020-11-20T12:07:05.951 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/29' 2020-11-20T12:07:05.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/32' 2020-11-20T12:07:05.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/22' 2020-11-20T12:07:05.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/25' 2020-11-20T12:07:05.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/30' 2020-11-20T12:07:05.952 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/33' 2020-11-20T12:07:05.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/23' 2020-11-20T12:07:05.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/35' 2020-11-20T12:07:05.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/26' 2020-11-20T12:07:05.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed './249/37' 2020-11-20T12:07:05.953 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './249' 2020-11-20T12:07:05.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/21' 2020-11-20T12:07:05.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/27' 2020-11-20T12:07:05.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/31' 2020-11-20T12:07:05.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/36' 2020-11-20T12:07:05.954 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/20' 2020-11-20T12:07:05.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/39' 2020-11-20T12:07:05.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/34' 2020-11-20T12:07:05.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/38' 2020-11-20T12:07:05.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/28' 2020-11-20T12:07:05.955 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/24' 2020-11-20T12:07:05.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/29' 2020-11-20T12:07:05.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/32' 2020-11-20T12:07:05.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/22' 2020-11-20T12:07:05.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/25' 2020-11-20T12:07:05.956 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/30' 2020-11-20T12:07:05.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/33' 2020-11-20T12:07:05.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/23' 2020-11-20T12:07:05.957 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/35' 2020-11-20T12:07:05.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/26' 2020-11-20T12:07:05.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './250/37' 2020-11-20T12:07:05.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './250' 2020-11-20T12:07:05.958 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/21' 2020-11-20T12:07:05.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/27' 2020-11-20T12:07:05.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/31' 2020-11-20T12:07:05.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/36' 2020-11-20T12:07:05.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/20' 2020-11-20T12:07:05.959 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/39' 2020-11-20T12:07:05.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/34' 2020-11-20T12:07:05.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/38' 2020-11-20T12:07:05.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/28' 2020-11-20T12:07:05.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/24' 2020-11-20T12:07:05.960 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/29' 2020-11-20T12:07:05.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/32' 2020-11-20T12:07:05.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/22' 2020-11-20T12:07:05.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/25' 2020-11-20T12:07:05.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/30' 2020-11-20T12:07:05.961 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/33' 2020-11-20T12:07:05.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/23' 2020-11-20T12:07:05.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/35' 2020-11-20T12:07:05.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/26' 2020-11-20T12:07:05.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed './251/37' 2020-11-20T12:07:05.962 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './251' 2020-11-20T12:07:05.963 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/21' 2020-11-20T12:07:05.963 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/27' 2020-11-20T12:07:05.963 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/31' 2020-11-20T12:07:05.963 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/36' 2020-11-20T12:07:05.963 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/20' 2020-11-20T12:07:05.964 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/39' 2020-11-20T12:07:05.964 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/34' 2020-11-20T12:07:05.964 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/38' 2020-11-20T12:07:05.964 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/28' 2020-11-20T12:07:05.964 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/24' 2020-11-20T12:07:05.965 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/29' 2020-11-20T12:07:05.965 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/32' 2020-11-20T12:07:05.965 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/22' 2020-11-20T12:07:05.965 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/25' 2020-11-20T12:07:05.966 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/30' 2020-11-20T12:07:05.966 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/33' 2020-11-20T12:07:05.966 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/23' 2020-11-20T12:07:05.966 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/35' 2020-11-20T12:07:05.966 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/26' 2020-11-20T12:07:05.967 INFO:tasks.workunit.client.0.smithi044.stdout:removed './252/37' 2020-11-20T12:07:05.967 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './252' 2020-11-20T12:07:05.967 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/21' 2020-11-20T12:07:05.967 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/27' 2020-11-20T12:07:05.967 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/31' 2020-11-20T12:07:05.968 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/36' 2020-11-20T12:07:05.968 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/20' 2020-11-20T12:07:05.968 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/39' 2020-11-20T12:07:05.968 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/34' 2020-11-20T12:07:05.969 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/38' 2020-11-20T12:07:05.969 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/28' 2020-11-20T12:07:05.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/24' 2020-11-20T12:07:05.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/29' 2020-11-20T12:07:05.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/32' 2020-11-20T12:07:05.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/22' 2020-11-20T12:07:05.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/25' 2020-11-20T12:07:05.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/30' 2020-11-20T12:07:05.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/33' 2020-11-20T12:07:05.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/23' 2020-11-20T12:07:05.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/35' 2020-11-20T12:07:05.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/26' 2020-11-20T12:07:05.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './253/37' 2020-11-20T12:07:05.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './253' 2020-11-20T12:07:05.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/21' 2020-11-20T12:07:05.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/27' 2020-11-20T12:07:05.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/31' 2020-11-20T12:07:05.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/36' 2020-11-20T12:07:05.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/20' 2020-11-20T12:07:05.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/39' 2020-11-20T12:07:05.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/34' 2020-11-20T12:07:05.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/38' 2020-11-20T12:07:05.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/28' 2020-11-20T12:07:05.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/24' 2020-11-20T12:07:05.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/29' 2020-11-20T12:07:05.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/32' 2020-11-20T12:07:05.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/22' 2020-11-20T12:07:05.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/25' 2020-11-20T12:07:05.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/30' 2020-11-20T12:07:05.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/33' 2020-11-20T12:07:05.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/23' 2020-11-20T12:07:05.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/35' 2020-11-20T12:07:05.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/26' 2020-11-20T12:07:05.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed './254/37' 2020-11-20T12:07:05.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './254' 2020-11-20T12:07:05.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/21' 2020-11-20T12:07:05.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/27' 2020-11-20T12:07:05.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/31' 2020-11-20T12:07:05.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/36' 2020-11-20T12:07:05.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/20' 2020-11-20T12:07:05.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/39' 2020-11-20T12:07:05.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/34' 2020-11-20T12:07:05.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/38' 2020-11-20T12:07:05.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/28' 2020-11-20T12:07:05.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/24' 2020-11-20T12:07:05.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/29' 2020-11-20T12:07:05.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/32' 2020-11-20T12:07:05.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/22' 2020-11-20T12:07:05.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/25' 2020-11-20T12:07:05.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/30' 2020-11-20T12:07:05.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/33' 2020-11-20T12:07:05.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/23' 2020-11-20T12:07:05.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/35' 2020-11-20T12:07:05.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/26' 2020-11-20T12:07:05.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed './255/37' 2020-11-20T12:07:05.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './255' 2020-11-20T12:07:05.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/21' 2020-11-20T12:07:05.983 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/27' 2020-11-20T12:07:05.983 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/31' 2020-11-20T12:07:05.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/36' 2020-11-20T12:07:05.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/20' 2020-11-20T12:07:05.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/39' 2020-11-20T12:07:05.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/34' 2020-11-20T12:07:05.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/38' 2020-11-20T12:07:05.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/28' 2020-11-20T12:07:05.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/24' 2020-11-20T12:07:05.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/29' 2020-11-20T12:07:05.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/32' 2020-11-20T12:07:05.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/22' 2020-11-20T12:07:05.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/25' 2020-11-20T12:07:05.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/30' 2020-11-20T12:07:06.384 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:07:06.387 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:07:06.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/33' 2020-11-20T12:07:06.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/23' 2020-11-20T12:07:06.720 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/35' 2020-11-20T12:07:06.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/26' 2020-11-20T12:07:06.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './256/37' 2020-11-20T12:07:06.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './256' 2020-11-20T12:07:06.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/21' 2020-11-20T12:07:06.721 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/27' 2020-11-20T12:07:06.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/31' 2020-11-20T12:07:06.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/36' 2020-11-20T12:07:06.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/20' 2020-11-20T12:07:06.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/39' 2020-11-20T12:07:06.722 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/34' 2020-11-20T12:07:06.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/38' 2020-11-20T12:07:06.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/28' 2020-11-20T12:07:06.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/24' 2020-11-20T12:07:06.723 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/29' 2020-11-20T12:07:06.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/32' 2020-11-20T12:07:06.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/22' 2020-11-20T12:07:06.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/25' 2020-11-20T12:07:06.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/30' 2020-11-20T12:07:06.724 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/33' 2020-11-20T12:07:06.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/23' 2020-11-20T12:07:06.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/35' 2020-11-20T12:07:06.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/26' 2020-11-20T12:07:06.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed './257/37' 2020-11-20T12:07:06.725 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './257' 2020-11-20T12:07:06.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/21' 2020-11-20T12:07:06.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/27' 2020-11-20T12:07:06.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/31' 2020-11-20T12:07:06.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/36' 2020-11-20T12:07:06.726 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/20' 2020-11-20T12:07:06.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/39' 2020-11-20T12:07:06.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/34' 2020-11-20T12:07:06.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/38' 2020-11-20T12:07:06.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/28' 2020-11-20T12:07:06.727 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/24' 2020-11-20T12:07:06.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/29' 2020-11-20T12:07:06.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/32' 2020-11-20T12:07:06.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/22' 2020-11-20T12:07:06.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/25' 2020-11-20T12:07:06.728 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/30' 2020-11-20T12:07:06.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/33' 2020-11-20T12:07:06.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/23' 2020-11-20T12:07:06.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/35' 2020-11-20T12:07:06.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/26' 2020-11-20T12:07:06.729 INFO:tasks.workunit.client.0.smithi044.stdout:removed './258/37' 2020-11-20T12:07:06.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './258' 2020-11-20T12:07:06.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/21' 2020-11-20T12:07:06.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/27' 2020-11-20T12:07:06.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/31' 2020-11-20T12:07:06.730 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/36' 2020-11-20T12:07:06.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/20' 2020-11-20T12:07:06.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/39' 2020-11-20T12:07:06.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/34' 2020-11-20T12:07:06.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/38' 2020-11-20T12:07:06.731 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/28' 2020-11-20T12:07:06.732 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/24' 2020-11-20T12:07:06.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/29' 2020-11-20T12:07:06.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/32' 2020-11-20T12:07:06.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/22' 2020-11-20T12:07:06.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/25' 2020-11-20T12:07:06.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/30' 2020-11-20T12:07:06.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/33' 2020-11-20T12:07:06.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/23' 2020-11-20T12:07:06.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/35' 2020-11-20T12:07:06.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/26' 2020-11-20T12:07:06.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './259/37' 2020-11-20T12:07:06.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './259' 2020-11-20T12:07:06.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/21' 2020-11-20T12:07:06.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/27' 2020-11-20T12:07:06.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/31' 2020-11-20T12:07:06.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/36' 2020-11-20T12:07:06.738 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/20' 2020-11-20T12:07:06.738 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/39' 2020-11-20T12:07:06.738 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/34' 2020-11-20T12:07:06.738 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/38' 2020-11-20T12:07:06.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/28' 2020-11-20T12:07:06.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/24' 2020-11-20T12:07:06.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/29' 2020-11-20T12:07:06.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/32' 2020-11-20T12:07:06.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/22' 2020-11-20T12:07:06.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/25' 2020-11-20T12:07:06.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/30' 2020-11-20T12:07:06.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/33' 2020-11-20T12:07:06.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/23' 2020-11-20T12:07:06.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/35' 2020-11-20T12:07:06.741 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/26' 2020-11-20T12:07:06.741 INFO:tasks.workunit.client.0.smithi044.stdout:removed './260/37' 2020-11-20T12:07:06.741 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './260' 2020-11-20T12:07:06.741 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/21' 2020-11-20T12:07:06.742 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/27' 2020-11-20T12:07:06.742 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/31' 2020-11-20T12:07:06.742 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/36' 2020-11-20T12:07:06.742 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/20' 2020-11-20T12:07:06.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/39' 2020-11-20T12:07:06.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/34' 2020-11-20T12:07:06.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/38' 2020-11-20T12:07:06.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/28' 2020-11-20T12:07:06.744 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/24' 2020-11-20T12:07:06.744 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/29' 2020-11-20T12:07:06.744 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/32' 2020-11-20T12:07:06.744 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/22' 2020-11-20T12:07:06.745 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/25' 2020-11-20T12:07:06.745 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/30' 2020-11-20T12:07:06.745 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/33' 2020-11-20T12:07:06.745 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/23' 2020-11-20T12:07:06.746 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/35' 2020-11-20T12:07:06.746 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/26' 2020-11-20T12:07:06.746 INFO:tasks.workunit.client.0.smithi044.stdout:removed './261/37' 2020-11-20T12:07:06.746 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './261' 2020-11-20T12:07:06.747 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/21' 2020-11-20T12:07:06.747 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/27' 2020-11-20T12:07:06.747 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/31' 2020-11-20T12:07:06.747 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/36' 2020-11-20T12:07:06.748 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/20' 2020-11-20T12:07:06.748 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/39' 2020-11-20T12:07:06.748 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/34' 2020-11-20T12:07:06.748 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/38' 2020-11-20T12:07:06.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/28' 2020-11-20T12:07:06.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/24' 2020-11-20T12:07:06.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/29' 2020-11-20T12:07:06.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/32' 2020-11-20T12:07:06.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/22' 2020-11-20T12:07:06.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/25' 2020-11-20T12:07:06.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/30' 2020-11-20T12:07:06.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/33' 2020-11-20T12:07:06.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/23' 2020-11-20T12:07:06.751 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/35' 2020-11-20T12:07:06.751 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/26' 2020-11-20T12:07:06.751 INFO:tasks.workunit.client.0.smithi044.stdout:removed './262/37' 2020-11-20T12:07:06.751 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './262' 2020-11-20T12:07:06.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/21' 2020-11-20T12:07:06.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/27' 2020-11-20T12:07:06.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/31' 2020-11-20T12:07:06.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/36' 2020-11-20T12:07:06.753 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/20' 2020-11-20T12:07:06.753 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/39' 2020-11-20T12:07:06.753 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/34' 2020-11-20T12:07:06.753 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/38' 2020-11-20T12:07:06.754 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/28' 2020-11-20T12:07:06.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/24' 2020-11-20T12:07:06.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/29' 2020-11-20T12:07:06.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/32' 2020-11-20T12:07:06.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/22' 2020-11-20T12:07:06.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/25' 2020-11-20T12:07:06.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/30' 2020-11-20T12:07:06.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/33' 2020-11-20T12:07:06.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/23' 2020-11-20T12:07:06.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/35' 2020-11-20T12:07:06.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/26' 2020-11-20T12:07:06.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './263/37' 2020-11-20T12:07:06.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './263' 2020-11-20T12:07:06.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/21' 2020-11-20T12:07:06.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/27' 2020-11-20T12:07:06.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/31' 2020-11-20T12:07:06.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/36' 2020-11-20T12:07:06.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/20' 2020-11-20T12:07:06.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/39' 2020-11-20T12:07:06.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/34' 2020-11-20T12:07:06.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/38' 2020-11-20T12:07:06.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/28' 2020-11-20T12:07:06.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/24' 2020-11-20T12:07:06.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/29' 2020-11-20T12:07:06.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/32' 2020-11-20T12:07:06.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/22' 2020-11-20T12:07:06.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/25' 2020-11-20T12:07:06.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/30' 2020-11-20T12:07:06.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/33' 2020-11-20T12:07:06.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/23' 2020-11-20T12:07:06.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/35' 2020-11-20T12:07:06.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/26' 2020-11-20T12:07:06.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './264/37' 2020-11-20T12:07:06.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './264' 2020-11-20T12:07:06.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/21' 2020-11-20T12:07:06.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/27' 2020-11-20T12:07:06.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/31' 2020-11-20T12:07:06.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/36' 2020-11-20T12:07:06.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/20' 2020-11-20T12:07:06.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/39' 2020-11-20T12:07:06.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/34' 2020-11-20T12:07:06.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/38' 2020-11-20T12:07:06.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/28' 2020-11-20T12:07:06.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/24' 2020-11-20T12:07:06.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/29' 2020-11-20T12:07:06.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/32' 2020-11-20T12:07:06.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/22' 2020-11-20T12:07:06.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/25' 2020-11-20T12:07:06.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/30' 2020-11-20T12:07:06.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/33' 2020-11-20T12:07:06.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/23' 2020-11-20T12:07:06.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/35' 2020-11-20T12:07:06.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/26' 2020-11-20T12:07:06.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './265/37' 2020-11-20T12:07:06.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './265' 2020-11-20T12:07:06.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/21' 2020-11-20T12:07:06.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/27' 2020-11-20T12:07:06.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/31' 2020-11-20T12:07:06.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/36' 2020-11-20T12:07:06.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/20' 2020-11-20T12:07:06.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/39' 2020-11-20T12:07:06.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/34' 2020-11-20T12:07:06.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/38' 2020-11-20T12:07:06.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/28' 2020-11-20T12:07:06.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/24' 2020-11-20T12:07:06.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/29' 2020-11-20T12:07:06.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/32' 2020-11-20T12:07:06.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/22' 2020-11-20T12:07:06.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/25' 2020-11-20T12:07:06.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/30' 2020-11-20T12:07:06.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/33' 2020-11-20T12:07:06.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/23' 2020-11-20T12:07:07.380 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/35' 2020-11-20T12:07:07.380 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/26' 2020-11-20T12:07:07.380 INFO:tasks.workunit.client.0.smithi044.stdout:removed './266/37' 2020-11-20T12:07:07.381 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './266' 2020-11-20T12:07:07.381 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/21' 2020-11-20T12:07:07.381 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/27' 2020-11-20T12:07:07.381 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/31' 2020-11-20T12:07:07.381 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/36' 2020-11-20T12:07:07.382 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/20' 2020-11-20T12:07:07.382 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/39' 2020-11-20T12:07:07.382 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/34' 2020-11-20T12:07:07.382 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/38' 2020-11-20T12:07:07.382 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/28' 2020-11-20T12:07:07.383 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/24' 2020-11-20T12:07:07.383 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/29' 2020-11-20T12:07:07.383 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/32' 2020-11-20T12:07:07.383 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/22' 2020-11-20T12:07:07.383 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/25' 2020-11-20T12:07:07.384 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/30' 2020-11-20T12:07:07.384 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/33' 2020-11-20T12:07:07.384 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/23' 2020-11-20T12:07:07.384 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/35' 2020-11-20T12:07:07.384 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/26' 2020-11-20T12:07:07.385 INFO:tasks.workunit.client.0.smithi044.stdout:removed './267/37' 2020-11-20T12:07:07.385 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './267' 2020-11-20T12:07:07.385 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/21' 2020-11-20T12:07:07.385 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/27' 2020-11-20T12:07:07.385 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/31' 2020-11-20T12:07:07.386 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/36' 2020-11-20T12:07:07.386 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/20' 2020-11-20T12:07:07.386 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/39' 2020-11-20T12:07:07.386 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/34' 2020-11-20T12:07:07.386 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/38' 2020-11-20T12:07:07.387 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/28' 2020-11-20T12:07:07.387 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/24' 2020-11-20T12:07:07.387 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/29' 2020-11-20T12:07:07.387 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/32' 2020-11-20T12:07:07.387 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/22' 2020-11-20T12:07:07.388 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/25' 2020-11-20T12:07:07.388 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/30' 2020-11-20T12:07:07.388 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/33' 2020-11-20T12:07:07.388 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/23' 2020-11-20T12:07:07.388 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/35' 2020-11-20T12:07:07.389 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/26' 2020-11-20T12:07:07.389 INFO:tasks.workunit.client.0.smithi044.stdout:removed './268/37' 2020-11-20T12:07:07.389 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './268' 2020-11-20T12:07:07.389 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/21' 2020-11-20T12:07:07.389 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/27' 2020-11-20T12:07:07.390 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/31' 2020-11-20T12:07:07.390 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/36' 2020-11-20T12:07:07.390 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/20' 2020-11-20T12:07:07.390 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/39' 2020-11-20T12:07:07.390 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/34' 2020-11-20T12:07:07.391 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/38' 2020-11-20T12:07:07.391 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/28' 2020-11-20T12:07:07.391 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/24' 2020-11-20T12:07:07.391 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/29' 2020-11-20T12:07:07.391 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/32' 2020-11-20T12:07:07.392 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/22' 2020-11-20T12:07:07.392 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/25' 2020-11-20T12:07:07.392 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/30' 2020-11-20T12:07:07.392 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/33' 2020-11-20T12:07:07.392 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/23' 2020-11-20T12:07:07.393 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/35' 2020-11-20T12:07:07.393 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/26' 2020-11-20T12:07:07.393 INFO:tasks.workunit.client.0.smithi044.stdout:removed './269/37' 2020-11-20T12:07:07.393 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './269' 2020-11-20T12:07:07.393 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/21' 2020-11-20T12:07:07.394 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/27' 2020-11-20T12:07:07.394 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/31' 2020-11-20T12:07:07.394 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/36' 2020-11-20T12:07:07.394 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/20' 2020-11-20T12:07:07.394 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/39' 2020-11-20T12:07:07.395 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/34' 2020-11-20T12:07:07.395 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/38' 2020-11-20T12:07:07.395 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/28' 2020-11-20T12:07:07.395 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/24' 2020-11-20T12:07:07.395 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/29' 2020-11-20T12:07:07.396 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/32' 2020-11-20T12:07:07.396 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/22' 2020-11-20T12:07:07.396 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/25' 2020-11-20T12:07:07.396 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/30' 2020-11-20T12:07:07.396 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/33' 2020-11-20T12:07:07.397 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/23' 2020-11-20T12:07:07.397 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/35' 2020-11-20T12:07:07.397 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/26' 2020-11-20T12:07:07.397 INFO:tasks.workunit.client.0.smithi044.stdout:removed './270/37' 2020-11-20T12:07:07.397 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './270' 2020-11-20T12:07:07.398 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/21' 2020-11-20T12:07:07.398 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/27' 2020-11-20T12:07:07.398 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/31' 2020-11-20T12:07:07.398 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/36' 2020-11-20T12:07:07.398 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/20' 2020-11-20T12:07:07.399 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/39' 2020-11-20T12:07:07.399 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/34' 2020-11-20T12:07:07.399 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/38' 2020-11-20T12:07:07.399 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/28' 2020-11-20T12:07:07.399 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/24' 2020-11-20T12:07:07.400 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/29' 2020-11-20T12:07:07.400 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/32' 2020-11-20T12:07:07.400 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/22' 2020-11-20T12:07:07.400 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/25' 2020-11-20T12:07:07.400 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/30' 2020-11-20T12:07:07.401 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/33' 2020-11-20T12:07:07.401 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/23' 2020-11-20T12:07:07.401 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/35' 2020-11-20T12:07:07.401 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/26' 2020-11-20T12:07:07.401 INFO:tasks.workunit.client.0.smithi044.stdout:removed './271/37' 2020-11-20T12:07:07.402 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './271' 2020-11-20T12:07:07.402 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/21' 2020-11-20T12:07:07.402 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/27' 2020-11-20T12:07:07.402 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/31' 2020-11-20T12:07:07.402 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/36' 2020-11-20T12:07:07.403 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/20' 2020-11-20T12:07:07.403 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/39' 2020-11-20T12:07:07.403 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/34' 2020-11-20T12:07:07.403 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/38' 2020-11-20T12:07:07.403 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/28' 2020-11-20T12:07:07.404 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/24' 2020-11-20T12:07:07.404 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/29' 2020-11-20T12:07:07.404 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/32' 2020-11-20T12:07:07.404 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/22' 2020-11-20T12:07:07.404 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/25' 2020-11-20T12:07:07.405 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/30' 2020-11-20T12:07:07.405 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/33' 2020-11-20T12:07:07.405 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/23' 2020-11-20T12:07:07.405 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/35' 2020-11-20T12:07:07.405 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/26' 2020-11-20T12:07:07.406 INFO:tasks.workunit.client.0.smithi044.stdout:removed './272/37' 2020-11-20T12:07:07.406 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './272' 2020-11-20T12:07:07.406 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/21' 2020-11-20T12:07:07.406 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/27' 2020-11-20T12:07:07.406 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/31' 2020-11-20T12:07:07.407 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/36' 2020-11-20T12:07:07.407 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/20' 2020-11-20T12:07:07.407 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/39' 2020-11-20T12:07:07.407 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/34' 2020-11-20T12:07:07.407 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/38' 2020-11-20T12:07:07.408 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/28' 2020-11-20T12:07:07.408 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/24' 2020-11-20T12:07:07.408 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/29' 2020-11-20T12:07:07.408 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/32' 2020-11-20T12:07:07.408 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/22' 2020-11-20T12:07:07.408 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/25' 2020-11-20T12:07:07.409 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/30' 2020-11-20T12:07:07.409 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/33' 2020-11-20T12:07:07.409 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/23' 2020-11-20T12:07:07.409 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/35' 2020-11-20T12:07:07.410 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/26' 2020-11-20T12:07:07.410 INFO:tasks.workunit.client.0.smithi044.stdout:removed './273/37' 2020-11-20T12:07:07.410 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './273' 2020-11-20T12:07:07.410 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/21' 2020-11-20T12:07:07.410 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/27' 2020-11-20T12:07:07.410 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/31' 2020-11-20T12:07:07.411 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/36' 2020-11-20T12:07:07.411 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/20' 2020-11-20T12:07:07.411 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/39' 2020-11-20T12:07:07.411 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/34' 2020-11-20T12:07:07.411 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/38' 2020-11-20T12:07:07.412 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/28' 2020-11-20T12:07:07.412 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/24' 2020-11-20T12:07:07.412 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/29' 2020-11-20T12:07:07.412 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/32' 2020-11-20T12:07:07.412 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/22' 2020-11-20T12:07:07.413 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/25' 2020-11-20T12:07:07.413 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/30' 2020-11-20T12:07:07.413 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/33' 2020-11-20T12:07:07.413 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/23' 2020-11-20T12:07:07.413 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/35' 2020-11-20T12:07:07.414 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/26' 2020-11-20T12:07:07.414 INFO:tasks.workunit.client.0.smithi044.stdout:removed './274/37' 2020-11-20T12:07:07.414 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './274' 2020-11-20T12:07:07.414 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/21' 2020-11-20T12:07:07.414 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/27' 2020-11-20T12:07:07.415 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/31' 2020-11-20T12:07:07.415 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/36' 2020-11-20T12:07:07.415 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/20' 2020-11-20T12:07:07.415 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/39' 2020-11-20T12:07:07.415 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/34' 2020-11-20T12:07:07.416 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/38' 2020-11-20T12:07:07.416 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/28' 2020-11-20T12:07:07.416 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/24' 2020-11-20T12:07:07.416 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/29' 2020-11-20T12:07:07.416 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/32' 2020-11-20T12:07:07.417 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/22' 2020-11-20T12:07:07.417 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/25' 2020-11-20T12:07:07.417 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/30' 2020-11-20T12:07:07.417 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/33' 2020-11-20T12:07:07.417 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/23' 2020-11-20T12:07:07.418 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/35' 2020-11-20T12:07:07.418 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/26' 2020-11-20T12:07:07.418 INFO:tasks.workunit.client.0.smithi044.stdout:removed './275/37' 2020-11-20T12:07:07.418 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './275' 2020-11-20T12:07:07.418 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/21' 2020-11-20T12:07:07.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/27' 2020-11-20T12:07:07.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/31' 2020-11-20T12:07:07.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/36' 2020-11-20T12:07:07.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/20' 2020-11-20T12:07:07.419 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/39' 2020-11-20T12:07:07.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/34' 2020-11-20T12:07:07.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/38' 2020-11-20T12:07:07.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/28' 2020-11-20T12:07:07.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/24' 2020-11-20T12:07:07.420 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/29' 2020-11-20T12:07:07.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/32' 2020-11-20T12:07:07.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/22' 2020-11-20T12:07:07.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/25' 2020-11-20T12:07:07.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/30' 2020-11-20T12:07:07.421 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/33' 2020-11-20T12:07:07.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/23' 2020-11-20T12:07:07.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/35' 2020-11-20T12:07:07.422 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/26' 2020-11-20T12:07:07.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.822+0000 7fc8937fe700 1 -- 172.21.15.44:0/1360299394 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fc8a80488b0 con 0x7fc8b40a4c30 2020-11-20T12:07:07.826 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":32,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:07.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc88c056d20 msgr2=0x7fc88c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc88c056d20 0x7fc88c0591e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fc8a400b1b0 tx=0x7fc8a4015000).stop 2020-11-20T12:07:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b40a4c30 msgr2=0x7fc8b413d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:07.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b40a4c30 0x7fc8b413d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fc8a8006c00 tx=0x7fc8a8008040).stop 2020-11-20T12:07:07.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 shutdown_connections 2020-11-20T12:07:07.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc88c056d20 0x7fc88c0591e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:07.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8b413ae80 0x7fc8b413c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:07.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8b4060330 0x7fc8b4134ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:07.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 --2- 172.21.15.44:0/1360299394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc8b40a4c30 0x7fc8b413d9e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:07.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 >> 172.21.15.44:0/1360299394 conn(0x7fc8b4062690 msgr2=0x7fc8b413a760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 shutdown_connections 2020-11-20T12:07:07.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:07.825+0000 7fc8b9e69700 1 -- 172.21.15.44:0/1360299394 wait complete. 2020-11-20T12:07:08.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './276/37' 2020-11-20T12:07:08.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './276' 2020-11-20T12:07:08.162 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/21' 2020-11-20T12:07:08.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/27' 2020-11-20T12:07:08.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/31' 2020-11-20T12:07:08.163 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/36' 2020-11-20T12:07:08.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/20' 2020-11-20T12:07:08.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/39' 2020-11-20T12:07:08.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/34' 2020-11-20T12:07:08.164 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/38' 2020-11-20T12:07:08.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/28' 2020-11-20T12:07:08.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/24' 2020-11-20T12:07:08.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/29' 2020-11-20T12:07:08.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/32' 2020-11-20T12:07:08.165 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/22' 2020-11-20T12:07:08.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/25' 2020-11-20T12:07:08.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/30' 2020-11-20T12:07:08.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/33' 2020-11-20T12:07:08.166 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/23' 2020-11-20T12:07:08.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/35' 2020-11-20T12:07:08.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/26' 2020-11-20T12:07:08.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './277/37' 2020-11-20T12:07:08.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './277' 2020-11-20T12:07:08.167 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/21' 2020-11-20T12:07:08.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/27' 2020-11-20T12:07:08.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/31' 2020-11-20T12:07:08.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/36' 2020-11-20T12:07:08.168 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/20' 2020-11-20T12:07:08.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/39' 2020-11-20T12:07:08.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/34' 2020-11-20T12:07:08.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/38' 2020-11-20T12:07:08.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/28' 2020-11-20T12:07:08.169 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/24' 2020-11-20T12:07:08.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/29' 2020-11-20T12:07:08.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/32' 2020-11-20T12:07:08.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/22' 2020-11-20T12:07:08.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/25' 2020-11-20T12:07:08.170 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/30' 2020-11-20T12:07:08.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/33' 2020-11-20T12:07:08.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/23' 2020-11-20T12:07:08.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/35' 2020-11-20T12:07:08.171 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/26' 2020-11-20T12:07:08.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './278/37' 2020-11-20T12:07:08.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './278' 2020-11-20T12:07:08.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/21' 2020-11-20T12:07:08.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/27' 2020-11-20T12:07:08.172 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/31' 2020-11-20T12:07:08.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/36' 2020-11-20T12:07:08.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/20' 2020-11-20T12:07:08.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/39' 2020-11-20T12:07:08.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/34' 2020-11-20T12:07:08.173 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/38' 2020-11-20T12:07:08.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/28' 2020-11-20T12:07:08.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/24' 2020-11-20T12:07:08.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/29' 2020-11-20T12:07:08.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/32' 2020-11-20T12:07:08.174 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/22' 2020-11-20T12:07:08.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/25' 2020-11-20T12:07:08.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/30' 2020-11-20T12:07:08.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/33' 2020-11-20T12:07:08.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/23' 2020-11-20T12:07:08.175 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/35' 2020-11-20T12:07:08.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/26' 2020-11-20T12:07:08.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './279/37' 2020-11-20T12:07:08.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './279' 2020-11-20T12:07:08.176 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/21' 2020-11-20T12:07:08.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/27' 2020-11-20T12:07:08.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/31' 2020-11-20T12:07:08.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/36' 2020-11-20T12:07:08.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/20' 2020-11-20T12:07:08.177 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/39' 2020-11-20T12:07:08.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/34' 2020-11-20T12:07:08.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/38' 2020-11-20T12:07:08.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/28' 2020-11-20T12:07:08.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/24' 2020-11-20T12:07:08.178 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/29' 2020-11-20T12:07:08.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/32' 2020-11-20T12:07:08.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/22' 2020-11-20T12:07:08.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/25' 2020-11-20T12:07:08.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/30' 2020-11-20T12:07:08.179 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/33' 2020-11-20T12:07:08.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/23' 2020-11-20T12:07:08.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/35' 2020-11-20T12:07:08.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/26' 2020-11-20T12:07:08.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed './280/37' 2020-11-20T12:07:08.180 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './280' 2020-11-20T12:07:08.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/21' 2020-11-20T12:07:08.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/27' 2020-11-20T12:07:08.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/31' 2020-11-20T12:07:08.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/36' 2020-11-20T12:07:08.181 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/20' 2020-11-20T12:07:08.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/39' 2020-11-20T12:07:08.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/34' 2020-11-20T12:07:08.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/38' 2020-11-20T12:07:08.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/28' 2020-11-20T12:07:08.182 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/24' 2020-11-20T12:07:08.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/29' 2020-11-20T12:07:08.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/32' 2020-11-20T12:07:08.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/22' 2020-11-20T12:07:08.183 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/25' 2020-11-20T12:07:08.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/30' 2020-11-20T12:07:08.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/33' 2020-11-20T12:07:08.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/23' 2020-11-20T12:07:08.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/35' 2020-11-20T12:07:08.184 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/26' 2020-11-20T12:07:08.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed './281/37' 2020-11-20T12:07:08.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './281' 2020-11-20T12:07:08.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/21' 2020-11-20T12:07:08.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/27' 2020-11-20T12:07:08.185 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/31' 2020-11-20T12:07:08.186 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/36' 2020-11-20T12:07:08.186 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/20' 2020-11-20T12:07:08.186 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/39' 2020-11-20T12:07:08.186 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/34' 2020-11-20T12:07:08.186 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/38' 2020-11-20T12:07:08.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/28' 2020-11-20T12:07:08.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/24' 2020-11-20T12:07:08.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/29' 2020-11-20T12:07:08.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/32' 2020-11-20T12:07:08.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/22' 2020-11-20T12:07:08.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/25' 2020-11-20T12:07:08.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/30' 2020-11-20T12:07:08.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/33' 2020-11-20T12:07:08.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/23' 2020-11-20T12:07:08.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/35' 2020-11-20T12:07:08.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/26' 2020-11-20T12:07:08.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed './282/37' 2020-11-20T12:07:08.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './282' 2020-11-20T12:07:08.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/21' 2020-11-20T12:07:08.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/27' 2020-11-20T12:07:08.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/31' 2020-11-20T12:07:08.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/36' 2020-11-20T12:07:08.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/20' 2020-11-20T12:07:08.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/39' 2020-11-20T12:07:08.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/34' 2020-11-20T12:07:08.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/38' 2020-11-20T12:07:08.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/28' 2020-11-20T12:07:08.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/24' 2020-11-20T12:07:08.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/29' 2020-11-20T12:07:08.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/32' 2020-11-20T12:07:08.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/22' 2020-11-20T12:07:08.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/25' 2020-11-20T12:07:08.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/30' 2020-11-20T12:07:08.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/33' 2020-11-20T12:07:08.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/23' 2020-11-20T12:07:08.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/35' 2020-11-20T12:07:08.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/26' 2020-11-20T12:07:08.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './283/37' 2020-11-20T12:07:08.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './283' 2020-11-20T12:07:08.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/21' 2020-11-20T12:07:08.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/27' 2020-11-20T12:07:08.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/31' 2020-11-20T12:07:08.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/36' 2020-11-20T12:07:08.195 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/20' 2020-11-20T12:07:08.195 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/39' 2020-11-20T12:07:08.195 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/34' 2020-11-20T12:07:08.195 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/38' 2020-11-20T12:07:08.195 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/28' 2020-11-20T12:07:08.196 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/24' 2020-11-20T12:07:08.196 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/29' 2020-11-20T12:07:08.196 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/32' 2020-11-20T12:07:08.196 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/22' 2020-11-20T12:07:08.196 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/25' 2020-11-20T12:07:08.197 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/30' 2020-11-20T12:07:08.197 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/33' 2020-11-20T12:07:08.197 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/23' 2020-11-20T12:07:08.197 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/35' 2020-11-20T12:07:08.197 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/26' 2020-11-20T12:07:08.198 INFO:tasks.workunit.client.0.smithi044.stdout:removed './284/37' 2020-11-20T12:07:08.198 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './284' 2020-11-20T12:07:08.198 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/21' 2020-11-20T12:07:08.198 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/27' 2020-11-20T12:07:08.199 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/31' 2020-11-20T12:07:08.199 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/36' 2020-11-20T12:07:08.199 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/20' 2020-11-20T12:07:08.199 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/39' 2020-11-20T12:07:08.199 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/34' 2020-11-20T12:07:08.200 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/38' 2020-11-20T12:07:08.200 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/28' 2020-11-20T12:07:08.200 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/24' 2020-11-20T12:07:08.200 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/29' 2020-11-20T12:07:08.200 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/32' 2020-11-20T12:07:08.201 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/22' 2020-11-20T12:07:08.201 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/25' 2020-11-20T12:07:08.201 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/30' 2020-11-20T12:07:08.201 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/33' 2020-11-20T12:07:08.201 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/23' 2020-11-20T12:07:08.202 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/35' 2020-11-20T12:07:08.202 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/26' 2020-11-20T12:07:08.202 INFO:tasks.workunit.client.0.smithi044.stdout:removed './285/37' 2020-11-20T12:07:08.202 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './285' 2020-11-20T12:07:08.202 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/21' 2020-11-20T12:07:08.203 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/27' 2020-11-20T12:07:08.203 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/31' 2020-11-20T12:07:08.203 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/36' 2020-11-20T12:07:08.203 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/20' 2020-11-20T12:07:08.203 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/39' 2020-11-20T12:07:08.204 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/34' 2020-11-20T12:07:08.204 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/38' 2020-11-20T12:07:08.204 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/28' 2020-11-20T12:07:08.204 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/24' 2020-11-20T12:07:08.204 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/29' 2020-11-20T12:07:08.205 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/32' 2020-11-20T12:07:08.205 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/22' 2020-11-20T12:07:08.205 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/25' 2020-11-20T12:07:08.205 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/30' 2020-11-20T12:07:08.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/33' 2020-11-20T12:07:08.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/23' 2020-11-20T12:07:08.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/35' 2020-11-20T12:07:08.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/26' 2020-11-20T12:07:08.206 INFO:tasks.workunit.client.0.smithi044.stdout:removed './286/37' 2020-11-20T12:07:08.207 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './286' 2020-11-20T12:07:08.968 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/21' 2020-11-20T12:07:08.968 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/27' 2020-11-20T12:07:08.969 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/31' 2020-11-20T12:07:08.969 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/36' 2020-11-20T12:07:08.969 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/20' 2020-11-20T12:07:08.969 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/39' 2020-11-20T12:07:08.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/34' 2020-11-20T12:07:08.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/38' 2020-11-20T12:07:08.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/28' 2020-11-20T12:07:08.970 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/24' 2020-11-20T12:07:08.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/29' 2020-11-20T12:07:08.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/32' 2020-11-20T12:07:08.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/22' 2020-11-20T12:07:08.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/25' 2020-11-20T12:07:08.971 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/30' 2020-11-20T12:07:08.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/33' 2020-11-20T12:07:08.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/23' 2020-11-20T12:07:08.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/35' 2020-11-20T12:07:08.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/26' 2020-11-20T12:07:08.972 INFO:tasks.workunit.client.0.smithi044.stdout:removed './287/37' 2020-11-20T12:07:08.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './287' 2020-11-20T12:07:08.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/21' 2020-11-20T12:07:08.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/27' 2020-11-20T12:07:08.973 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/31' 2020-11-20T12:07:08.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/36' 2020-11-20T12:07:08.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/20' 2020-11-20T12:07:08.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/39' 2020-11-20T12:07:08.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/34' 2020-11-20T12:07:08.974 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/38' 2020-11-20T12:07:08.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/28' 2020-11-20T12:07:08.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/24' 2020-11-20T12:07:08.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/29' 2020-11-20T12:07:08.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/32' 2020-11-20T12:07:08.975 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/22' 2020-11-20T12:07:08.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/25' 2020-11-20T12:07:08.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/30' 2020-11-20T12:07:08.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/33' 2020-11-20T12:07:08.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/23' 2020-11-20T12:07:08.976 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/35' 2020-11-20T12:07:08.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/26' 2020-11-20T12:07:08.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed './288/37' 2020-11-20T12:07:08.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './288' 2020-11-20T12:07:08.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/21' 2020-11-20T12:07:08.977 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/27' 2020-11-20T12:07:08.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/31' 2020-11-20T12:07:08.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/36' 2020-11-20T12:07:08.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/20' 2020-11-20T12:07:08.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/39' 2020-11-20T12:07:08.978 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/34' 2020-11-20T12:07:08.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/38' 2020-11-20T12:07:08.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/28' 2020-11-20T12:07:08.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/24' 2020-11-20T12:07:08.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/29' 2020-11-20T12:07:08.979 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/32' 2020-11-20T12:07:08.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/22' 2020-11-20T12:07:08.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/25' 2020-11-20T12:07:08.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/30' 2020-11-20T12:07:08.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/33' 2020-11-20T12:07:08.980 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/23' 2020-11-20T12:07:08.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/35' 2020-11-20T12:07:08.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/26' 2020-11-20T12:07:08.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed './289/37' 2020-11-20T12:07:08.981 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './289' 2020-11-20T12:07:08.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/21' 2020-11-20T12:07:08.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/27' 2020-11-20T12:07:08.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/31' 2020-11-20T12:07:08.982 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/36' 2020-11-20T12:07:08.983 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/20' 2020-11-20T12:07:08.983 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/39' 2020-11-20T12:07:08.983 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/34' 2020-11-20T12:07:08.983 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/38' 2020-11-20T12:07:08.983 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/28' 2020-11-20T12:07:08.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/24' 2020-11-20T12:07:08.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/29' 2020-11-20T12:07:08.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/32' 2020-11-20T12:07:08.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/22' 2020-11-20T12:07:08.984 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/25' 2020-11-20T12:07:08.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/30' 2020-11-20T12:07:08.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/33' 2020-11-20T12:07:08.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/23' 2020-11-20T12:07:08.985 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/35' 2020-11-20T12:07:08.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/26' 2020-11-20T12:07:08.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed './290/37' 2020-11-20T12:07:08.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './290' 2020-11-20T12:07:08.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/21' 2020-11-20T12:07:08.986 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/27' 2020-11-20T12:07:08.987 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/31' 2020-11-20T12:07:08.987 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/36' 2020-11-20T12:07:08.987 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/20' 2020-11-20T12:07:08.987 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/39' 2020-11-20T12:07:08.987 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/34' 2020-11-20T12:07:08.988 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/38' 2020-11-20T12:07:08.988 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/28' 2020-11-20T12:07:08.988 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/24' 2020-11-20T12:07:08.988 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/29' 2020-11-20T12:07:08.988 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/32' 2020-11-20T12:07:08.989 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/22' 2020-11-20T12:07:08.989 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/25' 2020-11-20T12:07:08.989 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/30' 2020-11-20T12:07:08.989 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/33' 2020-11-20T12:07:08.989 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/23' 2020-11-20T12:07:08.990 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/35' 2020-11-20T12:07:08.990 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/26' 2020-11-20T12:07:08.990 INFO:tasks.workunit.client.0.smithi044.stdout:removed './291/37' 2020-11-20T12:07:08.990 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './291' 2020-11-20T12:07:08.990 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/21' 2020-11-20T12:07:08.991 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/27' 2020-11-20T12:07:08.991 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/31' 2020-11-20T12:07:08.991 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/36' 2020-11-20T12:07:08.991 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/20' 2020-11-20T12:07:08.991 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/39' 2020-11-20T12:07:08.992 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/34' 2020-11-20T12:07:08.992 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/38' 2020-11-20T12:07:08.992 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/28' 2020-11-20T12:07:08.992 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/24' 2020-11-20T12:07:08.992 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/29' 2020-11-20T12:07:08.993 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/32' 2020-11-20T12:07:08.993 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/22' 2020-11-20T12:07:08.993 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/25' 2020-11-20T12:07:08.993 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/30' 2020-11-20T12:07:08.994 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/33' 2020-11-20T12:07:08.994 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/23' 2020-11-20T12:07:08.994 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/35' 2020-11-20T12:07:08.994 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/26' 2020-11-20T12:07:08.995 INFO:tasks.workunit.client.0.smithi044.stdout:removed './292/37' 2020-11-20T12:07:08.995 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './292' 2020-11-20T12:07:08.995 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/21' 2020-11-20T12:07:08.995 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/27' 2020-11-20T12:07:08.996 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/31' 2020-11-20T12:07:08.996 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/36' 2020-11-20T12:07:08.996 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/20' 2020-11-20T12:07:08.996 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/39' 2020-11-20T12:07:08.997 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/34' 2020-11-20T12:07:08.997 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/38' 2020-11-20T12:07:08.997 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/28' 2020-11-20T12:07:08.997 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/24' 2020-11-20T12:07:08.997 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/29' 2020-11-20T12:07:08.998 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/32' 2020-11-20T12:07:08.998 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/22' 2020-11-20T12:07:08.998 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/25' 2020-11-20T12:07:08.998 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/30' 2020-11-20T12:07:08.998 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/33' 2020-11-20T12:07:08.999 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/23' 2020-11-20T12:07:08.999 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/35' 2020-11-20T12:07:08.999 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/26' 2020-11-20T12:07:08.999 INFO:tasks.workunit.client.0.smithi044.stdout:removed './293/37' 2020-11-20T12:07:08.999 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './293' 2020-11-20T12:07:09.000 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/21' 2020-11-20T12:07:09.000 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/27' 2020-11-20T12:07:09.000 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/31' 2020-11-20T12:07:09.000 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/36' 2020-11-20T12:07:09.001 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/20' 2020-11-20T12:07:09.001 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/39' 2020-11-20T12:07:09.001 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/34' 2020-11-20T12:07:09.001 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/38' 2020-11-20T12:07:09.001 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/28' 2020-11-20T12:07:09.002 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/24' 2020-11-20T12:07:09.002 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/29' 2020-11-20T12:07:09.002 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/32' 2020-11-20T12:07:09.002 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/22' 2020-11-20T12:07:09.002 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/25' 2020-11-20T12:07:09.003 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/30' 2020-11-20T12:07:09.003 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/33' 2020-11-20T12:07:09.003 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/23' 2020-11-20T12:07:09.003 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/35' 2020-11-20T12:07:09.003 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/26' 2020-11-20T12:07:09.004 INFO:tasks.workunit.client.0.smithi044.stdout:removed './294/37' 2020-11-20T12:07:09.004 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './294' 2020-11-20T12:07:09.004 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/21' 2020-11-20T12:07:09.004 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/27' 2020-11-20T12:07:09.005 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/31' 2020-11-20T12:07:09.005 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/36' 2020-11-20T12:07:09.005 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/20' 2020-11-20T12:07:09.005 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/39' 2020-11-20T12:07:09.005 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/34' 2020-11-20T12:07:09.006 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/38' 2020-11-20T12:07:09.006 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/28' 2020-11-20T12:07:09.006 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/24' 2020-11-20T12:07:09.006 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/29' 2020-11-20T12:07:09.006 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/32' 2020-11-20T12:07:09.007 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/22' 2020-11-20T12:07:09.007 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/25' 2020-11-20T12:07:09.007 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/30' 2020-11-20T12:07:09.007 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/33' 2020-11-20T12:07:09.008 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/23' 2020-11-20T12:07:09.008 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/35' 2020-11-20T12:07:09.008 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/26' 2020-11-20T12:07:09.008 INFO:tasks.workunit.client.0.smithi044.stdout:removed './295/37' 2020-11-20T12:07:09.008 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './295' 2020-11-20T12:07:09.009 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/21' 2020-11-20T12:07:09.009 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/27' 2020-11-20T12:07:09.009 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/31' 2020-11-20T12:07:09.010 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/36' 2020-11-20T12:07:09.010 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/20' 2020-11-20T12:07:09.010 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/39' 2020-11-20T12:07:09.010 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/34' 2020-11-20T12:07:09.010 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/38' 2020-11-20T12:07:09.011 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/28' 2020-11-20T12:07:09.011 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/24' 2020-11-20T12:07:09.011 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/29' 2020-11-20T12:07:09.011 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/32' 2020-11-20T12:07:09.011 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/22' 2020-11-20T12:07:09.012 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/25' 2020-11-20T12:07:09.012 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/30' 2020-11-20T12:07:09.012 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/33' 2020-11-20T12:07:09.012 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/23' 2020-11-20T12:07:09.012 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/35' 2020-11-20T12:07:09.013 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/26' 2020-11-20T12:07:09.013 INFO:tasks.workunit.client.0.smithi044.stdout:removed './296/37' 2020-11-20T12:07:09.013 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './296' 2020-11-20T12:07:09.013 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/21' 2020-11-20T12:07:09.013 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/27' 2020-11-20T12:07:09.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/31' 2020-11-20T12:07:09.479 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/36' 2020-11-20T12:07:09.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/20' 2020-11-20T12:07:09.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/39' 2020-11-20T12:07:09.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/34' 2020-11-20T12:07:09.480 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/38' 2020-11-20T12:07:09.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/28' 2020-11-20T12:07:09.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/24' 2020-11-20T12:07:09.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/29' 2020-11-20T12:07:09.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/32' 2020-11-20T12:07:09.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/22' 2020-11-20T12:07:09.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/25' 2020-11-20T12:07:09.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/30' 2020-11-20T12:07:09.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/33' 2020-11-20T12:07:09.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/23' 2020-11-20T12:07:09.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/35' 2020-11-20T12:07:09.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/26' 2020-11-20T12:07:09.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './297/37' 2020-11-20T12:07:09.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './297' 2020-11-20T12:07:09.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/21' 2020-11-20T12:07:09.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/27' 2020-11-20T12:07:09.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/31' 2020-11-20T12:07:09.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/36' 2020-11-20T12:07:09.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/20' 2020-11-20T12:07:09.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/39' 2020-11-20T12:07:09.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/34' 2020-11-20T12:07:09.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/38' 2020-11-20T12:07:09.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/28' 2020-11-20T12:07:09.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/24' 2020-11-20T12:07:09.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/29' 2020-11-20T12:07:09.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/32' 2020-11-20T12:07:09.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/22' 2020-11-20T12:07:09.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/25' 2020-11-20T12:07:09.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/30' 2020-11-20T12:07:09.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/33' 2020-11-20T12:07:09.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/23' 2020-11-20T12:07:09.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/35' 2020-11-20T12:07:09.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/26' 2020-11-20T12:07:09.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './298/37' 2020-11-20T12:07:09.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './298' 2020-11-20T12:07:09.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/21' 2020-11-20T12:07:09.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/27' 2020-11-20T12:07:09.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/31' 2020-11-20T12:07:09.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/36' 2020-11-20T12:07:09.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/20' 2020-11-20T12:07:09.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/39' 2020-11-20T12:07:09.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/34' 2020-11-20T12:07:09.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/38' 2020-11-20T12:07:09.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/28' 2020-11-20T12:07:09.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/24' 2020-11-20T12:07:09.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/29' 2020-11-20T12:07:09.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/32' 2020-11-20T12:07:09.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/22' 2020-11-20T12:07:09.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/25' 2020-11-20T12:07:09.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/30' 2020-11-20T12:07:09.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/33' 2020-11-20T12:07:09.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/23' 2020-11-20T12:07:09.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/35' 2020-11-20T12:07:09.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/26' 2020-11-20T12:07:09.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './299/37' 2020-11-20T12:07:09.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './299' 2020-11-20T12:07:09.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/19' 2020-11-20T12:07:09.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/15' 2020-11-20T12:07:09.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/20' 2020-11-20T12:07:09.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/12' 2020-11-20T12:07:09.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/10' 2020-11-20T12:07:09.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/11' 2020-11-20T12:07:09.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/16' 2020-11-20T12:07:09.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/14' 2020-11-20T12:07:09.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/18' 2020-11-20T12:07:09.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/13' 2020-11-20T12:07:09.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/166/17' 2020-11-20T12:07:09.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/166' 2020-11-20T12:07:09.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/19' 2020-11-20T12:07:09.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/15' 2020-11-20T12:07:09.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/20' 2020-11-20T12:07:09.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/12' 2020-11-20T12:07:09.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/10' 2020-11-20T12:07:09.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/11' 2020-11-20T12:07:09.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/16' 2020-11-20T12:07:09.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/14' 2020-11-20T12:07:09.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/18' 2020-11-20T12:07:09.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/13' 2020-11-20T12:07:09.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/147/17' 2020-11-20T12:07:09.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/147' 2020-11-20T12:07:09.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/19' 2020-11-20T12:07:09.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/15' 2020-11-20T12:07:09.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/20' 2020-11-20T12:07:09.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/12' 2020-11-20T12:07:09.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/10' 2020-11-20T12:07:09.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/11' 2020-11-20T12:07:09.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/16' 2020-11-20T12:07:09.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/14' 2020-11-20T12:07:09.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/18' 2020-11-20T12:07:09.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/13' 2020-11-20T12:07:09.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/151/17' 2020-11-20T12:07:09.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/151' 2020-11-20T12:07:09.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/19' 2020-11-20T12:07:09.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/15' 2020-11-20T12:07:09.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/20' 2020-11-20T12:07:09.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/12' 2020-11-20T12:07:09.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/10' 2020-11-20T12:07:09.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/11' 2020-11-20T12:07:09.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/16' 2020-11-20T12:07:09.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/14' 2020-11-20T12:07:09.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/18' 2020-11-20T12:07:09.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/13' 2020-11-20T12:07:09.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/170/17' 2020-11-20T12:07:09.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/170' 2020-11-20T12:07:09.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/19' 2020-11-20T12:07:09.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/15' 2020-11-20T12:07:09.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/20' 2020-11-20T12:07:09.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/12' 2020-11-20T12:07:09.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/10' 2020-11-20T12:07:09.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/11' 2020-11-20T12:07:09.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/16' 2020-11-20T12:07:09.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/14' 2020-11-20T12:07:09.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/18' 2020-11-20T12:07:09.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/13' 2020-11-20T12:07:09.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/167/17' 2020-11-20T12:07:09.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/167' 2020-11-20T12:07:09.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/19' 2020-11-20T12:07:09.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/15' 2020-11-20T12:07:09.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/20' 2020-11-20T12:07:09.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/12' 2020-11-20T12:07:09.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/10' 2020-11-20T12:07:09.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/11' 2020-11-20T12:07:09.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/16' 2020-11-20T12:07:09.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/14' 2020-11-20T12:07:09.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/18' 2020-11-20T12:07:09.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/13' 2020-11-20T12:07:09.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/140/17' 2020-11-20T12:07:09.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/140' 2020-11-20T12:07:09.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/19' 2020-11-20T12:07:09.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/15' 2020-11-20T12:07:09.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/20' 2020-11-20T12:07:09.513 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/12' 2020-11-20T12:07:09.513 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/10' 2020-11-20T12:07:10.027 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/11' 2020-11-20T12:07:10.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/16' 2020-11-20T12:07:10.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/14' 2020-11-20T12:07:10.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/18' 2020-11-20T12:07:10.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/13' 2020-11-20T12:07:10.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/142/17' 2020-11-20T12:07:10.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/142' 2020-11-20T12:07:10.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/19' 2020-11-20T12:07:10.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/15' 2020-11-20T12:07:10.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/20' 2020-11-20T12:07:10.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/12' 2020-11-20T12:07:10.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/10' 2020-11-20T12:07:10.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/11' 2020-11-20T12:07:10.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/16' 2020-11-20T12:07:10.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/14' 2020-11-20T12:07:10.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/18' 2020-11-20T12:07:10.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/13' 2020-11-20T12:07:10.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/150/17' 2020-11-20T12:07:10.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/150' 2020-11-20T12:07:10.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/19' 2020-11-20T12:07:10.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/15' 2020-11-20T12:07:10.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/20' 2020-11-20T12:07:10.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/12' 2020-11-20T12:07:10.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/10' 2020-11-20T12:07:10.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/11' 2020-11-20T12:07:10.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/16' 2020-11-20T12:07:10.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/14' 2020-11-20T12:07:10.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/18' 2020-11-20T12:07:10.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/13' 2020-11-20T12:07:10.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/105/17' 2020-11-20T12:07:10.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/105' 2020-11-20T12:07:10.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/19' 2020-11-20T12:07:10.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/15' 2020-11-20T12:07:10.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/20' 2020-11-20T12:07:10.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/12' 2020-11-20T12:07:10.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/10' 2020-11-20T12:07:10.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/11' 2020-11-20T12:07:10.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/16' 2020-11-20T12:07:10.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/14' 2020-11-20T12:07:10.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/18' 2020-11-20T12:07:10.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/13' 2020-11-20T12:07:10.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/149/17' 2020-11-20T12:07:10.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/149' 2020-11-20T12:07:10.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/19' 2020-11-20T12:07:10.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/15' 2020-11-20T12:07:10.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/20' 2020-11-20T12:07:10.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/12' 2020-11-20T12:07:10.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/10' 2020-11-20T12:07:10.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/11' 2020-11-20T12:07:10.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/16' 2020-11-20T12:07:10.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/14' 2020-11-20T12:07:10.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/18' 2020-11-20T12:07:10.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/13' 2020-11-20T12:07:10.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/109/17' 2020-11-20T12:07:10.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/109' 2020-11-20T12:07:10.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/19' 2020-11-20T12:07:10.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/15' 2020-11-20T12:07:10.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/20' 2020-11-20T12:07:10.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/12' 2020-11-20T12:07:10.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/10' 2020-11-20T12:07:10.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/11' 2020-11-20T12:07:10.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/16' 2020-11-20T12:07:10.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/14' 2020-11-20T12:07:10.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/18' 2020-11-20T12:07:10.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/13' 2020-11-20T12:07:10.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/158/17' 2020-11-20T12:07:10.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/158' 2020-11-20T12:07:10.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/19' 2020-11-20T12:07:10.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/15' 2020-11-20T12:07:10.043 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/20' 2020-11-20T12:07:10.043 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/12' 2020-11-20T12:07:10.043 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/10' 2020-11-20T12:07:10.043 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/11' 2020-11-20T12:07:10.043 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/16' 2020-11-20T12:07:10.044 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/14' 2020-11-20T12:07:10.044 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/18' 2020-11-20T12:07:10.044 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/13' 2020-11-20T12:07:10.044 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/106/17' 2020-11-20T12:07:10.044 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/106' 2020-11-20T12:07:10.045 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/19' 2020-11-20T12:07:10.045 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/15' 2020-11-20T12:07:10.045 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/20' 2020-11-20T12:07:10.045 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/12' 2020-11-20T12:07:10.046 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/10' 2020-11-20T12:07:10.046 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/11' 2020-11-20T12:07:10.046 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/16' 2020-11-20T12:07:10.046 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/14' 2020-11-20T12:07:10.046 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/18' 2020-11-20T12:07:10.047 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/13' 2020-11-20T12:07:10.047 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/189/17' 2020-11-20T12:07:10.047 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/189' 2020-11-20T12:07:10.047 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/19' 2020-11-20T12:07:10.047 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/15' 2020-11-20T12:07:10.048 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/20' 2020-11-20T12:07:10.048 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/12' 2020-11-20T12:07:10.048 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/10' 2020-11-20T12:07:10.048 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/11' 2020-11-20T12:07:10.048 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/16' 2020-11-20T12:07:10.049 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/14' 2020-11-20T12:07:10.049 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/18' 2020-11-20T12:07:10.049 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/13' 2020-11-20T12:07:10.049 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/133/17' 2020-11-20T12:07:10.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/133' 2020-11-20T12:07:10.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/19' 2020-11-20T12:07:10.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/15' 2020-11-20T12:07:10.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/20' 2020-11-20T12:07:10.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/12' 2020-11-20T12:07:10.051 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/10' 2020-11-20T12:07:10.051 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/11' 2020-11-20T12:07:10.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/16' 2020-11-20T12:07:10.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/14' 2020-11-20T12:07:10.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/18' 2020-11-20T12:07:10.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/13' 2020-11-20T12:07:10.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/173/17' 2020-11-20T12:07:10.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/173' 2020-11-20T12:07:10.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/19' 2020-11-20T12:07:10.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/15' 2020-11-20T12:07:10.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/20' 2020-11-20T12:07:10.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/12' 2020-11-20T12:07:10.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/10' 2020-11-20T12:07:10.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/11' 2020-11-20T12:07:10.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/16' 2020-11-20T12:07:10.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/14' 2020-11-20T12:07:10.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/18' 2020-11-20T12:07:10.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/13' 2020-11-20T12:07:10.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/174/17' 2020-11-20T12:07:10.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/174' 2020-11-20T12:07:10.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/19' 2020-11-20T12:07:10.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/15' 2020-11-20T12:07:10.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/20' 2020-11-20T12:07:10.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/12' 2020-11-20T12:07:10.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/10' 2020-11-20T12:07:10.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/11' 2020-11-20T12:07:10.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/16' 2020-11-20T12:07:10.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/14' 2020-11-20T12:07:10.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/18' 2020-11-20T12:07:10.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/13' 2020-11-20T12:07:10.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/155/17' 2020-11-20T12:07:10.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/155' 2020-11-20T12:07:10.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/19' 2020-11-20T12:07:10.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/15' 2020-11-20T12:07:10.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/20' 2020-11-20T12:07:10.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/12' 2020-11-20T12:07:10.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/10' 2020-11-20T12:07:10.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/11' 2020-11-20T12:07:10.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/16' 2020-11-20T12:07:10.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/14' 2020-11-20T12:07:10.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/18' 2020-11-20T12:07:10.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/13' 2020-11-20T12:07:10.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/114/17' 2020-11-20T12:07:10.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/114' 2020-11-20T12:07:10.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/19' 2020-11-20T12:07:10.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/15' 2020-11-20T12:07:10.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/20' 2020-11-20T12:07:10.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/12' 2020-11-20T12:07:10.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/10' 2020-11-20T12:07:10.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/11' 2020-11-20T12:07:10.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/16' 2020-11-20T12:07:10.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/14' 2020-11-20T12:07:10.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/18' 2020-11-20T12:07:10.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/13' 2020-11-20T12:07:10.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/159/17' 2020-11-20T12:07:10.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/159' 2020-11-20T12:07:10.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/19' 2020-11-20T12:07:10.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/15' 2020-11-20T12:07:10.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/20' 2020-11-20T12:07:10.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/12' 2020-11-20T12:07:10.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/10' 2020-11-20T12:07:10.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/11' 2020-11-20T12:07:10.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/16' 2020-11-20T12:07:10.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/14' 2020-11-20T12:07:10.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/18' 2020-11-20T12:07:10.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/13' 2020-11-20T12:07:10.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/187/17' 2020-11-20T12:07:10.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/187' 2020-11-20T12:07:10.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/19' 2020-11-20T12:07:10.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/15' 2020-11-20T12:07:10.511 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/20' 2020-11-20T12:07:10.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/12' 2020-11-20T12:07:10.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/10' 2020-11-20T12:07:10.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/11' 2020-11-20T12:07:10.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/16' 2020-11-20T12:07:10.512 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/14' 2020-11-20T12:07:10.513 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/18' 2020-11-20T12:07:10.513 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/13' 2020-11-20T12:07:10.513 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/185/17' 2020-11-20T12:07:10.513 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/185' 2020-11-20T12:07:10.513 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/19' 2020-11-20T12:07:10.514 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/15' 2020-11-20T12:07:10.514 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/20' 2020-11-20T12:07:10.514 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/12' 2020-11-20T12:07:10.514 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/10' 2020-11-20T12:07:10.514 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/11' 2020-11-20T12:07:10.515 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/16' 2020-11-20T12:07:10.515 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/14' 2020-11-20T12:07:10.515 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/18' 2020-11-20T12:07:10.515 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/13' 2020-11-20T12:07:10.515 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/120/17' 2020-11-20T12:07:10.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/120' 2020-11-20T12:07:10.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/19' 2020-11-20T12:07:10.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/15' 2020-11-20T12:07:10.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/20' 2020-11-20T12:07:10.516 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/12' 2020-11-20T12:07:10.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/10' 2020-11-20T12:07:10.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/11' 2020-11-20T12:07:10.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/16' 2020-11-20T12:07:10.517 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/14' 2020-11-20T12:07:10.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/18' 2020-11-20T12:07:10.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/13' 2020-11-20T12:07:10.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/110/17' 2020-11-20T12:07:10.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/110' 2020-11-20T12:07:10.518 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/19' 2020-11-20T12:07:10.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/15' 2020-11-20T12:07:10.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/20' 2020-11-20T12:07:10.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/12' 2020-11-20T12:07:10.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/10' 2020-11-20T12:07:10.519 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/11' 2020-11-20T12:07:10.520 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/16' 2020-11-20T12:07:10.841 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:10.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 Processor -- start 2020-11-20T12:07:10.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 -- start start 2020-11-20T12:07:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:10.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc060b40 con 0x7fa6dc13e9d0 2020-11-20T12:07:10.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc13e080 con 0x7fa6dc0669d0 2020-11-20T12:07:10.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6e2108700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc13ee80 con 0x7fa6dc0ae050 2020-11-20T12:07:10.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.989+0000 7fa6dbfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:10.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6dbfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38226/0 (socket says 172.21.15.44:38226) 2020-11-20T12:07:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 learned_addr learned my addr 172.21.15.44:0/1365036326 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:10.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:07:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:10.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:07:11.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:11.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:10.990+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:11.049 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/14' 2020-11-20T12:07:11.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/18' 2020-11-20T12:07:11.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/13' 2020-11-20T12:07:11.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/137/17' 2020-11-20T12:07:11.050 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/137' 2020-11-20T12:07:11.051 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/19' 2020-11-20T12:07:11.051 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/15' 2020-11-20T12:07:11.051 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/20' 2020-11-20T12:07:11.051 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/12' 2020-11-20T12:07:11.052 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/10' 2020-11-20T12:07:11.052 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/11' 2020-11-20T12:07:11.052 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/16' 2020-11-20T12:07:11.052 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/14' 2020-11-20T12:07:11.053 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/18' 2020-11-20T12:07:11.053 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/13' 2020-11-20T12:07:11.053 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/198/17' 2020-11-20T12:07:11.053 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/198' 2020-11-20T12:07:11.054 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/19' 2020-11-20T12:07:11.054 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/15' 2020-11-20T12:07:11.054 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/20' 2020-11-20T12:07:11.054 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/12' 2020-11-20T12:07:11.055 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/10' 2020-11-20T12:07:11.055 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/11' 2020-11-20T12:07:11.055 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/16' 2020-11-20T12:07:11.055 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/14' 2020-11-20T12:07:11.055 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/18' 2020-11-20T12:07:11.056 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/13' 2020-11-20T12:07:11.056 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/136/17' 2020-11-20T12:07:11.056 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/136' 2020-11-20T12:07:11.056 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/19' 2020-11-20T12:07:11.057 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/15' 2020-11-20T12:07:11.057 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/20' 2020-11-20T12:07:11.057 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/12' 2020-11-20T12:07:11.057 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/10' 2020-11-20T12:07:11.058 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/11' 2020-11-20T12:07:11.058 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/16' 2020-11-20T12:07:11.058 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/14' 2020-11-20T12:07:11.058 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/18' 2020-11-20T12:07:11.059 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/13' 2020-11-20T12:07:11.059 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/125/17' 2020-11-20T12:07:11.059 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/125' 2020-11-20T12:07:11.059 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/19' 2020-11-20T12:07:11.059 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/15' 2020-11-20T12:07:11.060 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/20' 2020-11-20T12:07:11.060 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/12' 2020-11-20T12:07:11.060 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/10' 2020-11-20T12:07:11.060 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/11' 2020-11-20T12:07:11.060 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/16' 2020-11-20T12:07:11.061 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/14' 2020-11-20T12:07:11.061 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/18' 2020-11-20T12:07:11.061 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/13' 2020-11-20T12:07:11.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/194/17' 2020-11-20T12:07:11.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/194' 2020-11-20T12:07:11.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/19' 2020-11-20T12:07:11.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/15' 2020-11-20T12:07:11.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/20' 2020-11-20T12:07:11.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/12' 2020-11-20T12:07:11.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/10' 2020-11-20T12:07:11.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/11' 2020-11-20T12:07:11.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/16' 2020-11-20T12:07:11.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/14' 2020-11-20T12:07:11.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/18' 2020-11-20T12:07:11.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/13' 2020-11-20T12:07:11.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/119/17' 2020-11-20T12:07:11.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/119' 2020-11-20T12:07:11.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/19' 2020-11-20T12:07:11.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/15' 2020-11-20T12:07:11.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/20' 2020-11-20T12:07:11.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/12' 2020-11-20T12:07:11.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/10' 2020-11-20T12:07:11.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/11' 2020-11-20T12:07:11.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/16' 2020-11-20T12:07:11.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/14' 2020-11-20T12:07:11.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/18' 2020-11-20T12:07:11.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/13' 2020-11-20T12:07:11.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/157/17' 2020-11-20T12:07:11.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/157' 2020-11-20T12:07:11.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/19' 2020-11-20T12:07:11.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/15' 2020-11-20T12:07:11.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/20' 2020-11-20T12:07:11.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/12' 2020-11-20T12:07:11.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/10' 2020-11-20T12:07:11.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/11' 2020-11-20T12:07:11.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/16' 2020-11-20T12:07:11.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/14' 2020-11-20T12:07:11.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/18' 2020-11-20T12:07:11.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/13' 2020-11-20T12:07:11.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/172/17' 2020-11-20T12:07:11.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/172' 2020-11-20T12:07:11.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/19' 2020-11-20T12:07:11.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/15' 2020-11-20T12:07:11.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/20' 2020-11-20T12:07:11.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/12' 2020-11-20T12:07:11.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/10' 2020-11-20T12:07:11.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/11' 2020-11-20T12:07:11.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/16' 2020-11-20T12:07:11.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/14' 2020-11-20T12:07:11.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/18' 2020-11-20T12:07:11.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/13' 2020-11-20T12:07:11.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/186/17' 2020-11-20T12:07:11.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/186' 2020-11-20T12:07:11.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/19' 2020-11-20T12:07:11.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/15' 2020-11-20T12:07:11.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/20' 2020-11-20T12:07:11.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/12' 2020-11-20T12:07:11.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/10' 2020-11-20T12:07:11.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/11' 2020-11-20T12:07:11.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/16' 2020-11-20T12:07:11.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/14' 2020-11-20T12:07:11.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:11.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:07:11.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:07:11.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:11.197 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:07:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:07:11.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.191+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:07:11.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/18' 2020-11-20T12:07:11.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/13' 2020-11-20T12:07:11.481 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/190/17' 2020-11-20T12:07:11.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/190' 2020-11-20T12:07:11.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/19' 2020-11-20T12:07:11.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/15' 2020-11-20T12:07:11.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/20' 2020-11-20T12:07:11.482 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/12' 2020-11-20T12:07:11.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/10' 2020-11-20T12:07:11.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/11' 2020-11-20T12:07:11.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/16' 2020-11-20T12:07:11.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/14' 2020-11-20T12:07:11.483 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/18' 2020-11-20T12:07:11.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/13' 2020-11-20T12:07:11.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/104/17' 2020-11-20T12:07:11.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/104' 2020-11-20T12:07:11.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/19' 2020-11-20T12:07:11.484 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/15' 2020-11-20T12:07:11.485 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/20' 2020-11-20T12:07:11.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/12' 2020-11-20T12:07:11.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/10' 2020-11-20T12:07:11.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/11' 2020-11-20T12:07:11.486 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/16' 2020-11-20T12:07:11.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/14' 2020-11-20T12:07:11.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/18' 2020-11-20T12:07:11.487 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/13' 2020-11-20T12:07:11.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/164/17' 2020-11-20T12:07:11.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/164' 2020-11-20T12:07:11.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/19' 2020-11-20T12:07:11.488 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/15' 2020-11-20T12:07:11.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/20' 2020-11-20T12:07:11.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/12' 2020-11-20T12:07:11.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/10' 2020-11-20T12:07:11.489 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/11' 2020-11-20T12:07:11.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/16' 2020-11-20T12:07:11.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/14' 2020-11-20T12:07:11.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/18' 2020-11-20T12:07:11.490 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/13' 2020-11-20T12:07:11.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/112/17' 2020-11-20T12:07:11.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/112' 2020-11-20T12:07:11.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/19' 2020-11-20T12:07:11.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/15' 2020-11-20T12:07:11.491 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/20' 2020-11-20T12:07:11.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/12' 2020-11-20T12:07:11.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/10' 2020-11-20T12:07:11.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/11' 2020-11-20T12:07:11.492 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/16' 2020-11-20T12:07:11.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/14' 2020-11-20T12:07:11.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/18' 2020-11-20T12:07:11.493 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/13' 2020-11-20T12:07:11.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/100/17' 2020-11-20T12:07:11.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/100' 2020-11-20T12:07:11.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/19' 2020-11-20T12:07:11.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/15' 2020-11-20T12:07:11.494 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/20' 2020-11-20T12:07:11.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/12' 2020-11-20T12:07:11.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/10' 2020-11-20T12:07:11.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/11' 2020-11-20T12:07:11.495 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/16' 2020-11-20T12:07:11.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/14' 2020-11-20T12:07:11.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/18' 2020-11-20T12:07:11.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/13' 2020-11-20T12:07:11.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/134/17' 2020-11-20T12:07:11.496 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/134' 2020-11-20T12:07:11.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/19' 2020-11-20T12:07:11.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/15' 2020-11-20T12:07:11.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/20' 2020-11-20T12:07:11.497 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/12' 2020-11-20T12:07:11.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/10' 2020-11-20T12:07:11.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/11' 2020-11-20T12:07:11.498 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/16' 2020-11-20T12:07:11.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/14' 2020-11-20T12:07:11.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/18' 2020-11-20T12:07:11.499 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/13' 2020-11-20T12:07:11.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/115/17' 2020-11-20T12:07:11.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/115' 2020-11-20T12:07:11.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/19' 2020-11-20T12:07:11.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/15' 2020-11-20T12:07:11.500 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/20' 2020-11-20T12:07:11.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/12' 2020-11-20T12:07:11.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/10' 2020-11-20T12:07:11.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/11' 2020-11-20T12:07:11.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/16' 2020-11-20T12:07:11.501 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/14' 2020-11-20T12:07:11.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/18' 2020-11-20T12:07:11.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/13' 2020-11-20T12:07:11.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/116/17' 2020-11-20T12:07:11.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/116' 2020-11-20T12:07:11.502 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/19' 2020-11-20T12:07:11.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/15' 2020-11-20T12:07:11.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/20' 2020-11-20T12:07:11.503 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/12' 2020-11-20T12:07:11.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/10' 2020-11-20T12:07:11.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/11' 2020-11-20T12:07:11.504 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/16' 2020-11-20T12:07:11.505 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/14' 2020-11-20T12:07:11.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/18' 2020-11-20T12:07:11.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/13' 2020-11-20T12:07:11.506 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/195/17' 2020-11-20T12:07:11.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/195' 2020-11-20T12:07:11.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/19' 2020-11-20T12:07:11.507 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/15' 2020-11-20T12:07:11.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/20' 2020-11-20T12:07:11.508 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/12' 2020-11-20T12:07:11.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/10' 2020-11-20T12:07:11.509 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/11' 2020-11-20T12:07:11.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/16' 2020-11-20T12:07:11.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/14' 2020-11-20T12:07:11.510 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/18' 2020-11-20T12:07:11.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:11.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:11.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:11.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:07:11.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:07:11.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:07:11.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:11.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:11.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:11.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:07:11.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:11.592+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:07:12.020 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/13' 2020-11-20T12:07:12.020 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/138/17' 2020-11-20T12:07:12.020 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/138' 2020-11-20T12:07:12.021 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/19' 2020-11-20T12:07:12.021 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/15' 2020-11-20T12:07:12.021 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/20' 2020-11-20T12:07:12.021 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/12' 2020-11-20T12:07:12.022 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/10' 2020-11-20T12:07:12.022 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/11' 2020-11-20T12:07:12.022 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/16' 2020-11-20T12:07:12.022 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/14' 2020-11-20T12:07:12.022 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/18' 2020-11-20T12:07:12.023 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/13' 2020-11-20T12:07:12.023 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/135/17' 2020-11-20T12:07:12.023 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/135' 2020-11-20T12:07:12.023 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/19' 2020-11-20T12:07:12.023 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/15' 2020-11-20T12:07:12.024 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/20' 2020-11-20T12:07:12.024 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/12' 2020-11-20T12:07:12.024 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/10' 2020-11-20T12:07:12.024 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/11' 2020-11-20T12:07:12.024 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/16' 2020-11-20T12:07:12.025 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/14' 2020-11-20T12:07:12.025 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/18' 2020-11-20T12:07:12.025 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/13' 2020-11-20T12:07:12.025 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/117/17' 2020-11-20T12:07:12.025 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/117' 2020-11-20T12:07:12.026 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/19' 2020-11-20T12:07:12.026 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/15' 2020-11-20T12:07:12.026 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/20' 2020-11-20T12:07:12.026 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/12' 2020-11-20T12:07:12.026 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/10' 2020-11-20T12:07:12.027 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/11' 2020-11-20T12:07:12.027 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/16' 2020-11-20T12:07:12.027 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/14' 2020-11-20T12:07:12.027 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/18' 2020-11-20T12:07:12.027 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/13' 2020-11-20T12:07:12.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/143/17' 2020-11-20T12:07:12.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/143' 2020-11-20T12:07:12.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/19' 2020-11-20T12:07:12.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/15' 2020-11-20T12:07:12.028 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/20' 2020-11-20T12:07:12.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/12' 2020-11-20T12:07:12.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/10' 2020-11-20T12:07:12.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/11' 2020-11-20T12:07:12.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/16' 2020-11-20T12:07:12.029 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/14' 2020-11-20T12:07:12.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/18' 2020-11-20T12:07:12.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/13' 2020-11-20T12:07:12.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/180/17' 2020-11-20T12:07:12.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/180' 2020-11-20T12:07:12.030 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/19' 2020-11-20T12:07:12.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/15' 2020-11-20T12:07:12.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/20' 2020-11-20T12:07:12.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/12' 2020-11-20T12:07:12.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/10' 2020-11-20T12:07:12.031 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/11' 2020-11-20T12:07:12.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/16' 2020-11-20T12:07:12.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/14' 2020-11-20T12:07:12.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/18' 2020-11-20T12:07:12.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/13' 2020-11-20T12:07:12.032 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/126/17' 2020-11-20T12:07:12.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/126' 2020-11-20T12:07:12.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/19' 2020-11-20T12:07:12.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/15' 2020-11-20T12:07:12.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/20' 2020-11-20T12:07:12.033 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/12' 2020-11-20T12:07:12.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/10' 2020-11-20T12:07:12.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/11' 2020-11-20T12:07:12.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/16' 2020-11-20T12:07:12.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/14' 2020-11-20T12:07:12.034 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/18' 2020-11-20T12:07:12.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/13' 2020-11-20T12:07:12.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/161/17' 2020-11-20T12:07:12.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/161' 2020-11-20T12:07:12.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/19' 2020-11-20T12:07:12.035 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/15' 2020-11-20T12:07:12.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/20' 2020-11-20T12:07:12.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/12' 2020-11-20T12:07:12.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/10' 2020-11-20T12:07:12.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/11' 2020-11-20T12:07:12.036 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/16' 2020-11-20T12:07:12.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/14' 2020-11-20T12:07:12.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/18' 2020-11-20T12:07:12.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/13' 2020-11-20T12:07:12.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/176/17' 2020-11-20T12:07:12.037 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/176' 2020-11-20T12:07:12.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/19' 2020-11-20T12:07:12.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/15' 2020-11-20T12:07:12.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/20' 2020-11-20T12:07:12.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/12' 2020-11-20T12:07:12.038 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/10' 2020-11-20T12:07:12.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/11' 2020-11-20T12:07:12.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/16' 2020-11-20T12:07:12.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/14' 2020-11-20T12:07:12.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/18' 2020-11-20T12:07:12.039 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/13' 2020-11-20T12:07:12.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/111/17' 2020-11-20T12:07:12.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/111' 2020-11-20T12:07:12.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/19' 2020-11-20T12:07:12.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/15' 2020-11-20T12:07:12.040 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/20' 2020-11-20T12:07:12.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/12' 2020-11-20T12:07:12.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/10' 2020-11-20T12:07:12.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/11' 2020-11-20T12:07:12.041 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/16' 2020-11-20T12:07:12.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/14' 2020-11-20T12:07:12.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/18' 2020-11-20T12:07:12.042 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/13' 2020-11-20T12:07:12.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:12.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:12.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:12.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:07:12.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:12.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:12.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:07:12.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:12.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6daffd700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:12.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:07:12.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:12.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:12.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:12.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:07:12.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:12.393+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:07:12.433 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/191/17' 2020-11-20T12:07:12.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/191' 2020-11-20T12:07:12.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/19' 2020-11-20T12:07:12.434 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/15' 2020-11-20T12:07:12.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/20' 2020-11-20T12:07:12.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/12' 2020-11-20T12:07:12.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/10' 2020-11-20T12:07:12.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/11' 2020-11-20T12:07:12.435 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/16' 2020-11-20T12:07:12.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/14' 2020-11-20T12:07:12.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/18' 2020-11-20T12:07:12.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/13' 2020-11-20T12:07:12.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/184/17' 2020-11-20T12:07:12.436 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/184' 2020-11-20T12:07:12.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/19' 2020-11-20T12:07:12.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/15' 2020-11-20T12:07:12.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/20' 2020-11-20T12:07:12.437 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/12' 2020-11-20T12:07:12.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/10' 2020-11-20T12:07:12.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/11' 2020-11-20T12:07:12.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/16' 2020-11-20T12:07:12.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/14' 2020-11-20T12:07:12.438 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/18' 2020-11-20T12:07:12.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/13' 2020-11-20T12:07:12.439 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/192/17' 2020-11-20T12:07:12.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/192' 2020-11-20T12:07:12.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/19' 2020-11-20T12:07:12.440 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/15' 2020-11-20T12:07:12.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/20' 2020-11-20T12:07:12.441 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/12' 2020-11-20T12:07:12.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/10' 2020-11-20T12:07:12.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/11' 2020-11-20T12:07:12.442 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/16' 2020-11-20T12:07:12.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/14' 2020-11-20T12:07:12.443 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/18' 2020-11-20T12:07:12.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/13' 2020-11-20T12:07:12.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/103/17' 2020-11-20T12:07:12.444 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/103' 2020-11-20T12:07:12.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/19' 2020-11-20T12:07:12.445 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/15' 2020-11-20T12:07:12.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/20' 2020-11-20T12:07:12.446 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/12' 2020-11-20T12:07:12.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/10' 2020-11-20T12:07:12.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/11' 2020-11-20T12:07:12.447 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/16' 2020-11-20T12:07:12.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/14' 2020-11-20T12:07:12.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/18' 2020-11-20T12:07:12.448 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/13' 2020-11-20T12:07:12.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/128/17' 2020-11-20T12:07:12.449 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/128' 2020-11-20T12:07:12.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/19' 2020-11-20T12:07:12.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/15' 2020-11-20T12:07:12.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/20' 2020-11-20T12:07:12.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/12' 2020-11-20T12:07:12.450 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/10' 2020-11-20T12:07:12.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/11' 2020-11-20T12:07:12.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/16' 2020-11-20T12:07:12.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/14' 2020-11-20T12:07:12.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/18' 2020-11-20T12:07:12.451 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/13' 2020-11-20T12:07:12.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/144/17' 2020-11-20T12:07:12.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/144' 2020-11-20T12:07:12.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/19' 2020-11-20T12:07:12.452 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/15' 2020-11-20T12:07:12.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/20' 2020-11-20T12:07:12.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/12' 2020-11-20T12:07:12.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/10' 2020-11-20T12:07:12.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/11' 2020-11-20T12:07:12.453 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/16' 2020-11-20T12:07:12.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/14' 2020-11-20T12:07:12.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/18' 2020-11-20T12:07:12.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/13' 2020-11-20T12:07:12.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/146/17' 2020-11-20T12:07:12.454 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/146' 2020-11-20T12:07:12.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/19' 2020-11-20T12:07:12.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/15' 2020-11-20T12:07:12.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/20' 2020-11-20T12:07:12.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/12' 2020-11-20T12:07:12.455 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/10' 2020-11-20T12:07:12.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/11' 2020-11-20T12:07:12.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/16' 2020-11-20T12:07:12.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/14' 2020-11-20T12:07:12.456 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/18' 2020-11-20T12:07:12.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/13' 2020-11-20T12:07:12.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/141/17' 2020-11-20T12:07:12.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/141' 2020-11-20T12:07:12.457 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/19' 2020-11-20T12:07:12.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/15' 2020-11-20T12:07:12.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/20' 2020-11-20T12:07:12.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/12' 2020-11-20T12:07:12.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/10' 2020-11-20T12:07:12.458 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/11' 2020-11-20T12:07:12.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/16' 2020-11-20T12:07:12.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/14' 2020-11-20T12:07:12.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/18' 2020-11-20T12:07:12.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/13' 2020-11-20T12:07:12.459 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/182/17' 2020-11-20T12:07:12.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/182' 2020-11-20T12:07:12.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/19' 2020-11-20T12:07:12.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/15' 2020-11-20T12:07:12.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/20' 2020-11-20T12:07:12.460 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/12' 2020-11-20T12:07:12.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/10' 2020-11-20T12:07:12.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/11' 2020-11-20T12:07:12.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/16' 2020-11-20T12:07:12.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/14' 2020-11-20T12:07:12.461 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/18' 2020-11-20T12:07:12.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/13' 2020-11-20T12:07:12.462 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/107/17' 2020-11-20T12:07:12.889 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/107' 2020-11-20T12:07:12.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/19' 2020-11-20T12:07:12.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/15' 2020-11-20T12:07:12.890 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/20' 2020-11-20T12:07:12.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/12' 2020-11-20T12:07:12.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/10' 2020-11-20T12:07:12.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/11' 2020-11-20T12:07:12.891 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/16' 2020-11-20T12:07:12.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/14' 2020-11-20T12:07:12.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/18' 2020-11-20T12:07:12.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/13' 2020-11-20T12:07:12.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/178/17' 2020-11-20T12:07:12.892 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/178' 2020-11-20T12:07:12.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/19' 2020-11-20T12:07:12.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/15' 2020-11-20T12:07:12.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/20' 2020-11-20T12:07:12.893 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/12' 2020-11-20T12:07:12.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/10' 2020-11-20T12:07:12.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/11' 2020-11-20T12:07:12.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/16' 2020-11-20T12:07:12.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/14' 2020-11-20T12:07:12.894 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/18' 2020-11-20T12:07:12.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/13' 2020-11-20T12:07:12.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/156/17' 2020-11-20T12:07:12.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/156' 2020-11-20T12:07:12.895 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/19' 2020-11-20T12:07:12.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/15' 2020-11-20T12:07:12.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/20' 2020-11-20T12:07:12.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/12' 2020-11-20T12:07:12.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/10' 2020-11-20T12:07:12.896 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/11' 2020-11-20T12:07:12.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/16' 2020-11-20T12:07:12.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/14' 2020-11-20T12:07:12.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/18' 2020-11-20T12:07:12.897 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/13' 2020-11-20T12:07:12.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/131/17' 2020-11-20T12:07:12.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/131' 2020-11-20T12:07:12.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/19' 2020-11-20T12:07:12.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/15' 2020-11-20T12:07:12.898 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/20' 2020-11-20T12:07:12.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/12' 2020-11-20T12:07:12.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/10' 2020-11-20T12:07:12.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/11' 2020-11-20T12:07:12.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/16' 2020-11-20T12:07:12.899 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/14' 2020-11-20T12:07:12.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/18' 2020-11-20T12:07:12.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/13' 2020-11-20T12:07:12.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/123/17' 2020-11-20T12:07:12.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/123' 2020-11-20T12:07:12.900 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/19' 2020-11-20T12:07:12.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/15' 2020-11-20T12:07:12.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/20' 2020-11-20T12:07:12.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/12' 2020-11-20T12:07:12.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/10' 2020-11-20T12:07:12.901 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/11' 2020-11-20T12:07:12.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/16' 2020-11-20T12:07:12.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/14' 2020-11-20T12:07:12.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/18' 2020-11-20T12:07:12.902 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/13' 2020-11-20T12:07:12.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/160/17' 2020-11-20T12:07:12.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/160' 2020-11-20T12:07:12.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/19' 2020-11-20T12:07:12.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/15' 2020-11-20T12:07:12.903 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/20' 2020-11-20T12:07:12.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/12' 2020-11-20T12:07:12.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/10' 2020-11-20T12:07:12.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/11' 2020-11-20T12:07:12.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/16' 2020-11-20T12:07:12.904 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/14' 2020-11-20T12:07:12.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/18' 2020-11-20T12:07:12.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/13' 2020-11-20T12:07:12.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/165/17' 2020-11-20T12:07:12.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/165' 2020-11-20T12:07:12.905 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/19' 2020-11-20T12:07:12.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/15' 2020-11-20T12:07:12.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/20' 2020-11-20T12:07:12.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/12' 2020-11-20T12:07:12.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/10' 2020-11-20T12:07:12.906 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/11' 2020-11-20T12:07:12.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/16' 2020-11-20T12:07:12.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/14' 2020-11-20T12:07:12.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/18' 2020-11-20T12:07:12.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/13' 2020-11-20T12:07:12.907 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/101/17' 2020-11-20T12:07:12.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/101' 2020-11-20T12:07:12.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/19' 2020-11-20T12:07:12.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/15' 2020-11-20T12:07:12.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/20' 2020-11-20T12:07:12.908 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/12' 2020-11-20T12:07:12.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/10' 2020-11-20T12:07:12.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/11' 2020-11-20T12:07:12.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/16' 2020-11-20T12:07:12.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/14' 2020-11-20T12:07:12.909 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/18' 2020-11-20T12:07:12.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/13' 2020-11-20T12:07:12.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/148/17' 2020-11-20T12:07:12.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/148' 2020-11-20T12:07:12.910 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/19' 2020-11-20T12:07:12.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/15' 2020-11-20T12:07:12.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/20' 2020-11-20T12:07:12.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/12' 2020-11-20T12:07:12.911 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/10' 2020-11-20T12:07:12.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/11' 2020-11-20T12:07:12.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/16' 2020-11-20T12:07:12.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/14' 2020-11-20T12:07:12.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/18' 2020-11-20T12:07:12.912 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/13' 2020-11-20T12:07:12.913 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/168/17' 2020-11-20T12:07:12.913 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/168' 2020-11-20T12:07:13.212 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/19' 2020-11-20T12:07:13.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/15' 2020-11-20T12:07:13.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/20' 2020-11-20T12:07:13.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/12' 2020-11-20T12:07:13.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/10' 2020-11-20T12:07:13.213 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/11' 2020-11-20T12:07:13.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/16' 2020-11-20T12:07:13.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/14' 2020-11-20T12:07:13.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/18' 2020-11-20T12:07:13.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/13' 2020-11-20T12:07:13.214 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/179/17' 2020-11-20T12:07:13.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/179' 2020-11-20T12:07:13.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/19' 2020-11-20T12:07:13.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/15' 2020-11-20T12:07:13.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/20' 2020-11-20T12:07:13.215 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/12' 2020-11-20T12:07:13.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/10' 2020-11-20T12:07:13.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/11' 2020-11-20T12:07:13.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/16' 2020-11-20T12:07:13.216 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/14' 2020-11-20T12:07:13.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/18' 2020-11-20T12:07:13.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/13' 2020-11-20T12:07:13.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/132/17' 2020-11-20T12:07:13.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/132' 2020-11-20T12:07:13.217 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/19' 2020-11-20T12:07:13.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/15' 2020-11-20T12:07:13.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/20' 2020-11-20T12:07:13.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/12' 2020-11-20T12:07:13.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/10' 2020-11-20T12:07:13.218 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/11' 2020-11-20T12:07:13.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/16' 2020-11-20T12:07:13.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/14' 2020-11-20T12:07:13.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/18' 2020-11-20T12:07:13.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/13' 2020-11-20T12:07:13.219 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/199/17' 2020-11-20T12:07:13.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/199' 2020-11-20T12:07:13.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/19' 2020-11-20T12:07:13.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/15' 2020-11-20T12:07:13.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/20' 2020-11-20T12:07:13.220 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/12' 2020-11-20T12:07:13.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/10' 2020-11-20T12:07:13.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/11' 2020-11-20T12:07:13.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/16' 2020-11-20T12:07:13.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/14' 2020-11-20T12:07:13.221 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/18' 2020-11-20T12:07:13.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/13' 2020-11-20T12:07:13.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/163/17' 2020-11-20T12:07:13.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/163' 2020-11-20T12:07:13.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/19' 2020-11-20T12:07:13.222 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/15' 2020-11-20T12:07:13.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/20' 2020-11-20T12:07:13.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/12' 2020-11-20T12:07:13.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/10' 2020-11-20T12:07:13.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/11' 2020-11-20T12:07:13.223 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/16' 2020-11-20T12:07:13.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/14' 2020-11-20T12:07:13.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/18' 2020-11-20T12:07:13.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/13' 2020-11-20T12:07:13.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/153/17' 2020-11-20T12:07:13.224 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/153' 2020-11-20T12:07:13.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/19' 2020-11-20T12:07:13.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/15' 2020-11-20T12:07:13.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/20' 2020-11-20T12:07:13.225 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/12' 2020-11-20T12:07:13.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/10' 2020-11-20T12:07:13.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/11' 2020-11-20T12:07:13.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/16' 2020-11-20T12:07:13.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/14' 2020-11-20T12:07:13.226 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/18' 2020-11-20T12:07:13.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/13' 2020-11-20T12:07:13.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/122/17' 2020-11-20T12:07:13.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/122' 2020-11-20T12:07:13.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/19' 2020-11-20T12:07:13.227 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/15' 2020-11-20T12:07:13.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/20' 2020-11-20T12:07:13.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/12' 2020-11-20T12:07:13.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/10' 2020-11-20T12:07:13.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/11' 2020-11-20T12:07:13.228 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/16' 2020-11-20T12:07:13.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/14' 2020-11-20T12:07:13.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/18' 2020-11-20T12:07:13.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/13' 2020-11-20T12:07:13.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/162/17' 2020-11-20T12:07:13.229 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/162' 2020-11-20T12:07:13.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/19' 2020-11-20T12:07:13.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/15' 2020-11-20T12:07:13.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/20' 2020-11-20T12:07:13.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/12' 2020-11-20T12:07:13.230 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/10' 2020-11-20T12:07:13.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/11' 2020-11-20T12:07:13.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/16' 2020-11-20T12:07:13.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/14' 2020-11-20T12:07:13.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/18' 2020-11-20T12:07:13.231 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/13' 2020-11-20T12:07:13.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/197/17' 2020-11-20T12:07:13.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/197' 2020-11-20T12:07:13.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/19' 2020-11-20T12:07:13.232 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/15' 2020-11-20T12:07:13.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/20' 2020-11-20T12:07:13.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/12' 2020-11-20T12:07:13.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/10' 2020-11-20T12:07:13.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/11' 2020-11-20T12:07:13.233 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/16' 2020-11-20T12:07:13.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/14' 2020-11-20T12:07:13.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/18' 2020-11-20T12:07:13.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/13' 2020-11-20T12:07:13.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/113/17' 2020-11-20T12:07:13.234 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/113' 2020-11-20T12:07:13.235 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/19' 2020-11-20T12:07:13.734 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/15' 2020-11-20T12:07:13.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/20' 2020-11-20T12:07:13.735 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/12' 2020-11-20T12:07:13.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/10' 2020-11-20T12:07:13.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/11' 2020-11-20T12:07:13.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/16' 2020-11-20T12:07:13.736 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/14' 2020-11-20T12:07:13.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/18' 2020-11-20T12:07:13.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/13' 2020-11-20T12:07:13.737 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/152/17' 2020-11-20T12:07:13.738 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/152' 2020-11-20T12:07:13.738 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/19' 2020-11-20T12:07:13.738 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/15' 2020-11-20T12:07:13.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/20' 2020-11-20T12:07:13.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/12' 2020-11-20T12:07:13.739 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/10' 2020-11-20T12:07:13.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/11' 2020-11-20T12:07:13.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/16' 2020-11-20T12:07:13.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/14' 2020-11-20T12:07:13.740 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/18' 2020-11-20T12:07:13.741 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/13' 2020-11-20T12:07:13.741 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/121/17' 2020-11-20T12:07:13.741 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/121' 2020-11-20T12:07:13.742 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/19' 2020-11-20T12:07:13.742 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/15' 2020-11-20T12:07:13.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/20' 2020-11-20T12:07:13.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/12' 2020-11-20T12:07:13.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/10' 2020-11-20T12:07:13.743 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/11' 2020-11-20T12:07:13.744 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/16' 2020-11-20T12:07:13.744 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/14' 2020-11-20T12:07:13.744 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/18' 2020-11-20T12:07:13.745 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/13' 2020-11-20T12:07:13.745 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/118/17' 2020-11-20T12:07:13.745 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/118' 2020-11-20T12:07:13.746 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/19' 2020-11-20T12:07:13.746 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/15' 2020-11-20T12:07:13.746 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/20' 2020-11-20T12:07:13.747 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/12' 2020-11-20T12:07:13.747 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/10' 2020-11-20T12:07:13.747 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/11' 2020-11-20T12:07:13.748 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/16' 2020-11-20T12:07:13.748 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/14' 2020-11-20T12:07:13.748 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/18' 2020-11-20T12:07:13.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/13' 2020-11-20T12:07:13.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/196/17' 2020-11-20T12:07:13.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/196' 2020-11-20T12:07:13.749 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/19' 2020-11-20T12:07:13.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/15' 2020-11-20T12:07:13.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/20' 2020-11-20T12:07:13.750 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/12' 2020-11-20T12:07:13.751 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/10' 2020-11-20T12:07:13.751 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/11' 2020-11-20T12:07:13.751 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/16' 2020-11-20T12:07:13.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/14' 2020-11-20T12:07:13.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/18' 2020-11-20T12:07:13.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/13' 2020-11-20T12:07:13.752 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/124/17' 2020-11-20T12:07:13.753 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/124' 2020-11-20T12:07:13.753 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/19' 2020-11-20T12:07:13.753 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/15' 2020-11-20T12:07:13.754 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/20' 2020-11-20T12:07:13.754 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/12' 2020-11-20T12:07:13.754 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/10' 2020-11-20T12:07:13.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/11' 2020-11-20T12:07:13.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/16' 2020-11-20T12:07:13.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/14' 2020-11-20T12:07:13.755 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/18' 2020-11-20T12:07:13.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/13' 2020-11-20T12:07:13.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/188/17' 2020-11-20T12:07:13.756 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/188' 2020-11-20T12:07:13.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/19' 2020-11-20T12:07:13.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/15' 2020-11-20T12:07:13.757 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/20' 2020-11-20T12:07:13.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/12' 2020-11-20T12:07:13.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/10' 2020-11-20T12:07:13.758 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/11' 2020-11-20T12:07:13.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/16' 2020-11-20T12:07:13.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/14' 2020-11-20T12:07:13.759 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/18' 2020-11-20T12:07:13.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/13' 2020-11-20T12:07:13.760 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/145/17' 2020-11-20T12:07:13.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/145' 2020-11-20T12:07:13.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/19' 2020-11-20T12:07:13.761 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/15' 2020-11-20T12:07:13.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/20' 2020-11-20T12:07:13.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/12' 2020-11-20T12:07:13.762 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/10' 2020-11-20T12:07:13.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/11' 2020-11-20T12:07:13.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/16' 2020-11-20T12:07:13.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/14' 2020-11-20T12:07:13.763 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/18' 2020-11-20T12:07:13.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/13' 2020-11-20T12:07:13.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/130/17' 2020-11-20T12:07:13.764 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/130' 2020-11-20T12:07:13.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/19' 2020-11-20T12:07:13.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/15' 2020-11-20T12:07:13.765 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/20' 2020-11-20T12:07:13.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/12' 2020-11-20T12:07:13.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/10' 2020-11-20T12:07:13.766 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/11' 2020-11-20T12:07:13.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/16' 2020-11-20T12:07:13.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/14' 2020-11-20T12:07:13.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/18' 2020-11-20T12:07:13.767 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/13' 2020-11-20T12:07:13.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/193/17' 2020-11-20T12:07:13.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/193' 2020-11-20T12:07:13.768 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/19' 2020-11-20T12:07:13.769 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/15' 2020-11-20T12:07:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 msgr2=0x7fa6dc060650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6dc0ae050 0x7fa6dc060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 msgr2=0x7fa6dc13e4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.79:3300/0 conn(0x7fa6dc0669d0 0x7fa6dc13e4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 -- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 msgr2=0x7fa6dc137b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 --2- 172.21.15.44:0/1365036326 >> v2:172.21.15.44:3300/0 conn(0x7fa6dc13e9d0 0x7fa6dc137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 -- 172.21.15.44:0/1365036326 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc060d60 con 0x7fa6c8006ff0 2020-11-20T12:07:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.989+0000 7fa6d97fa700 1 -- 172.21.15.44:0/1365036326 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c800bd60 con 0x7fa6c8003360 2020-11-20T12:07:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027458380 0 0) 0x7fa6c800bd60 con 0x7fa6c8003360 2020-11-20T12:07:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6bc003240 con 0x7fa6c8003360 2020-11-20T12:07:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 785129593 0 0) 0x7fa6bc003240 con 0x7fa6c8003360 2020-11-20T12:07:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 >> v1:172.21.15.44:6789/0 conn(0x7fa6c8006ff0 legacy=0x7fa6c80094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c800cf60 con 0x7fa6c8003360 2020-11-20T12:07:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fa6cc00d1c0 con 0x7fa6c8003360 2020-11-20T12:07:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 >> v1:172.21.15.79:6789/0 conn(0x7fa6c8003360 legacy=0x7fa6c80057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6dbfff700 1 -- 172.21.15.44:0/1365036326 reap_dead start 2020-11-20T12:07:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6bc003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc0044e0 0x7fa6bc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc004e50 0x7fa6bc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc005380 con 0x7fa6bc003780 2020-11-20T12:07:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc00bc20 con 0x7fa6bc0044e0 2020-11-20T12:07:14.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc00bde0 con 0x7fa6bc004e50 2020-11-20T12:07:14.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6bc003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.990+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc0044e0 0x7fa6bc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc004e50 0x7fa6bc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc004e50 msgr2=0x7fa6bc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:14.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc004e50 0x7fa6bc00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc0044e0 msgr2=0x7fa6bc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:14.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc0044e0 0x7fa6bc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6db7fe700 1 -- 172.21.15.44:0/1365036326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc003240 con 0x7fa6bc003780 2020-11-20T12:07:14.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6daffd700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc0044e0 0x7fa6bc004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:14.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6bc003b80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa6cc006c00 tx=0x7fa6cc008040).ready entity=mon.0 client_cookie=b5ce345b9268bca0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:14.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00d7c0 con 0x7fa6bc003780 2020-11-20T12:07:14.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc016460 con 0x7fa6bc003780 2020-11-20T12:07:14.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.991+0000 7fa6da7fc700 1 -- 172.21.15.44:0/1365036326 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc016820 con 0x7fa6bc003780 2020-11-20T12:07:14.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 -- 172.21.15.44:0/1365036326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 msgr2=0x7fa6bc003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6bc003b80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa6cc006c00 tx=0x7fa6cc008040).stop 2020-11-20T12:07:14.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 -- 172.21.15.44:0/1365036326 shutdown_connections 2020-11-20T12:07:14.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc004e50 0x7fa6bc00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc0044e0 0x7fa6bc004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 --2- 172.21.15.44:0/1365036326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6bc003b80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 -- 172.21.15.44:0/1365036326 >> 172.21.15.44:0/1365036326 conn(0x7fa6dc062bc0 msgr2=0x7fa6dc062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:14.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 -- 172.21.15.44:0/1365036326 shutdown_connections 2020-11-20T12:07:14.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.992+0000 7fa6e2108700 1 -- 172.21.15.44:0/1365036326 wait complete. 2020-11-20T12:07:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.994+0000 7fa6e2108700 1 Processor -- start 2020-11-20T12:07:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.994+0000 7fa6e2108700 1 -- start start 2020-11-20T12:07:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6e2108700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6dc138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6e2108700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc0044e0 0x7fa6dc136c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6e2108700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc004e50 0x7fa6dc14ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6e2108700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc13deb0 con 0x7fa6bc003780 2020-11-20T12:07:14.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6e2108700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc136560 con 0x7fa6bc004e50 2020-11-20T12:07:14.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6e2108700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc13c830 con 0x7fa6bc0044e0 2020-11-20T12:07:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6dc138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6db7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6dc138a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38252/0 (socket says 172.21.15.44:38252) 2020-11-20T12:07:14.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6db7fe700 1 -- 172.21.15.44:0/2238976869 learned_addr learned my addr 172.21.15.44:0/2238976869 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc004e50 0x7fa6dc14ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6daffd700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc0044e0 0x7fa6dc136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6db7fe700 1 -- 172.21.15.44:0/2238976869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 msgr2=0x7fa6dc138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:07:14.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6db7fe700 1 -- 172.21.15.44:0/2238976869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 msgr2=0x7fa6dc138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:07:14.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6dc138a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:14.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6dc138a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:14.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6dbfff700 1 -- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc0044e0 msgr2=0x7fa6dc136c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc0044e0 0x7fa6dc136c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6dbfff700 1 -- 172.21.15.44:0/2238976869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 msgr2=0x7fa6dc138a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6dc138a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6dbfff700 1 -- 172.21.15.44:0/2238976869 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc136790 con 0x7fa6bc004e50 2020-11-20T12:07:14.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6daffd700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc0044e0 0x7fa6dc136c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:14.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.995+0000 7fa6dbfff700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc004e50 0x7fa6dc14ce20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fa6dc13edb0 tx=0x7fa6d4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:14.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.996+0000 7fa6bb7fe700 1 -- 172.21.15.44:0/2238976869 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40104e0 con 0x7fa6bc004e50 2020-11-20T12:07:14.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.996+0000 7fa6bb7fe700 1 -- 172.21.15.44:0/2238976869 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d400c040 con 0x7fa6bc004e50 2020-11-20T12:07:14.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.996+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc137120 con 0x7fa6bc004e50 2020-11-20T12:07:14.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.996+0000 7fa6bb7fe700 1 -- 172.21.15.44:0/2238976869 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4015b80 con 0x7fa6bc004e50 2020-11-20T12:07:14.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.996+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc1359a0 con 0x7fa6bc004e50 2020-11-20T12:07:14.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.997+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc136920 con 0x7fa6bc004e50 2020-11-20T12:07:14.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.997+0000 7fa6bb7fe700 1 -- 172.21.15.44:0/2238976869 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa6d4028070 con 0x7fa6bc004e50 2020-11-20T12:07:14.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.997+0000 7fa6bb7fe700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa6b4056d50 0x7fa6b4059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.997+0000 7fa6bb7fe700 1 -- 172.21.15.44:0/2238976869 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa6d4070c40 con 0x7fa6bc004e50 2020-11-20T12:07:14.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.997+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa6b4056d50 0x7fa6b4059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:13.998+0000 7fa6db7fe700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa6b4056d50 0x7fa6b4059210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fa6cc013af0 tx=0x7fa6cc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:14.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.001+0000 7fa6bb7fe700 1 -- 172.21.15.44:0/2238976869 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa6d4045f10 con 0x7fa6bc004e50 2020-11-20T12:07:14.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/20' 2020-11-20T12:07:14.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/12' 2020-11-20T12:07:14.062 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/10' 2020-11-20T12:07:14.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/11' 2020-11-20T12:07:14.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/16' 2020-11-20T12:07:14.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/14' 2020-11-20T12:07:14.063 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/18' 2020-11-20T12:07:14.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/13' 2020-11-20T12:07:14.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/175/17' 2020-11-20T12:07:14.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/175' 2020-11-20T12:07:14.064 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/19' 2020-11-20T12:07:14.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/15' 2020-11-20T12:07:14.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/20' 2020-11-20T12:07:14.065 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/12' 2020-11-20T12:07:14.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/10' 2020-11-20T12:07:14.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/11' 2020-11-20T12:07:14.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/16' 2020-11-20T12:07:14.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/14' 2020-11-20T12:07:14.066 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/18' 2020-11-20T12:07:14.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/13' 2020-11-20T12:07:14.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/181/17' 2020-11-20T12:07:14.067 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/181' 2020-11-20T12:07:14.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/19' 2020-11-20T12:07:14.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/15' 2020-11-20T12:07:14.068 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/20' 2020-11-20T12:07:14.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/12' 2020-11-20T12:07:14.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/10' 2020-11-20T12:07:14.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/11' 2020-11-20T12:07:14.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/16' 2020-11-20T12:07:14.069 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/14' 2020-11-20T12:07:14.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/18' 2020-11-20T12:07:14.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/13' 2020-11-20T12:07:14.070 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/127/17' 2020-11-20T12:07:14.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/127' 2020-11-20T12:07:14.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/19' 2020-11-20T12:07:14.071 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/15' 2020-11-20T12:07:14.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/20' 2020-11-20T12:07:14.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/12' 2020-11-20T12:07:14.072 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/10' 2020-11-20T12:07:14.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/11' 2020-11-20T12:07:14.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/16' 2020-11-20T12:07:14.073 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/14' 2020-11-20T12:07:14.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/18' 2020-11-20T12:07:14.074 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/13' 2020-11-20T12:07:14.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/108/17' 2020-11-20T12:07:14.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/108' 2020-11-20T12:07:14.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/19' 2020-11-20T12:07:14.075 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/15' 2020-11-20T12:07:14.076 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/20' 2020-11-20T12:07:14.076 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/12' 2020-11-20T12:07:14.076 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/10' 2020-11-20T12:07:14.077 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/11' 2020-11-20T12:07:14.077 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/16' 2020-11-20T12:07:14.077 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/14' 2020-11-20T12:07:14.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/18' 2020-11-20T12:07:14.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/13' 2020-11-20T12:07:14.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/183/17' 2020-11-20T12:07:14.078 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/183' 2020-11-20T12:07:14.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/19' 2020-11-20T12:07:14.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/15' 2020-11-20T12:07:14.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/20' 2020-11-20T12:07:14.079 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/12' 2020-11-20T12:07:14.080 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/10' 2020-11-20T12:07:14.080 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/11' 2020-11-20T12:07:14.080 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/16' 2020-11-20T12:07:14.080 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/14' 2020-11-20T12:07:14.081 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/18' 2020-11-20T12:07:14.081 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/13' 2020-11-20T12:07:14.081 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/177/17' 2020-11-20T12:07:14.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/177' 2020-11-20T12:07:14.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/19' 2020-11-20T12:07:14.082 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/15' 2020-11-20T12:07:14.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/20' 2020-11-20T12:07:14.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/12' 2020-11-20T12:07:14.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/10' 2020-11-20T12:07:14.083 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/11' 2020-11-20T12:07:14.084 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/16' 2020-11-20T12:07:14.084 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/14' 2020-11-20T12:07:14.084 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/18' 2020-11-20T12:07:14.085 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/13' 2020-11-20T12:07:14.085 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/169/17' 2020-11-20T12:07:14.085 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/169' 2020-11-20T12:07:14.086 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/19' 2020-11-20T12:07:14.086 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/15' 2020-11-20T12:07:14.086 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/20' 2020-11-20T12:07:14.086 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/12' 2020-11-20T12:07:14.087 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/10' 2020-11-20T12:07:14.087 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/11' 2020-11-20T12:07:14.087 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/16' 2020-11-20T12:07:14.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/14' 2020-11-20T12:07:14.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/18' 2020-11-20T12:07:14.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/13' 2020-11-20T12:07:14.088 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/154/17' 2020-11-20T12:07:14.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/154' 2020-11-20T12:07:14.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/19' 2020-11-20T12:07:14.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/15' 2020-11-20T12:07:14.089 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/20' 2020-11-20T12:07:14.090 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/12' 2020-11-20T12:07:14.090 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/10' 2020-11-20T12:07:14.090 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/11' 2020-11-20T12:07:14.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/16' 2020-11-20T12:07:14.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/14' 2020-11-20T12:07:14.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/18' 2020-11-20T12:07:14.091 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/13' 2020-11-20T12:07:14.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/171/17' 2020-11-20T12:07:14.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/171' 2020-11-20T12:07:14.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/19' 2020-11-20T12:07:14.092 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/15' 2020-11-20T12:07:14.093 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/20' 2020-11-20T12:07:14.186 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/12' 2020-11-20T12:07:14.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/10' 2020-11-20T12:07:14.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/11' 2020-11-20T12:07:14.187 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/16' 2020-11-20T12:07:14.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/14' 2020-11-20T12:07:14.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/18' 2020-11-20T12:07:14.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/13' 2020-11-20T12:07:14.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/139/17' 2020-11-20T12:07:14.188 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/139' 2020-11-20T12:07:14.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/19' 2020-11-20T12:07:14.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/15' 2020-11-20T12:07:14.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/20' 2020-11-20T12:07:14.189 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/12' 2020-11-20T12:07:14.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/10' 2020-11-20T12:07:14.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/11' 2020-11-20T12:07:14.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/16' 2020-11-20T12:07:14.190 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/14' 2020-11-20T12:07:14.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/18' 2020-11-20T12:07:14.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/13' 2020-11-20T12:07:14.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/102/17' 2020-11-20T12:07:14.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/102' 2020-11-20T12:07:14.191 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/19' 2020-11-20T12:07:14.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/15' 2020-11-20T12:07:14.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/20' 2020-11-20T12:07:14.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/12' 2020-11-20T12:07:14.192 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/10' 2020-11-20T12:07:14.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/11' 2020-11-20T12:07:14.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/16' 2020-11-20T12:07:14.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/14' 2020-11-20T12:07:14.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/18' 2020-11-20T12:07:14.193 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/13' 2020-11-20T12:07:14.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed './copyofsnap1/test1/129/17' 2020-11-20T12:07:14.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1/129' 2020-11-20T12:07:14.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1/test1' 2020-11-20T12:07:14.194 INFO:tasks.workunit.client.0.smithi044.stdout:removed directory './copyofsnap1' 2020-11-20T12:07:14.195 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:07:14.195 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:07:14.195 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:07:14.237 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2020-11-20T12:07:14.237 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2020-11-20T12:07:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.275+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa6dc0a50c0 con 0x7fa6bc004e50 2020-11-20T12:07:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.275+0000 7fa6bb7fe700 1 -- 172.21.15.44:0/2238976869 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fa6d4026070 con 0x7fa6bc004e50 2020-11-20T12:07:14.280 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:14.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa6b4056d50 msgr2=0x7fa6b4059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa6b4056d50 0x7fa6b4059210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fa6cc013af0 tx=0x7fa6cc008040).stop 2020-11-20T12:07:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc004e50 msgr2=0x7fa6dc14ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc004e50 0x7fa6dc14ce20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fa6dc13edb0 tx=0x7fa6d4008040).stop 2020-11-20T12:07:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 shutdown_connections 2020-11-20T12:07:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa6b4056d50 0x7fa6b4059210 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa6bc004e50 0x7fa6dc14ce20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6bc003780 0x7fa6dc138a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 --2- 172.21.15.44:0/2238976869 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6bc0044e0 0x7fa6dc136c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 >> 172.21.15.44:0/2238976869 conn(0x7fa6dc062bc0 msgr2=0x7fa6dc134900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:14.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 shutdown_connections 2020-11-20T12:07:14.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:14.277+0000 7fa6e2108700 1 -- 172.21.15.44:0/2238976869 wait complete. 2020-11-20T12:07:14.301 INFO:tasks.workunit.client.0.smithi044.stderr:+ set -e 2020-11-20T12:07:14.302 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:07:14.426 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.422+0000 7f2c0d190700 1 Processor -- start 2020-11-20T12:07:14.427 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.422+0000 7f2c0d190700 1 -- start start 2020-11-20T12:07:14.428 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.423+0000 7f2c0d190700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 0x7f2c080a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.428 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.423+0000 7f2c0d190700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c080a8ac0 con 0x7f2c080604d0 2020-11-20T12:07:14.428 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.423+0000 7f2c0d190700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c080a6d10 con 0x7f2c080a7050 2020-11-20T12:07:14.429 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.423+0000 7f2c0d190700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c080a8690 con 0x7f2c080600f0 2020-11-20T12:07:14.429 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.424+0000 7f2bfffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2c080604d0 0x7f2c080a7870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42010/0 (socket says 172.21.15.44:42010) 2020-11-20T12:07:14.429 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.424+0000 7f2bfffff700 1 -- 172.21.15.44:0/729546583 learned_addr learned my addr 172.21.15.44:0/729546583 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:14.429 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.424+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/729546583 >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 0x7f2c080a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.430 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.424+0000 7f2c06d9d700 1 -- 172.21.15.44:0/729546583 >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 msgr2=0x7f2c080a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:14.430 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.424+0000 7f2c06d9d700 1 -- 172.21.15.44:0/729546583 >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 msgr2=0x7f2c080a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:14.430 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.424+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/729546583 >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 0x7f2c080a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:14.430 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.424+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/729546583 >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 0x7f2c080a39a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:14.434 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769878015 0 0) 0x7f2c080a6d10 con 0x7f2c080a7050 2020-11-20T12:07:14.434 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be8003ad0 con 0x7f2c080a7050 2020-11-20T12:07:14.434 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 670624510 0 0) 0x7f2be8003ad0 con 0x7f2c080a7050 2020-11-20T12:07:14.434 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 msgr2=0x7f2c080a39a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:14.435 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 --2- 172.21.15.44:0/729546583 >> v2:172.21.15.44:3300/0 conn(0x7f2c080600f0 0x7f2c080a39a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.435 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 >> v1:172.21.15.44:6789/0 conn(0x7f2c080604d0 legacy=0x7f2c080a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.435 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080a8cf0 con 0x7f2c080a7050 2020-11-20T12:07:14.435 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f2c00001860 con 0x7f2c080a7050 2020-11-20T12:07:14.435 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 >> v1:172.21.15.79:6789/0 conn(0x7f2c080a7050 legacy=0x7f2c080a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.436 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2be8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.436 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be80044b0 0x7f2be8004d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.436 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8004930 0x7f2be8005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.436 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080a8690 con 0x7f2be80044b0 2020-11-20T12:07:14.436 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8005750 con 0x7f2be8004930 2020-11-20T12:07:14.437 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.425+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be8005910 con 0x7f2be8003160 2020-11-20T12:07:14.437 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2be8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.437 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c0759e700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8004930 0x7f2be8005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.437 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2bfffff700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be80044b0 0x7f2be8004d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.438 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c06d9d700 1 -- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8004930 msgr2=0x7f2be8005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:14.438 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2be8004930 0x7f2be8005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.438 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c06d9d700 1 -- 172.21.15.44:0/729546583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be80044b0 msgr2=0x7f2be8004d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:14.438 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be80044b0 0x7f2be8004d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.439 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c06d9d700 1 -- 172.21.15.44:0/729546583 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080a8cf0 con 0x7f2be8003160 2020-11-20T12:07:14.439 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2be8003fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2bf8006c00 tx=0x7f2bf8008040).ready entity=mon.2 client_cookie=5c8ac215416e803d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:14.439 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.427+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf800be70 con 0x7f2be8003160 2020-11-20T12:07:14.439 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.427+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf8014a00 con 0x7f2be8003160 2020-11-20T12:07:14.440 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.427+0000 7f2c0659c700 1 -- 172.21.15.44:0/729546583 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8014d80 con 0x7f2be8003160 2020-11-20T12:07:14.440 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.426+0000 7f2c0759e700 1 -- 172.21.15.44:0/729546583 reap_dead start 2020-11-20T12:07:14.441 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.428+0000 7f2c0d190700 1 -- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 msgr2=0x7f2be8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.441 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.428+0000 7f2c0d190700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2be8003fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f2bf8006c00 tx=0x7f2bf8008040).stop 2020-11-20T12:07:14.441 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.428+0000 7f2c0d190700 1 -- 172.21.15.44:0/729546583 shutdown_connections 2020-11-20T12:07:14.442 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.428+0000 7f2c0d190700 1 --2- 172.21.15.44:0/729546583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2be8003fc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.442 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.428+0000 7f2c0d190700 1 -- 172.21.15.44:0/729546583 >> 172.21.15.44:0/729546583 conn(0x7f2c080aced0 msgr2=0x7f2c080ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:14.442 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.429+0000 7f2c0d190700 1 -- 172.21.15.44:0/729546583 shutdown_connections 2020-11-20T12:07:14.442 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.430+0000 7f2c0d190700 1 -- 172.21.15.44:0/729546583 wait complete. 2020-11-20T12:07:14.444 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.431+0000 7f2c0d190700 1 Processor -- start 2020-11-20T12:07:14.444 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.431+0000 7f2c0d190700 1 -- start start 2020-11-20T12:07:14.445 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c0d190700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2c08148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.445 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c0d190700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c080672d0 0x7f2c08064f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.445 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c0d190700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c08065480 0x7f2c0807dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.445 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c0d190700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0814d790 con 0x7f2c08065480 2020-11-20T12:07:14.446 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c0d190700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080645d0 con 0x7f2c080672d0 2020-11-20T12:07:14.446 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c0d190700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c08064870 con 0x7f2be8003160 2020-11-20T12:07:14.446 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c06d9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2c08148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.446 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2bfffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c080672d0 0x7f2c08064f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.446 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c06d9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2c08148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:49710/0 (socket says 172.21.15.44:49710) 2020-11-20T12:07:14.447 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.432+0000 7f2c06d9d700 1 -- 172.21.15.44:0/699061848 learned_addr learned my addr 172.21.15.44:0/699061848 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:14.447 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c06d9d700 1 -- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c080672d0 msgr2=0x7f2c08064f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:14.447 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c080672d0 0x7f2c08064f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:14.447 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c06d9d700 1 -- 172.21.15.44:0/699061848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c08065480 msgr2=0x7f2c0807dcd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:14.447 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c08065480 0x7f2c0807dcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:14.448 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c06d9d700 1 -- 172.21.15.44:0/699061848 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c08064b10 con 0x7f2be8003160 2020-11-20T12:07:14.448 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2bfffff700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c080672d0 0x7f2c08064f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:14.448 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c06d9d700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2c08148720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2be8005aa0 tx=0x7f2bf8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:14.448 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8023070 con 0x7f2be8003160 2020-11-20T12:07:14.449 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf801d420 con 0x7f2be8003160 2020-11-20T12:07:14.449 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c080658c0 con 0x7f2be8003160 2020-11-20T12:07:14.449 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0814c930 con 0x7f2be8003160 2020-11-20T12:07:14.449 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.433+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf801d7c0 con 0x7f2be8003160 2020-11-20T12:07:14.450 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.435+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c08064ca0 con 0x7f2be8003160 2020-11-20T12:07:14.450 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.437+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2bf801dbf0 con 0x7f2be8003160 2020-11-20T12:07:14.450 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.438+0000 7f2bfeffd700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2be4056d20 0x7f2be40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:14.450 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.438+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2bf8075a10 con 0x7f2be8003160 2020-11-20T12:07:14.450 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.438+0000 7f2bfffff700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2be4056d20 0x7f2be40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:14.451 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.439+0000 7f2bfffff700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2be4056d20 0x7f2be40591e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2bf0007620 tx=0x7f2bf0007c40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:14.451 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.440+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2bf804abd0 con 0x7f2be8003160 2020-11-20T12:07:14.824 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.821+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2c0805c590 con 0x7f2be8003160 2020-11-20T12:07:14.909 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:14.906+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v20) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2bf804e1f0 con 0x7f2be8003160 2020-11-20T12:07:15.087 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.084+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2c080446d0 con 0x7f2be8003160 2020-11-20T12:07:15.308 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.304+0000 7f2bfeffd700 1 -- 172.21.15.44:0/699061848 <== mon.2 v2:172.21.15.79:3301/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v21) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2bf804e3d0 con 0x7f2be8003160 2020-11-20T12:07:15.311 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2be4056d20 msgr2=0x7f2be40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:15.311 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2be4056d20 0x7f2be40591e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2bf0007620 tx=0x7f2bf0007c40).stop 2020-11-20T12:07:15.312 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 msgr2=0x7f2c08148720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:15.312 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2c08148720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2be8005aa0 tx=0x7f2bf8008040).stop 2020-11-20T12:07:15.312 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 shutdown_connections 2020-11-20T12:07:15.313 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2be4056d20 0x7f2be40591e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:15.313 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c08065480 0x7f2c0807dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:15.313 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2be8003160 0x7f2c08148720 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:15.314 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 --2- 172.21.15.44:0/699061848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c080672d0 0x7f2c08064f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:15.314 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 >> 172.21.15.44:0/699061848 conn(0x7f2c080aced0 msgr2=0x7f2c080608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:15.314 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 shutdown_connections 2020-11-20T12:07:15.315 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.306+0000 7f2c0d190700 1 -- 172.21.15.44:0/699061848 wait complete. 2020-11-20T12:07:15.315 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:07:15.320 INFO:tasks.workunit.client.0.smithi044.stderr:+ touch foo 2020-11-20T12:07:15.325 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod +x foo 2020-11-20T12:07:15.326 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/s 2020-11-20T12:07:15.354 INFO:tasks.workunit.client.0.smithi044.stderr:+ find .snap/s/foo -executable 2020-11-20T12:07:15.355 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep foo 2020-11-20T12:07:15.360 INFO:tasks.workunit.client.0.smithi044.stdout:.snap/s/foo 2020-11-20T12:07:15.360 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir .snap/s 2020-11-20T12:07:15.368 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm foo 2020-11-20T12:07:15.371 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:07:15.371 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:07:15.372 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:07:15.372 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:07:15.480 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2020-11-20T12:07:15.481 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2020-11-20T12:07:15.530 INFO:tasks.workunit.client.0.smithi044.stderr:+ set -e 2020-11-20T12:07:15.531 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir foo 2020-11-20T12:07:15.535 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:07:15.696 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.692+0000 7f6bde744700 1 Processor -- start 2020-11-20T12:07:15.696 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.692+0000 7f6bde744700 1 -- start start 2020-11-20T12:07:15.699 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bde744700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6bd80609c0 0x7f6bd80aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:15.700 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bde744700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 0x7f6bd80aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:15.700 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bde744700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd80a9db0 con 0x7f6bd813b4f0 2020-11-20T12:07:15.700 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bde744700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd80a8800 con 0x7f6bd80609c0 2020-11-20T12:07:15.700 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bde744700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd80a8aa0 con 0x7f6bd80ab290 2020-11-20T12:07:15.701 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bdcce1700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6bd813b4f0 0x7f6bd813b9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42028/0 (socket says 172.21.15.44:42028) 2020-11-20T12:07:15.701 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bdcce1700 1 -- 172.21.15.44:0/852953535 learned_addr learned my addr 172.21.15.44:0/852953535 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:15.701 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bd7fff700 1 --2- 172.21.15.44:0/852953535 >> v2:172.21.15.79:3300/0 conn(0x7f6bd80609c0 0x7f6bd80aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:15.702 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.693+0000 7f6bd6ffd700 1 -- 172.21.15.44:0/852953535 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367799322 0 0) 0x7f6bd80a9db0 con 0x7f6bd813b4f0 2020-11-20T12:07:15.702 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.694+0000 7f6bd6ffd700 1 -- 172.21.15.44:0/852953535 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc4003240 con 0x7f6bd813b4f0 2020-11-20T12:07:15.702 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.694+0000 7f6bd7fff700 1 -- 172.21.15.44:0/852953535 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 msgr2=0x7f6bd80aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:15.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.694+0000 7f6bd7fff700 1 --2- 172.21.15.44:0/852953535 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 0x7f6bd80aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:15.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.694+0000 7f6bd7fff700 1 -- 172.21.15.44:0/852953535 >> v1:172.21.15.44:6789/0 conn(0x7f6bd813b4f0 legacy=0x7f6bd813b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:15.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.694+0000 7f6bd7fff700 1 -- 172.21.15.44:0/852953535 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd8060da0 con 0x7f6bd80609c0 2020-11-20T12:07:15.703 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.694+0000 7f6bd7fff700 1 --2- 172.21.15.44:0/852953535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80aa1e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6bc800b9a0 tx=0x7f6bc8006df0).ready entity=mon.1 client_cookie=92fc3ba00f6e7e90 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:15.704 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.694+0000 7f6bd6ffd700 1 -- 172.21.15.44:0/852953535 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0001860 con 0x7f6bd80609c0 2020-11-20T12:07:15.704 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.695+0000 7f6bd6ffd700 1 -- 172.21.15.44:0/852953535 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8012460 con 0x7f6bd80609c0 2020-11-20T12:07:15.704 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.695+0000 7f6bd6ffd700 1 -- 172.21.15.44:0/852953535 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80127e0 con 0x7f6bd80609c0 2020-11-20T12:07:15.705 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.696+0000 7f6bde744700 1 -- 172.21.15.44:0/852953535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 msgr2=0x7f6bd80aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:15.705 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.696+0000 7f6bde744700 1 --2- 172.21.15.44:0/852953535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80aa1e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6bc800b9a0 tx=0x7f6bc8006df0).stop 2020-11-20T12:07:15.709 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.696+0000 7f6bde744700 1 -- 172.21.15.44:0/852953535 shutdown_connections 2020-11-20T12:07:15.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.696+0000 7f6bde744700 1 --2- 172.21.15.44:0/852953535 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80aa1e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:15.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.696+0000 7f6bde744700 1 --2- 172.21.15.44:0/852953535 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 0x7f6bd80aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:15.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.696+0000 7f6bde744700 1 -- 172.21.15.44:0/852953535 >> 172.21.15.44:0/852953535 conn(0x7f6bd8064300 msgr2=0x7f6bd80600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:15.711 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.697+0000 7f6bde744700 1 -- 172.21.15.44:0/852953535 shutdown_connections 2020-11-20T12:07:15.711 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.697+0000 7f6bde744700 1 -- 172.21.15.44:0/852953535 wait complete. 2020-11-20T12:07:15.711 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.698+0000 7f6bde744700 1 Processor -- start 2020-11-20T12:07:15.712 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.698+0000 7f6bde744700 1 -- start start 2020-11-20T12:07:15.712 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bde744700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:15.712 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bde744700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 0x7f6bd814a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:15.712 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bde744700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd813b4f0 0x7f6bd8066fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:15.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bde744700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd814a7f0 con 0x7f6bd813b4f0 2020-11-20T12:07:15.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bde744700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd814aa90 con 0x7f6bd80609c0 2020-11-20T12:07:15.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bde744700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd814ad30 con 0x7f6bd80ab290 2020-11-20T12:07:15.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bd7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:15.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bd7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:58568/0 (socket says 172.21.15.44:58568) 2020-11-20T12:07:15.714 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.699+0000 7f6bd7fff700 1 -- 172.21.15.44:0/2356725045 learned_addr learned my addr 172.21.15.44:0/2356725045 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:15.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bdcce1700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd813b4f0 0x7f6bd8066fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:15.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bdcce1700 1 -- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 msgr2=0x7f6bd814a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:15.715 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bdcce1700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 0x7f6bd814a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:15.716 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bdcce1700 1 -- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 msgr2=0x7f6bd80a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:15.716 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bdcce1700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:15.716 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bdcce1700 1 -- 172.21.15.44:0/2356725045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd814a4f0 con 0x7f6bd813b4f0 2020-11-20T12:07:15.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bdcce1700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd813b4f0 0x7f6bd8066fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6bd000ce60 tx=0x7f6bd0001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:15.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bd7fff700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:15.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000ab40 con 0x7f6bd813b4f0 2020-11-20T12:07:15.718 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.700+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd000aca0 con 0x7f6bd813b4f0 2020-11-20T12:07:15.718 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.701+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0007380 con 0x7f6bd813b4f0 2020-11-20T12:07:15.718 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.701+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd80a3130 con 0x7f6bd813b4f0 2020-11-20T12:07:15.718 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.701+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd80674a0 con 0x7f6bd813b4f0 2020-11-20T12:07:15.719 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.702+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd814aef0 con 0x7f6bd813b4f0 2020-11-20T12:07:15.719 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.702+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f6bd0010070 con 0x7f6bd813b4f0 2020-11-20T12:07:15.719 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.703+0000 7f6bb7fff700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6bb0056d50 0x7f6bb0059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:15.720 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.703+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f6bd0070060 con 0x7f6bd813b4f0 2020-11-20T12:07:15.720 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.703+0000 7f6bd7fff700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6bb0056d50 0x7f6bb0059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:15.720 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.703+0000 7f6bd7fff700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6bb0056d50 0x7f6bb0059210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f6bc800c010 tx=0x7f6bd80aa0c0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:15.721 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:15.709+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6bd0045220 con 0x7f6bd813b4f0 2020-11-20T12:07:16.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:16.003+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6bd80626f0 con 0x7f6bd813b4f0 2020-11-20T12:07:16.307 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:16.303+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v22) v1 ==== 167+0+0 (secure 0 0 0) 0x7f6bd0048840 con 0x7f6bd813b4f0 2020-11-20T12:07:16.569 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:16.565+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6bd80446d0 con 0x7f6bd813b4f0 2020-11-20T12:07:17.294 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:17.318 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.314+0000 7f6bb7fff700 1 -- 172.21.15.44:0/2356725045 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v23) v1 ==== 167+0+0 (secure 0 0 0) 0x7f6bd0048a20 con 0x7f6bd813b4f0 2020-11-20T12:07:17.320 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6bb0056d50 msgr2=0x7f6bb0059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.320 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6bb0056d50 0x7f6bb0059210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f6bc800c010 tx=0x7f6bd80aa0c0).stop 2020-11-20T12:07:17.321 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd813b4f0 msgr2=0x7f6bd8066fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.321 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd813b4f0 0x7f6bd8066fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6bd000ce60 tx=0x7f6bd0001e50).stop 2020-11-20T12:07:17.321 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 shutdown_connections 2020-11-20T12:07:17.321 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6bb0056d50 0x7f6bb0059210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.321 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bd813b4f0 0x7f6bd8066fb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.322 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bd80609c0 0x7f6bd80a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.322 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 --2- 172.21.15.44:0/2356725045 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6bd80ab290 0x7f6bd814a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:17.322 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 >> 172.21.15.44:0/2356725045 conn(0x7f6bd8064300 msgr2=0x7f6bd8068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:17.322 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 shutdown_connections 2020-11-20T12:07:17.323 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:17.316+0000 7f6bde744700 1 -- 172.21.15.44:0/2356725045 wait complete. 2020-11-20T12:07:17.323 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:07:17.332 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo x 2020-11-20T12:07:17.334 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:17.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.425+0000 7fc3104d2700 1 Processor -- start 2020-11-20T12:07:17.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.425+0000 7fc3104d2700 1 -- start start 2020-11-20T12:07:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.425+0000 7fc3104d2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3080aa020 0x7fc3080a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc3104d2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc3080a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc3104d2700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3080a9770 con 0x7fc30813ac60 2020-11-20T12:07:17.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc3104d2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3080a6da0 con 0x7fc3080aa020 2020-11-20T12:07:17.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc3104d2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3080a7040 con 0x7fc3080aa400 2020-11-20T12:07:17.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3080aa020 0x7fc3080a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3080aa020 0x7fc3080a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38294/0 (socket says 172.21.15.44:38294) 2020-11-20T12:07:17.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 -- 172.21.15.44:0/3935968504 learned_addr learned my addr 172.21.15.44:0/3935968504 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30da6d700 1 --2- 172.21.15.44:0/3935968504 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc3080a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 -- 172.21.15.44:0/3935968504 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 msgr2=0x7fc3080a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 --2- 172.21.15.44:0/3935968504 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc3080a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 -- 172.21.15.44:0/3935968504 >> v1:172.21.15.79:6789/0 conn(0x7fc30813ac60 legacy=0x7fc3080a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 -- 172.21.15.44:0/3935968504 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3080a6a90 con 0x7fc3080aa020 2020-11-20T12:07:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30da6d700 1 --2- 172.21.15.44:0/3935968504 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc3080a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30e26e700 1 --2- 172.21.15.44:0/3935968504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc3080a9280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc300009d50 tx=0x7fc300009290).ready entity=mon.0 client_cookie=9f186648f9675ad6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30d26c700 1 -- 172.21.15.44:0/3935968504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000bc80 con 0x7fc3080aa020 2020-11-20T12:07:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30d26c700 1 -- 172.21.15.44:0/3935968504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000bde0 con 0x7fc3080aa020 2020-11-20T12:07:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.426+0000 7fc30d26c700 1 -- 172.21.15.44:0/3935968504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300011610 con 0x7fc3080aa020 2020-11-20T12:07:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 -- 172.21.15.44:0/3935968504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 msgr2=0x7fc3080a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 --2- 172.21.15.44:0/3935968504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc3080a9280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc300009d50 tx=0x7fc300009290).stop 2020-11-20T12:07:17.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 -- 172.21.15.44:0/3935968504 shutdown_connections 2020-11-20T12:07:17.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 --2- 172.21.15.44:0/3935968504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc3080a9280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 --2- 172.21.15.44:0/3935968504 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc3080a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 -- 172.21.15.44:0/3935968504 >> 172.21.15.44:0/3935968504 conn(0x7fc308062690 msgr2=0x7fc3080a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:17.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 -- 172.21.15.44:0/3935968504 shutdown_connections 2020-11-20T12:07:17.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.427+0000 7fc3104d2700 1 -- 172.21.15.44:0/3935968504 wait complete. 2020-11-20T12:07:17.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 Processor -- start 2020-11-20T12:07:17.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 -- start start 2020-11-20T12:07:17.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc308060330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc308149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc30813ac60 0x7fc3080a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3081478e0 con 0x7fc3080aa020 2020-11-20T12:07:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30814a2b0 con 0x7fc30813ac60 2020-11-20T12:07:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.428+0000 7fc3104d2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30814a540 con 0x7fc3080aa400 2020-11-20T12:07:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc308060330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc308060330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38300/0 (socket says 172.21.15.44:38300) 2020-11-20T12:07:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 -- 172.21.15.44:0/2480881819 learned_addr learned my addr 172.21.15.44:0/2480881819 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30da6d700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc308149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30ea6f700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc30813ac60 0x7fc3080a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 -- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 msgr2=0x7fc308149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc308149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 -- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc30813ac60 msgr2=0x7fc3080a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc30813ac60 0x7fc3080a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 -- 172.21.15.44:0/2480881819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30814a7e0 con 0x7fc3080aa020 2020-11-20T12:07:17.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc30e26e700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc308060330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc3000092d0 tx=0x7fc300016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc3067fc700 1 -- 172.21.15.44:0/2480881819 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30001b5e0 con 0x7fc3080aa020 2020-11-20T12:07:17.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc3067fc700 1 -- 172.21.15.44:0/2480881819 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30001b740 con 0x7fc3080aa020 2020-11-20T12:07:17.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc3067fc700 1 -- 172.21.15.44:0/2480881819 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30001bae0 con 0x7fc3080aa020 2020-11-20T12:07:17.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30814aa80 con 0x7fc3080aa020 2020-11-20T12:07:17.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.429+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3080ac5d0 con 0x7fc3080aa020 2020-11-20T12:07:17.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.430+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc308061650 con 0x7fc3080aa020 2020-11-20T12:07:17.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.430+0000 7fc3067fc700 1 -- 172.21.15.44:0/2480881819 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc300017070 con 0x7fc3080aa020 2020-11-20T12:07:17.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.431+0000 7fc3067fc700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc2e0056d20 0x7fc2e00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.431+0000 7fc3067fc700 1 -- 172.21.15.44:0/2480881819 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc300071e80 con 0x7fc3080aa020 2020-11-20T12:07:17.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.431+0000 7fc30da6d700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc2e0056d20 0x7fc2e00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.431+0000 7fc30da6d700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc2e0056d20 0x7fc2e00591e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc2fc00ba80 tx=0x7fc2fc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.434+0000 7fc3067fc700 1 -- 172.21.15.44:0/2480881819 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc300047210 con 0x7fc3080aa020 2020-11-20T12:07:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.682+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc3080446d0 con 0x7fc3080aa020 2020-11-20T12:07:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.683+0000 7fc3067fc700 1 -- 172.21.15.44:0/2480881819 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7fc300015070 con 0x7fc3080aa020 2020-11-20T12:07:17.687 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc2e0056d20 msgr2=0x7fc2e00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc2e0056d20 0x7fc2e00591e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc2fc00ba80 tx=0x7fc2fc008040).stop 2020-11-20T12:07:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 msgr2=0x7fc308060330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc308060330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc3000092d0 tx=0x7fc300016000).stop 2020-11-20T12:07:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 shutdown_connections 2020-11-20T12:07:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc2e0056d20 0x7fc2e00591e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc30813ac60 0x7fc3080a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3080aa020 0x7fc308060330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 --2- 172.21.15.44:0/2480881819 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3080aa400 0x7fc308149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 >> 172.21.15.44:0/2480881819 conn(0x7fc308062690 msgr2=0x7fc3080a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 shutdown_connections 2020-11-20T12:07:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.684+0000 7fc3104d2700 1 -- 172.21.15.44:0/2480881819 wait complete. 2020-11-20T12:07:17.699 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:07:17.699 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:07:17.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.864+0000 7f2f4cdcc700 1 Processor -- start 2020-11-20T12:07:17.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.864+0000 7f2f4cdcc700 1 -- start start 2020-11-20T12:07:17.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f4cdcc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2f4813a990 0x7f2f4813ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f4cdcc700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480a6260 con 0x7f2f4813f800 2020-11-20T12:07:17.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f4cdcc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480a6490 con 0x7f2f48060330 2020-11-20T12:07:17.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f4cdcc700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f480a52e0 con 0x7f2f4813a990 2020-11-20T12:07:17.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f46d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2f4813f800 0x7f2f48141bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42060/0 (socket says 172.21.15.44:42060) 2020-11-20T12:07:17.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f46d9d700 1 -- 172.21.15.44:0/2803331667 learned_addr learned my addr 172.21.15.44:0/2803331667 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:17.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2803331667 >> v2:172.21.15.79:3300/0 conn(0x7f2f4813a990 0x7f2f4813ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846383660 0 0) 0x7f2f480a6260 con 0x7f2f4813f800 2020-11-20T12:07:17.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.865+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c003240 con 0x7f2f4813f800 2020-11-20T12:07:17.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635054315 0 0) 0x7f2f480a6490 con 0x7f2f48060330 2020-11-20T12:07:17.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f480a6260 con 0x7f2f48060330 2020-11-20T12:07:17.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1822588593 0 0) 0x7f2f2c003240 con 0x7f2f4813f800 2020-11-20T12:07:17.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 >> v2:172.21.15.79:3300/0 conn(0x7f2f4813a990 msgr2=0x7f2f4813ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:17.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 --2- 172.21.15.44:0/2803331667 >> v2:172.21.15.79:3300/0 conn(0x7f2f4813a990 0x7f2f4813ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 >> v1:172.21.15.79:6789/0 conn(0x7f2f48060330 legacy=0x7f2f480a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480a6730 con 0x7f2f4813f800 2020-11-20T12:07:17.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f2f40001860 con 0x7f2f4813f800 2020-11-20T12:07:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 >> v1:172.21.15.44:6789/0 conn(0x7f2f4813f800 legacy=0x7f2f48141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f2c003910 0x7f2f2c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0044e0 0x7f2f2c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c004e50 0x7f2f2c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480a52e0 con 0x7f2f2c0044e0 2020-11-20T12:07:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c00bc20 con 0x7f2f2c003910 2020-11-20T12:07:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c00ce70 con 0x7f2f2c004e50 2020-11-20T12:07:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0044e0 0x7f2f2c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4659c700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f2c003910 0x7f2f2c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f46d9d700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c004e50 0x7f2f2c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f45d9b700 1 -- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c004e50 msgr2=0x7f2f2c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c004e50 0x7f2f2c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f45d9b700 1 -- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f2c003910 msgr2=0x7f2f2c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f2c003910 0x7f2f2c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f45d9b700 1 -- 172.21.15.44:0/2803331667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c003240 con 0x7f2f2c0044e0 2020-11-20T12:07:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f46d9d700 1 -- 172.21.15.44:0/2803331667 reap_dead start 2020-11-20T12:07:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0044e0 0x7f2f2c004960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2f3800bc00 tx=0x7f2f38006df0).ready entity=mon.0 client_cookie=3ffadc4f5d412268 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.866+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800b310 con 0x7f2f2c0044e0 2020-11-20T12:07:17.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3800ea30 con 0x7f2f2c0044e0 2020-11-20T12:07:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4559a700 1 -- 172.21.15.44:0/2803331667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800edb0 con 0x7f2f2c0044e0 2020-11-20T12:07:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2803331667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0044e0 msgr2=0x7f2f2c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4cdcc700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0044e0 0x7f2f2c004960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2f3800bc00 tx=0x7f2f38006df0).stop 2020-11-20T12:07:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2803331667 shutdown_connections 2020-11-20T12:07:17.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4cdcc700 1 --2- 172.21.15.44:0/2803331667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f2c0044e0 0x7f2f2c004960 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2803331667 >> 172.21.15.44:0/2803331667 conn(0x7f2f48062690 msgr2=0x7f2f480a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2803331667 shutdown_connections 2020-11-20T12:07:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.867+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2803331667 wait complete. 2020-11-20T12:07:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.868+0000 7f2f4cdcc700 1 Processor -- start 2020-11-20T12:07:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.868+0000 7f2f4cdcc700 1 -- start start 2020-11-20T12:07:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c0044e0 0x7f2f480aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f480acc80 0x7f2f48133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f481493b0 0x7f2f48149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48133520 con 0x7f2f480acc80 2020-11-20T12:07:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48135350 con 0x7f2f481493b0 2020-11-20T12:07:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480ad2b0 con 0x7f2f2c0044e0 2020-11-20T12:07:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f480acc80 0x7f2f48133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f480acc80 0x7f2f48133f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38322/0 (socket says 172.21.15.44:38322) 2020-11-20T12:07:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 -- 172.21.15.44:0/2472647152 learned_addr learned my addr 172.21.15.44:0/2472647152 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4659c700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c0044e0 0x7f2f480aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f46d9d700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f481493b0 0x7f2f48149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c0044e0 msgr2=0x7f2f480aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c0044e0 0x7f2f480aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f481493b0 msgr2=0x7f2f48149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f481493b0 0x7f2f48149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 -- 172.21.15.44:0/2472647152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480ad550 con 0x7f2f480acc80 2020-11-20T12:07:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4659c700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f2c0044e0 0x7f2f480aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:07:17.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f46d9d700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f481493b0 0x7f2f48149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:07:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f45d9b700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f480acc80 0x7f2f48133f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2f38012930 tx=0x7f2f38019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38012e50 con 0x7f2f480acc80 2020-11-20T12:07:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3801d410 con 0x7f2f480acc80 2020-11-20T12:07:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3801d5c0 con 0x7f2f480acc80 2020-11-20T12:07:17.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f48135040 con 0x7f2f480acc80 2020-11-20T12:07:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.869+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480ad9f0 con 0x7f2f480acc80 2020-11-20T12:07:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.870+0000 7f2f4cdcc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f48149d20 0x7f2f4812ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.870+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480ad6e0 con 0x7f2f48149d20 2020-11-20T12:07:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.870+0000 7f2f4659c700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f48149d20 0x7f2f4812ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.870+0000 7f2f4659c700 1 -- 172.21.15.44:0/2472647152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f48061650 con 0x7f2f48149d20 2020-11-20T12:07:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f4659c700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f48149d20 0x7f2f4812ec40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f2f3c0029e0 tx=0x7f2f3c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2f3802d070 con 0x7f2f480acc80 2020-11-20T12:07:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f1c056cc0 0x7f2f1c059180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2f38071e20 con 0x7f2f480acc80 2020-11-20T12:07:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c007020 con 0x7f2f48149d20 2020-11-20T12:07:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f2f3c007180 con 0x7f2f48149d20 2020-11-20T12:07:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f48149d20 msgr2=0x7f2f4812ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f48149d20 0x7f2f4812ec40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f2f3c0029e0 tx=0x7f2f3c009040).stop 2020-11-20T12:07:17.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f48149d20 msgr2=0x7f2f4812ec40 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f367fc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f48149d20 0x7f2f4812ec40 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0x7f2f3c0029e0 tx=0x7f2f3c009040).stop 2020-11-20T12:07:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.871+0000 7f2f46d9d700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f1c056cc0 0x7f2f1c059180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.872+0000 7f2f46d9d700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f1c056cc0 0x7f2f1c059180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f2f48137f70 tx=0x7f2f40009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.879+0000 7f2f4cdcc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f4812f130 0x7f2f4812f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.879+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480446d0 con 0x7f2f4812f130 2020-11-20T12:07:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.879+0000 7f2f4659c700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f4812f130 0x7f2f4812f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.879+0000 7f2f4659c700 1 -- 172.21.15.44:0/2472647152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f480680f0 con 0x7f2f4812f130 2020-11-20T12:07:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4659c700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f4812f130 0x7f2f4812f540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2f3c016000 tx=0x7f2f3c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c006da0 con 0x7f2f4812f130 2020-11-20T12:07:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7f2f3c0107f0 con 0x7f2f4812f130 2020-11-20T12:07:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f4812f130 msgr2=0x7f2f4812f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f367fc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f4812f130 0x7f2f4812f540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2f3c016000 tx=0x7f2f3c019000).stop 2020-11-20T12:07:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f367fc700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f4812f130 msgr2=0x7f2f4812f540 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:17.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f367fc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f4812f130 0x7f2f4812f540 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0x7f2f3c016000 tx=0x7f2f3c019000).stop 2020-11-20T12:07:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f1c056cc0 msgr2=0x7f2f1c059180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f1c056cc0 0x7f2f1c059180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f2f48137f70 tx=0x7f2f40009040).stop 2020-11-20T12:07:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f480acc80 msgr2=0x7f2f48133f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f480acc80 0x7f2f48133f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2f38012930 tx=0x7f2f38019000).stop 2020-11-20T12:07:17.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f46d9d700 1 -- 172.21.15.44:0/2472647152 reap_dead start 2020-11-20T12:07:17.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 shutdown_connections 2020-11-20T12:07:17.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 --2- 172.21.15.44:0/2472647152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f480acc80 0x7f2f48133f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:17.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 >> 172.21.15.44:0/2472647152 conn(0x7f2f48062690 msgr2=0x7f2f481393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:17.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 shutdown_connections 2020-11-20T12:07:17.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:17.880+0000 7f2f4cdcc700 1 -- 172.21.15.44:0/2472647152 wait complete. 2020-11-20T12:07:17.899 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:17.899 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:07:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 38, 2020-11-20T12:07:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:07:17.900 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:07:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:17.901 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:17.902 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:17.903 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:17.904 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:17.905 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:17.906 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:17.907 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:17.909 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:17.910 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.911 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:17.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:17.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:17.915 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:17.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:17.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:17.920 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:17.921 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:17.922 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:17.923 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.924 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:17.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:17.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:17.927 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:17.928 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:07:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:17.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:17.930 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:17.930 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:17.931 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:07:18.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.054+0000 7f20704f1700 1 Processor -- start 2020-11-20T12:07:18.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.054+0000 7f20704f1700 1 -- start start 2020-11-20T12:07:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f20704f1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f20680a3e10 0x7f20680a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f20704f1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20680a9770 con 0x7f20680aa400 2020-11-20T12:07:18.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f20704f1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20680a6da0 con 0x7f20680aa020 2020-11-20T12:07:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f20704f1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20680a56c0 con 0x7f20680a3e10 2020-11-20T12:07:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f206da8c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f20680aa400 0x7f20680a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42082/0 (socket says 172.21.15.44:42082) 2020-11-20T12:07:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f206da8c700 1 -- 172.21.15.44:0/3051450181 learned_addr learned my addr 172.21.15.44:0/3051450181 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f206ea8e700 1 --2- 172.21.15.44:0/3051450181 >> v2:172.21.15.79:3300/0 conn(0x7f20680a3e10 0x7f20680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274239081 0 0) 0x7f2064001450 con 0x7f20680aa400 2020-11-20T12:07:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2050003240 con 0x7f20680aa400 2020-11-20T12:07:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.055+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075700260 0 0) 0x7f20680a6da0 con 0x7f20680aa020 2020-11-20T12:07:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2064001450 con 0x7f20680aa020 2020-11-20T12:07:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206ea8e700 1 -- 172.21.15.44:0/3051450181 >> v1:172.21.15.79:6789/0 conn(0x7f20680aa020 legacy=0x7f20680a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206ea8e700 1 -- 172.21.15.44:0/3051450181 >> v1:172.21.15.44:6789/0 conn(0x7f20680aa400 legacy=0x7f20680a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 238194462 0 0) 0x7f2050003240 con 0x7f20680aa400 2020-11-20T12:07:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206ea8e700 1 -- 172.21.15.44:0/3051450181 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680a7040 con 0x7f20680a3e10 2020-11-20T12:07:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206ea8e700 1 --2- 172.21.15.44:0/3051450181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680a3e10 0x7f20680a9280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f206000b9a0 tx=0x7f2060006df0).ready entity=mon.1 client_cookie=d6ad32262ebd9f53 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640019f0 con 0x7f20680a3e10 2020-11-20T12:07:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060011400 con 0x7f20680a3e10 2020-11-20T12:07:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f206d28b700 1 -- 172.21.15.44:0/3051450181 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060011780 con 0x7f20680a3e10 2020-11-20T12:07:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f20704f1700 1 -- 172.21.15.44:0/3051450181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680a3e10 msgr2=0x7f20680a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f20704f1700 1 --2- 172.21.15.44:0/3051450181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680a3e10 0x7f20680a9280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f206000b9a0 tx=0x7f2060006df0).stop 2020-11-20T12:07:18.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f20704f1700 1 -- 172.21.15.44:0/3051450181 shutdown_connections 2020-11-20T12:07:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f20704f1700 1 --2- 172.21.15.44:0/3051450181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680a3e10 0x7f20680a9280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.056+0000 7f20704f1700 1 -- 172.21.15.44:0/3051450181 >> 172.21.15.44:0/3051450181 conn(0x7f2068062690 msgr2=0x7f20680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.057+0000 7f20704f1700 1 -- 172.21.15.44:0/3051450181 shutdown_connections 2020-11-20T12:07:18.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.057+0000 7f20704f1700 1 -- 172.21.15.44:0/3051450181 wait complete. 2020-11-20T12:07:18.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.057+0000 7f20704f1700 1 Processor -- start 2020-11-20T12:07:18.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.057+0000 7f20704f1700 1 -- start start 2020-11-20T12:07:18.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f20704f1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20680a3e10 0x7f2068065260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f20704f1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680aa020 0x7f2068066600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f20704f1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20680aa400 0x7f20680abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f20704f1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20680a3320 con 0x7f20680a3e10 2020-11-20T12:07:18.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f20704f1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206809d890 con 0x7f20680aa020 2020-11-20T12:07:18.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f20704f1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20680adc40 con 0x7f20680aa400 2020-11-20T12:07:18.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f206e28d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20680a3e10 0x7f2068065260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f206e28d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20680a3e10 0x7f2068065260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38338/0 (socket says 172.21.15.44:38338) 2020-11-20T12:07:18.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f206e28d700 1 -- 172.21.15.44:0/1905591540 learned_addr learned my addr 172.21.15.44:0/1905591540 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:18.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f206ea8e700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20680aa400 0x7f20680abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.058+0000 7f206da8c700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680aa020 0x7f2068066600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f206e28d700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20680aa400 msgr2=0x7f20680abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f206e28d700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20680aa400 0x7f20680abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f206e28d700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680aa020 msgr2=0x7f2068066600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f206e28d700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680aa020 0x7f2068066600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f206e28d700 1 -- 172.21.15.44:0/1905591540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206809d5f0 con 0x7f20680a3e10 2020-11-20T12:07:18.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f206e28d700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20680a3e10 0x7f2068065260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f205800c740 tx=0x7f2058007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800e490 con 0x7f20680a3e10 2020-11-20T12:07:18.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205800e5f0 con 0x7f20680a3e10 2020-11-20T12:07:18.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206813a320 con 0x7f20680a3e10 2020-11-20T12:07:18.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800e990 con 0x7f20680a3e10 2020-11-20T12:07:18.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.059+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2068068080 con 0x7f20680a3e10 2020-11-20T12:07:18.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f20704f1700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2068066120 0x7f20680ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2068066af0 con 0x7f2068066120 2020-11-20T12:07:18.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f206da8c700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2068066120 0x7f20680ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f206da8c700 1 -- 172.21.15.44:0/1905591540 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2068061650 con 0x7f2068066120 2020-11-20T12:07:18.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2058019030 con 0x7f20680a3e10 2020-11-20T12:07:18.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f206da8c700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2068066120 0x7f20680ac120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f20680a9b90 tx=0x7f2064001670).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f20567fc700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2040056d20 0x7f20400591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.060+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f205806f5e0 con 0x7f20680a3e10 2020-11-20T12:07:18.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f206ea8e700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2040056d20 0x7f20400591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400de30 con 0x7f2068066120 2020-11-20T12:07:18.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f20640143b0 con 0x7f2068066120 2020-11-20T12:07:18.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2068066120 msgr2=0x7f20680ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f20567fc700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2068066120 0x7f20680ac120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f20680a9b90 tx=0x7f2064001670).stop 2020-11-20T12:07:18.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2068066120 msgr2=0x7f20680ac120 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:18.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f20567fc700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2068066120 0x7f20680ac120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.061+0000 7f206ea8e700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2040056d20 0x7f20400591e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f206000b150 tx=0x7f2060013000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.069+0000 7f20704f1700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680ac610 0x7f20680aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.069+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20680acf10 con 0x7f20680ac610 2020-11-20T12:07:18.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.069+0000 7f206da8c700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680ac610 0x7f20680aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.069+0000 7f206da8c700 1 -- 172.21.15.44:0/1905591540 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f20680a6a90 con 0x7f20680ac610 2020-11-20T12:07:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.069+0000 7f206da8c700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680ac610 0x7f20680aca20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2064015000 tx=0x7f206400a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064016b40 con 0x7f20680ac610 2020-11-20T12:07:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4775 (secure 0 0 0) 0x7f206400dc20 con 0x7f20680ac610 2020-11-20T12:07:18.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680ac610 msgr2=0x7f20680aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20567fc700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680ac610 0x7f20680aca20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2064015000 tx=0x7f206400a000).stop 2020-11-20T12:07:18.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20567fc700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680ac610 msgr2=0x7f20680aca20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:18.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20567fc700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20680ac610 0x7f20680aca20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2040056d20 msgr2=0x7f20400591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2040056d20 0x7f20400591e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f206000b150 tx=0x7f2060013000).stop 2020-11-20T12:07:18.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20680a3e10 msgr2=0x7f2068065260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 --2- 172.21.15.44:0/1905591540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20680a3e10 0x7f2068065260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f205800c740 tx=0x7f2058007a90).stop 2020-11-20T12:07:18.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f206ea8e700 1 -- 172.21.15.44:0/1905591540 reap_dead start 2020-11-20T12:07:18.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f206ea8e700 1 -- 172.21.15.44:0/1905591540 reap_dead start 2020-11-20T12:07:18.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 shutdown_connections 2020-11-20T12:07:18.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 >> 172.21.15.44:0/1905591540 conn(0x7f2068062690 msgr2=0x7f2068136b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:18.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 shutdown_connections 2020-11-20T12:07:18.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.070+0000 7f20704f1700 1 -- 172.21.15.44:0/1905591540 wait complete. 2020-11-20T12:07:18.083 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:18.084 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:18.084 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:18.084 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:07:18.084 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 38, 2020-11-20T12:07:18.084 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:18.085 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:18.085 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:07:18.085 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:18.085 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.086 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:07:18.086 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:18.086 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:18.086 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:18.086 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:18.087 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:18.087 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:18.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:18.087 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:18.087 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:18.088 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:18.088 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:18.088 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.088 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:18.088 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:18.089 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:18.091 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:18.092 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:18.092 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:18.092 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:18.092 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:18.092 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:18.093 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:18.094 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.095 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:18.096 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:18.097 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:18.097 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:18.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.097 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:18.097 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:18.098 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:18.098 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:18.099 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.099 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.099 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.099 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:18.100 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:18.100 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:18.100 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:18.100 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:18.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.101 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:18.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:18.101 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:18.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:18.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.102 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:18.102 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:18.102 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:18.103 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:18.103 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.104 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.104 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:18.104 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:18.104 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:07:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:07:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:07:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:07:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:18.118 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:18.119 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:07:18.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.243+0000 7fb89baaa700 1 Processor -- start 2020-11-20T12:07:18.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.243+0000 7fb89baaa700 1 -- start start 2020-11-20T12:07:18.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb89baaa700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8940a7810 0x7fb89413c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb89baaa700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8940a8bf0 con 0x7fb894066250 2020-11-20T12:07:18.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb89baaa700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8940a86b0 con 0x7fb8940aa1e0 2020-11-20T12:07:18.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb89baaa700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8940a8950 con 0x7fb8940a7810 2020-11-20T12:07:18.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb899045700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8940a7810 0x7fb89413c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb899045700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8940a7810 0x7fb89413c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38352/0 (socket says 172.21.15.44:38352) 2020-11-20T12:07:18.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb89a047700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb894066250 0x7fb89413c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42102/0 (socket says 172.21.15.44:42102) 2020-11-20T12:07:18.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb899045700 1 -- 172.21.15.44:0/695301443 learned_addr learned my addr 172.21.15.44:0/695301443 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:18.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb899045700 1 -- 172.21.15.44:0/695301443 >> v1:172.21.15.79:6789/0 conn(0x7fb8940aa1e0 legacy=0x7fb894065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb899045700 1 -- 172.21.15.44:0/695301443 >> v1:172.21.15.44:6789/0 conn(0x7fb894066250 legacy=0x7fb89413c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb899045700 1 -- 172.21.15.44:0/695301443 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940aa5c0 con 0x7fb8940a7810 2020-11-20T12:07:18.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.244+0000 7fb898844700 1 -- 172.21.15.44:0/695301443 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227292703 0 0) 0x7fb8940a8bf0 con 0x7fb894066250 2020-11-20T12:07:18.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb899045700 1 --2- 172.21.15.44:0/695301443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8940a7810 0x7fb89413c170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb8940a8880 tx=0x7fb890006eb0).ready entity=mon.0 client_cookie=86e2fefa2619f82b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb898844700 1 -- 172.21.15.44:0/695301443 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8900098d0 con 0x7fb8940a7810 2020-11-20T12:07:18.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb898844700 1 -- 172.21.15.44:0/695301443 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890009a30 con 0x7fb8940a7810 2020-11-20T12:07:18.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb898844700 1 -- 172.21.15.44:0/695301443 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890009db0 con 0x7fb8940a7810 2020-11-20T12:07:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb89baaa700 1 -- 172.21.15.44:0/695301443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8940a7810 msgr2=0x7fb89413c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb89baaa700 1 --2- 172.21.15.44:0/695301443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8940a7810 0x7fb89413c170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb8940a8880 tx=0x7fb890006eb0).stop 2020-11-20T12:07:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb89baaa700 1 -- 172.21.15.44:0/695301443 shutdown_connections 2020-11-20T12:07:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb89baaa700 1 --2- 172.21.15.44:0/695301443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8940a7810 0x7fb89413c170 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb89baaa700 1 -- 172.21.15.44:0/695301443 >> 172.21.15.44:0/695301443 conn(0x7fb894062690 msgr2=0x7fb8940a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb89baaa700 1 -- 172.21.15.44:0/695301443 shutdown_connections 2020-11-20T12:07:18.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.245+0000 7fb89baaa700 1 -- 172.21.15.44:0/695301443 wait complete. 2020-11-20T12:07:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.246+0000 7fb89baaa700 1 Processor -- start 2020-11-20T12:07:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.246+0000 7fb89baaa700 1 -- start start 2020-11-20T12:07:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb89baaa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894066250 0x7fb89413a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb89baaa700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8940a7810 0x7fb8940a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb89baaa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8940aa1e0 0x7fb894148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb89baaa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb894139b50 con 0x7fb894066250 2020-11-20T12:07:18.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb89baaa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940a37d0 con 0x7fb8940aa1e0 2020-11-20T12:07:18.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb89baaa700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89414b410 con 0x7fb8940a7810 2020-11-20T12:07:18.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894066250 0x7fb89413a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894066250 0x7fb89413a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38356/0 (socket says 172.21.15.44:38356) 2020-11-20T12:07:18.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 -- 172.21.15.44:0/750493755 learned_addr learned my addr 172.21.15.44:0/750493755 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:18.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb89a047700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8940aa1e0 0x7fb894148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899045700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8940a7810 0x7fb8940a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8940a7810 msgr2=0x7fb8940a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8940a7810 0x7fb8940a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8940aa1e0 msgr2=0x7fb894148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8940aa1e0 0x7fb894148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 -- 172.21.15.44:0/750493755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89414b6b0 con 0x7fb894066250 2020-11-20T12:07:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899045700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8940a7810 0x7fb8940a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.247+0000 7fb899846700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894066250 0x7fb89413a5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb88400c710 tx=0x7fb884007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.248+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400e560 con 0x7fb894066250 2020-11-20T12:07:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.248+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88400e6c0 con 0x7fb894066250 2020-11-20T12:07:18.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.248+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb894147e40 con 0x7fb894066250 2020-11-20T12:07:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.248+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400ea40 con 0x7fb894066250 2020-11-20T12:07:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.248+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8940a33b0 con 0x7fb894066250 2020-11-20T12:07:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.248+0000 7fb89baaa700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb894148b40 0x7fb89414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.248+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940a3660 con 0x7fb894148b40 2020-11-20T12:07:18.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb899045700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb894148b40 0x7fb89414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb884019030 con 0x7fb894066250 2020-11-20T12:07:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb899045700 1 -- 172.21.15.44:0/750493755 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8940add50 con 0x7fb894148b40 2020-11-20T12:07:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb881ffb700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb86c056d20 0x7fb86c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb88406f550 con 0x7fb894066250 2020-11-20T12:07:18.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb899045700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb894148b40 0x7fb89414b840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fb8940a8880 tx=0x7fb890015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb89a047700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb86c056d20 0x7fb86c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.249+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890011e00 con 0x7fb894148b40 2020-11-20T12:07:18.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.250+0000 7fb89a047700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb86c056d20 0x7fb86c0591e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb894115330 tx=0x7fb88c007ba0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.250+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb89001f650 con 0x7fb894148b40 2020-11-20T12:07:18.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.250+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb894148b40 msgr2=0x7fb89414b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.250+0000 7fb881ffb700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb894148b40 0x7fb89414b840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fb8940a8880 tx=0x7fb890015000).stop 2020-11-20T12:07:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.250+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb894148b40 msgr2=0x7fb89414b840 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:18.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.250+0000 7fb881ffb700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb894148b40 0x7fb89414b840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.258+0000 7fb89baaa700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb89414bd30 0x7fb8940a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.258+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb894061650 con 0x7fb89414bd30 2020-11-20T12:07:18.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.258+0000 7fb899045700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb89414bd30 0x7fb8940a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.258+0000 7fb899045700 1 -- 172.21.15.44:0/750493755 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb8940446d0 con 0x7fb89414bd30 2020-11-20T12:07:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.258+0000 7fb899045700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb89414bd30 0x7fb8940a1c60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fb8940a37a0 tx=0x7fb890020000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890028450 con 0x7fb89414bd30 2020-11-20T12:07:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4775 (secure 0 0 0) 0x7fb89001f440 con 0x7fb89414bd30 2020-11-20T12:07:18.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb89414bd30 msgr2=0x7fb8940a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb881ffb700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb89414bd30 0x7fb8940a1c60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fb8940a37a0 tx=0x7fb890020000).stop 2020-11-20T12:07:18.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb881ffb700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb89414bd30 msgr2=0x7fb8940a1c60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:18.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb881ffb700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb89414bd30 0x7fb8940a1c60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb86c056d20 msgr2=0x7fb86c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89baaa700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb86c056d20 0x7fb86c0591e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb894115330 tx=0x7fb88c007ba0).stop 2020-11-20T12:07:18.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894066250 msgr2=0x7fb89413a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89baaa700 1 --2- 172.21.15.44:0/750493755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb894066250 0x7fb89413a5d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb88400c710 tx=0x7fb884007c00).stop 2020-11-20T12:07:18.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89a047700 1 -- 172.21.15.44:0/750493755 reap_dead start 2020-11-20T12:07:18.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89a047700 1 -- 172.21.15.44:0/750493755 reap_dead start 2020-11-20T12:07:18.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 shutdown_connections 2020-11-20T12:07:18.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.259+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 >> 172.21.15.44:0/750493755 conn(0x7fb894062690 msgr2=0x7fb8940ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:18.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.260+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 shutdown_connections 2020-11-20T12:07:18.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.260+0000 7fb89baaa700 1 -- 172.21.15.44:0/750493755 wait complete. 2020-11-20T12:07:18.271 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:18.272 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:18.272 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:18.272 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:07:18.272 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 38, 2020-11-20T12:07:18.272 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:18.273 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:18.273 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:07:18.273 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:18.273 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.274 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:07:18.274 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:18.274 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:18.274 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:18.274 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:18.275 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:18.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:18.275 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:18.275 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:18.275 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:18.276 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:18.276 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:18.276 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.276 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:18.276 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:18.277 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:18.277 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:18.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:18.277 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:18.277 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:18.278 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:18.278 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:18.278 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:18.278 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.279 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:18.279 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:18.279 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:18.279 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:18.279 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:18.280 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:18.280 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:18.280 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:18.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:18.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:18.281 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:18.281 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:18.281 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:18.281 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:18.281 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:18.282 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:18.282 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:18.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:18.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:18.282 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:18.283 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:18.283 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:18.283 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:18.283 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.283 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:18.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:18.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.284 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:18.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:18.285 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:18.285 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:18.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:18.286 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:18.286 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:18.287 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:18.287 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.287 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.287 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.288 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:18.288 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:18.288 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:18.288 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:18.288 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:18.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.289 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:18.289 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:18.289 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:18.290 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:18.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.290 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:18.290 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:18.290 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:18.291 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:18.291 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.292 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.292 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.292 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:18.292 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:18.293 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:18.293 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:18.293 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:18.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.294 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:18.294 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:18.294 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:18.294 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:18.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:18.295 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:18.295 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:18.296 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:18.296 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:18.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.296 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.297 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:18.297 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:18.297 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:18.297 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:18.298 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:18.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.298 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.298 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:18.299 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:18.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.299 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.299 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.299 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:18.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.300 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.300 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:07:18.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.300 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.301 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.301 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:18.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.301 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.301 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:07:18.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.302 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.302 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.302 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:07:18.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.303 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.303 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:18.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.303 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.303 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.304 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:07:18.304 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.304 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:18.304 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:07:18.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:18.305 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:18.305 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:18.305 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:18.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:18.305 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:18.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:18.306 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:18.306 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:18.307 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:07:18.308 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:07:18.436 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.432+0000 7fcd166ca700 1 Processor -- start 2020-11-20T12:07:18.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.432+0000 7fcd166ca700 1 -- start start 2020-11-20T12:07:18.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd166ca700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcd1013a990 0x7fcd1013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd166ca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 0x7fcd10141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd166ca700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd100a6260 con 0x7fcd10060330 2020-11-20T12:07:18.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd166ca700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100a52e0 con 0x7fcd1013a990 2020-11-20T12:07:18.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd166ca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100a6490 con 0x7fcd1013f800 2020-11-20T12:07:18.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0ffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcd10060330 0x7fcd100a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42116/0 (socket says 172.21.15.44:42116) 2020-11-20T12:07:18.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0ffff700 1 -- 172.21.15.44:0/2142187314 learned_addr learned my addr 172.21.15.44:0/2142187314 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:18.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd07fff700 1 --2- 172.21.15.44:0/2142187314 >> v2:172.21.15.79:3300/0 conn(0x7fcd1013a990 0x7fcd1013ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd14c67700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 0x7fcd10141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611839545 0 0) 0x7fcd100a6260 con 0x7fcd10060330 2020-11-20T12:07:18.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccf0003240 con 0x7fcd10060330 2020-11-20T12:07:18.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1423232452 0 0) 0x7fccf0003240 con 0x7fcd10060330 2020-11-20T12:07:18.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 msgr2=0x7fcd10141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 0x7fcd10141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 >> v2:172.21.15.79:3300/0 conn(0x7fcd1013a990 msgr2=0x7fcd1013ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 --2- 172.21.15.44:0/2142187314 >> v2:172.21.15.79:3300/0 conn(0x7fcd1013a990 0x7fcd1013ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100a6730 con 0x7fcd10060330 2020-11-20T12:07:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.433+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fcd000014e0 con 0x7fcd10060330 2020-11-20T12:07:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 >> v1:172.21.15.44:6789/0 conn(0x7fcd10060330 legacy=0x7fcd100a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf0003730 0x7fccf0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fccf00044e0 0x7fccf0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 0x7fccf0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100a52e0 con 0x7fccf0003730 2020-11-20T12:07:18.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100a6490 con 0x7fccf00044e0 2020-11-20T12:07:18.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf0004270 con 0x7fcd1013f800 2020-11-20T12:07:18.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0ffff700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf0003730 0x7fccf0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd07fff700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fccf00044e0 0x7fccf0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd14c67700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 0x7fccf0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0ffff700 1 -- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 msgr2=0x7fccf0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:18.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0ffff700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd1013f800 0x7fccf0004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0ffff700 1 -- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fccf00044e0 msgr2=0x7fccf0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:18.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0ffff700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fccf00044e0 0x7fccf0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0ffff700 1 -- 172.21.15.44:0/2142187314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf0003240 con 0x7fccf0003730 2020-11-20T12:07:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0ffff700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf0003730 0x7fccf0003b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcd0000c440 tx=0x7fcd100a68c0).ready entity=mon.0 client_cookie=78dd4a7d2d178500 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000ce20 con 0x7fccf0003730 2020-11-20T12:07:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd000072e0 con 0x7fccf0003730 2020-11-20T12:07:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.434+0000 7fcd0f7fe700 1 -- 172.21.15.44:0/2142187314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000055f0 con 0x7fccf0003730 2020-11-20T12:07:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.435+0000 7fcd166ca700 1 -- 172.21.15.44:0/2142187314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf0003730 msgr2=0x7fccf0003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.435+0000 7fcd166ca700 1 --2- 172.21.15.44:0/2142187314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccf0003730 0x7fccf0003b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcd0000c440 tx=0x7fcd100a68c0).stop 2020-11-20T12:07:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.435+0000 7fcd14c67700 1 -- 172.21.15.44:0/2142187314 reap_dead start 2020-11-20T12:07:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.435+0000 7fcd166ca700 1 -- 172.21.15.44:0/2142187314 shutdown_connections 2020-11-20T12:07:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.435+0000 7fcd166ca700 1 -- 172.21.15.44:0/2142187314 >> 172.21.15.44:0/2142187314 conn(0x7fcd10062690 msgr2=0x7fcd100a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.435+0000 7fcd166ca700 1 -- 172.21.15.44:0/2142187314 shutdown_connections 2020-11-20T12:07:18.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.435+0000 7fcd166ca700 1 -- 172.21.15.44:0/2142187314 wait complete. 2020-11-20T12:07:18.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.436+0000 7fcd166ca700 1 Processor -- start 2020-11-20T12:07:18.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.436+0000 7fcd166ca700 1 -- start start 2020-11-20T12:07:18.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.436+0000 7fcd166ca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd10137500 0x7fcd1014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.436+0000 7fcd166ca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd101320c0 0x7fcd100ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.436+0000 7fcd166ca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd100abae0 0x7fcd100abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.436+0000 7fcd166ca700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10131660 con 0x7fcd101320c0 2020-11-20T12:07:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.436+0000 7fcd166ca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10130c00 con 0x7fcd10137500 2020-11-20T12:07:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd166ca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100ac450 con 0x7fcd100abae0 2020-11-20T12:07:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd101320c0 0x7fcd100ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd101320c0 0x7fcd100ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38382/0 (socket says 172.21.15.44:38382) 2020-11-20T12:07:18.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 -- 172.21.15.44:0/612943955 learned_addr learned my addr 172.21.15.44:0/612943955 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:18.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd0ffff700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd10137500 0x7fcd1014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd14c67700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd100abae0 0x7fcd100abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 -- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd100abae0 msgr2=0x7fcd100abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd100abae0 0x7fcd100abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 -- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd10137500 msgr2=0x7fcd1014a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd10137500 0x7fcd1014a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 -- 172.21.15.44:0/612943955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd101325d0 con 0x7fcd101320c0 2020-11-20T12:07:18.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd14c67700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd100abae0 0x7fcd100abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:18.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd07fff700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd101320c0 0x7fcd100ab5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fccf8011cc0 tx=0x7fccf8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd0cff9700 1 -- 172.21.15.44:0/612943955 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf800faa0 con 0x7fcd101320c0 2020-11-20T12:07:18.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd0cff9700 1 -- 172.21.15.44:0/612943955 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf800fc00 con 0x7fcd101320c0 2020-11-20T12:07:18.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10131850 con 0x7fcd101320c0 2020-11-20T12:07:18.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd0cff9700 1 -- 172.21.15.44:0/612943955 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8014440 con 0x7fcd101320c0 2020-11-20T12:07:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.437+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd100ac700 con 0x7fcd101320c0 2020-11-20T12:07:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.438+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd101319e0 con 0x7fcd101320c0 2020-11-20T12:07:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.438+0000 7fcd0cff9700 1 -- 172.21.15.44:0/612943955 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fccf8014830 con 0x7fcd101320c0 2020-11-20T12:07:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.439+0000 7fcd0cff9700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fccec056d20 0x7fccec0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.439+0000 7fcd0cff9700 1 -- 172.21.15.44:0/612943955 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fccf8070b50 con 0x7fcd101320c0 2020-11-20T12:07:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.439+0000 7fcd0ffff700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fccec056d20 0x7fccec0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.439+0000 7fcd0ffff700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fccec056d20 0x7fccec0591e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fcd10045400 tx=0x7fcd00013000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.442+0000 7fcd0cff9700 1 -- 172.21.15.44:0/612943955 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fccf8045e20 con 0x7fcd101320c0 2020-11-20T12:07:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.684+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fcd10061650 con 0x7fcd101320c0 2020-11-20T12:07:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.685+0000 7fcd0cff9700 1 -- 172.21.15.44:0/612943955 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fccf8018070 con 0x7fcd101320c0 2020-11-20T12:07:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.686+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fccec056d20 msgr2=0x7fccec0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fccec056d20 0x7fccec0591e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fcd10045400 tx=0x7fcd00013000).stop 2020-11-20T12:07:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd101320c0 msgr2=0x7fcd100ab5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd101320c0 0x7fcd100ab5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fccf8011cc0 tx=0x7fccf8006df0).stop 2020-11-20T12:07:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 shutdown_connections 2020-11-20T12:07:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fccec056d20 0x7fccec0591e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcd100abae0 0x7fcd100abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd10137500 0x7fcd1014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 --2- 172.21.15.44:0/612943955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd101320c0 0x7fcd100ab5f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 >> 172.21.15.44:0/612943955 conn(0x7fcd10062690 msgr2=0x7fcd10139b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 shutdown_connections 2020-11-20T12:07:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:18.687+0000 7fcd166ca700 1 -- 172.21.15.44:0/612943955 wait complete. 2020-11-20T12:07:18.703 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:07:24.674 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir foo/.snap/ss 2020-11-20T12:07:24.685 INFO:tasks.workunit.client.0.smithi044.stderr:+ ln foo/x foo/xx 2020-11-20T12:07:24.691 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat foo/.snap/ss/x 2020-11-20T12:07:24.701 INFO:tasks.workunit.client.0.smithi044.stdout:x 2020-11-20T12:07:24.701 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir foo/.snap/ss 2020-11-20T12:07:24.710 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo a 2020-11-20T12:07:24.712 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo b 2020-11-20T12:07:24.715 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir foo/.snap/s 2020-11-20T12:07:24.731 INFO:tasks.workunit.client.0.smithi044.stderr:++ cat foo/.snap/s/a 2020-11-20T12:07:24.742 INFO:tasks.workunit.client.0.smithi044.stderr:+ r=a 2020-11-20T12:07:24.742 INFO:tasks.workunit.client.0.smithi044.stderr:+ '[' -z a ']' 2020-11-20T12:07:24.742 INFO:tasks.workunit.client.0.smithi044.stderr:+ ln foo/b foo/b2 2020-11-20T12:07:24.745 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat foo/.snap/s/b 2020-11-20T12:07:24.753 INFO:tasks.workunit.client.0.smithi044.stdout:b 2020-11-20T12:07:24.753 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 'this used to hang:' 2020-11-20T12:07:24.754 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo more 2020-11-20T12:07:24.754 INFO:tasks.workunit.client.0.smithi044.stdout:this used to hang: 2020-11-20T12:07:24.757 INFO:tasks.workunit.client.0.smithi044.stdout:oh, it didn't hang! good job. 2020-11-20T12:07:24.757 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 'oh, it didn'\''t hang! good job.' 2020-11-20T12:07:24.757 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat foo/b 2020-11-20T12:07:24.758 INFO:tasks.workunit.client.0.smithi044.stdout:b 2020-11-20T12:07:24.758 INFO:tasks.workunit.client.0.smithi044.stdout:more 2020-11-20T12:07:24.759 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir foo/.snap/s 2020-11-20T12:07:24.769 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm -r foo 2020-11-20T12:07:24.793 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:07:24.793 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:07:24.794 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:07:24.794 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:07:24.833 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2020-11-20T12:07:24.833 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2020-11-20T12:07:24.896 INFO:tasks.workunit.client.0.smithi044.stderr:+ set -e 2020-11-20T12:07:24.897 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:07:24.998 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.994+0000 7f5a76d71700 1 Processor -- start 2020-11-20T12:07:24.998 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.994+0000 7f5a76d71700 1 -- start start 2020-11-20T12:07:25.000 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a76d71700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a700a8b40 0x7f5a7013aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:25.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a76d71700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5a700a3930 0x7f5a701387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:25.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a76d71700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a70064ed0 con 0x7f5a7013b3b0 2020-11-20T12:07:25.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a76d71700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a700a6320 con 0x7f5a700a3930 2020-11-20T12:07:25.001 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a76d71700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700a5be0 con 0x7f5a700a8b40 2020-11-20T12:07:25.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a7530e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5a700a3930 0x7f5a701387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:25.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a6ffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5a7013b3b0 0x7f5a701380b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42136/0 (socket says 172.21.15.44:42136) 2020-11-20T12:07:25.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a6ffff700 1 -- 172.21.15.44:0/2530669751 learned_addr learned my addr 172.21.15.44:0/2530669751 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:25.002 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a74b0d700 1 --2- 172.21.15.44:0/2530669751 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a700a8b40 0x7f5a7013aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:25.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a7530e700 1 -- 172.21.15.44:0/2530669751 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a700a8b40 msgr2=0x7f5a7013aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:25.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a7530e700 1 --2- 172.21.15.44:0/2530669751 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a700a8b40 0x7f5a7013aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:25.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a7530e700 1 -- 172.21.15.44:0/2530669751 >> v1:172.21.15.44:6789/0 conn(0x7f5a7013b3b0 legacy=0x7f5a701380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:25.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a6f7fe700 1 -- 172.21.15.44:0/2530669751 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838042258 0 0) 0x7f5a70064ed0 con 0x7f5a7013b3b0 2020-11-20T12:07:25.003 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a7530e700 1 -- 172.21.15.44:0/2530669751 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700a5e80 con 0x7f5a700a3930 2020-11-20T12:07:25.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a74b0d700 1 --2- 172.21.15.44:0/2530669751 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a700a8b40 0x7f5a7013aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:07:25.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a7530e700 1 --2- 172.21.15.44:0/2530669751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a701387c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5a68009c90 tx=0x7f5a680091d0).ready entity=mon.0 client_cookie=8483316e703a28e9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:25.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a6f7fe700 1 -- 172.21.15.44:0/2530669751 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800bbc0 con 0x7f5a700a3930 2020-11-20T12:07:25.004 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a6f7fe700 1 -- 172.21.15.44:0/2530669751 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6800bd20 con 0x7f5a700a3930 2020-11-20T12:07:25.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.995+0000 7f5a6f7fe700 1 -- 172.21.15.44:0/2530669751 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68011610 con 0x7f5a700a3930 2020-11-20T12:07:25.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 -- 172.21.15.44:0/2530669751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 msgr2=0x7f5a701387c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:25.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2530669751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a701387c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5a68009c90 tx=0x7f5a680091d0).stop 2020-11-20T12:07:25.005 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 -- 172.21.15.44:0/2530669751 shutdown_connections 2020-11-20T12:07:25.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2530669751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a701387c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:25.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2530669751 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a700a8b40 0x7f5a7013aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:25.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 -- 172.21.15.44:0/2530669751 >> 172.21.15.44:0/2530669751 conn(0x7f5a700aced0 msgr2=0x7f5a700ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:25.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 -- 172.21.15.44:0/2530669751 shutdown_connections 2020-11-20T12:07:25.006 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.996+0000 7f5a76d71700 1 -- 172.21.15.44:0/2530669751 wait complete. 2020-11-20T12:07:25.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.997+0000 7f5a76d71700 1 Processor -- start 2020-11-20T12:07:25.007 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.997+0000 7f5a76d71700 1 -- start start 2020-11-20T12:07:25.009 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a700a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:25.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a700a8b40 0x7f5a7013a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:25.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a7013b3b0 0x7f5a70145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:25.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700a1f80 con 0x7f5a700a3930 2020-11-20T12:07:25.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700a01d0 con 0x7f5a700a8b40 2020-11-20T12:07:25.010 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a70145d40 con 0x7f5a7013b3b0 2020-11-20T12:07:25.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a700a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:25.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a700a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38394/0 (socket says 172.21.15.44:38394) 2020-11-20T12:07:25.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 -- 172.21.15.44:0/2806253348 learned_addr learned my addr 172.21.15.44:0/2806253348 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:25.011 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a6ffff700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a700a8b40 0x7f5a7013a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:25.012 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a7530e700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a7013b3b0 0x7f5a70145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:25.012 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 -- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a7013b3b0 msgr2=0x7f5a70145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:25.012 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a7013b3b0 0x7f5a70145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:25.012 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 -- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a700a8b40 msgr2=0x7f5a7013a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:25.012 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a700a8b40 0x7f5a7013a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:25.013 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 -- 172.21.15.44:0/2806253348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700a0430 con 0x7f5a700a3930 2020-11-20T12:07:25.013 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a74b0d700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a700a0b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5a6000cb50 tx=0x7f5a60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:25.013 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600076b0 con 0x7f5a700a3930 2020-11-20T12:07:25.013 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60007810 con 0x7f5a700a3930 2020-11-20T12:07:25.014 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a700a32f0 con 0x7f5a700a3930 2020-11-20T12:07:25.014 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60007b90 con 0x7f5a700a3930 2020-11-20T12:07:25.014 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.998+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a700a1a00 con 0x7f5a700a3930 2020-11-20T12:07:25.014 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.999+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7005c590 con 0x7f5a700a3930 2020-11-20T12:07:25.015 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:24.999+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f5a6001b460 con 0x7f5a700a3930 2020-11-20T12:07:25.015 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.000+0000 7f5a6cff9700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5a48056d20 0x7f5a480591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:25.015 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.000+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f5a600701c0 con 0x7f5a700a3930 2020-11-20T12:07:25.015 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.000+0000 7f5a6ffff700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5a48056d20 0x7f5a480591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:25.015 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.000+0000 7f5a6ffff700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5a48056d20 0x7f5a480591e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5a700a1050 tx=0x7f5a64007c20).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:25.016 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.004+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5a60045410 con 0x7f5a700a3930 2020-11-20T12:07:25.240 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.237+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5a700a8f20 con 0x7f5a700a3930 2020-11-20T12:07:25.691 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.687+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v24) v1 ==== 167+0+0 (secure 0 0 0) 0x7f5a60048a30 con 0x7f5a700a3930 2020-11-20T12:07:25.868 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:25.865+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5a700446d0 con 0x7f5a700a3930 2020-11-20T12:07:26.700 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.696+0000 7f5a6cff9700 1 -- 172.21.15.44:0/2806253348 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v25) v1 ==== 167+0+0 (secure 0 0 0) 0x7f5a60048c10 con 0x7f5a700a3930 2020-11-20T12:07:26.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5a48056d20 msgr2=0x7f5a480591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:26.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5a48056d20 0x7f5a480591e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5a700a1050 tx=0x7f5a64007c20).stop 2020-11-20T12:07:26.708 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 msgr2=0x7f5a700a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:26.708 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a700a0b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5a6000cb50 tx=0x7f5a60008040).stop 2020-11-20T12:07:26.708 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 shutdown_connections 2020-11-20T12:07:26.709 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5a48056d20 0x7f5a480591e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:26.709 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5a7013b3b0 0x7f5a70145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:26.709 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a700a3930 0x7f5a700a0b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:26.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 --2- 172.21.15.44:0/2806253348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a700a8b40 0x7f5a7013a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:26.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.698+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 >> 172.21.15.44:0/2806253348 conn(0x7f5a700aced0 msgr2=0x7f5a700a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:26.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.699+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 shutdown_connections 2020-11-20T12:07:26.710 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:26.699+0000 7f5a76d71700 1 -- 172.21.15.44:0/2806253348 wait complete. 2020-11-20T12:07:26.711 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:07:26.716 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:26.719 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/bar 2020-11-20T12:07:26.733 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/foo 2020-11-20T12:07:26.741 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/foo 2020-11-20T12:07:26.750 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:26.754 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/bar/a 2020-11-20T12:07:26.762 INFO:tasks.workunit.client.0.smithi044.stdout: File: .snap/bar/a 2020-11-20T12:07:26.763 INFO:tasks.workunit.client.0.smithi044.stdout: Size: 0 Blocks: 1 IO Block: 4096 directory 2020-11-20T12:07:26.763 INFO:tasks.workunit.client.0.smithi044.stdout:Device: 2fh/47d Inode: 59955269551003528 Links: 2 2020-11-20T12:07:26.763 INFO:tasks.workunit.client.0.smithi044.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1236/ ubuntu) 2020-11-20T12:07:26.763 INFO:tasks.workunit.client.0.smithi044.stdout:Context: system_u:object_r:fusefs_t:s0 2020-11-20T12:07:26.764 INFO:tasks.workunit.client.0.smithi044.stdout:Access: 2020-11-20 12:07:26.714823107 +0000 2020-11-20T12:07:26.764 INFO:tasks.workunit.client.0.smithi044.stdout:Modify: 2020-11-20 12:07:26.714823107 +0000 2020-11-20T12:07:26.764 INFO:tasks.workunit.client.0.smithi044.stdout:Change: 2020-11-20 12:07:26.714823107 +0000 2020-11-20T12:07:26.764 INFO:tasks.workunit.client.0.smithi044.stdout: Birth: - 2020-11-20T12:07:26.765 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir .snap/bar 2020-11-20T12:07:26.772 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:07:26.772 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:07:26.773 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:07:26.773 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:07:26.855 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2020-11-20T12:07:26.856 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2020-11-20T12:07:26.921 INFO:tasks.workunit.client.0.smithi044.stderr:+ set -e 2020-11-20T12:07:26.922 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:07:27.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.020+0000 7f285abd2700 1 Processor -- start 2020-11-20T12:07:27.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.020+0000 7f285abd2700 1 -- start start 2020-11-20T12:07:27.026 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.020+0000 7f285abd2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:27.027 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285abd2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f28540a9490 0x7f28540a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:27.027 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285abd2700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540a7230 con 0x7f28540a7870 2020-11-20T12:07:27.027 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285abd2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28540a6160 con 0x7f28540a9490 2020-11-20T12:07:27.028 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285abd2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540a4bc0 con 0x7f2854060b40 2020-11-20T12:07:27.028 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f2853fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f28540a9490 0x7f28540a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:27.028 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285916f700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f28540a7870 0x7f28540a7d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42154/0 (socket says 172.21.15.44:42154) 2020-11-20T12:07:27.029 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f2853fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f28540a9490 0x7f28540a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38402/0 (socket says 172.21.15.44:38402) 2020-11-20T12:07:27.029 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285916f700 1 -- 172.21.15.44:0/2614639989 learned_addr learned my addr 172.21.15.44:0/2614639989 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:27.029 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285896e700 1 --2- 172.21.15.44:0/2614639989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:27.029 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f2853fff700 1 -- 172.21.15.44:0/2614639989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 msgr2=0x7f28540a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:27.030 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f2853fff700 1 --2- 172.21.15.44:0/2614639989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28540a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:27.030 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f2853fff700 1 -- 172.21.15.44:0/2614639989 >> v1:172.21.15.44:6789/0 conn(0x7f28540a7870 legacy=0x7f28540a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:27.030 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f28537fe700 1 -- 172.21.15.44:0/2614639989 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35543483 0 0) 0x7f28540a7230 con 0x7f28540a7870 2020-11-20T12:07:27.031 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f2853fff700 1 -- 172.21.15.44:0/2614639989 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28540a4480 con 0x7f28540a9490 2020-11-20T12:07:27.031 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f285896e700 1 --2- 172.21.15.44:0/2614639989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28540a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:27.031 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f2853fff700 1 --2- 172.21.15.44:0/2614639989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a9490 0x7f28540a5640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f284800b9a0 tx=0x7f2848006df0).ready entity=mon.0 client_cookie=35b81ccb3df52fba server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:27.035 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f28537fe700 1 -- 172.21.15.44:0/2614639989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848009840 con 0x7f28540a9490 2020-11-20T12:07:27.036 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f28537fe700 1 -- 172.21.15.44:0/2614639989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28480099a0 con 0x7f28540a9490 2020-11-20T12:07:27.036 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.021+0000 7f28537fe700 1 -- 172.21.15.44:0/2614639989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848009d20 con 0x7f28540a9490 2020-11-20T12:07:27.036 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 -- 172.21.15.44:0/2614639989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a9490 msgr2=0x7f28540a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:27.036 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 --2- 172.21.15.44:0/2614639989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a9490 0x7f28540a5640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f284800b9a0 tx=0x7f2848006df0).stop 2020-11-20T12:07:27.037 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 -- 172.21.15.44:0/2614639989 shutdown_connections 2020-11-20T12:07:27.037 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 --2- 172.21.15.44:0/2614639989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28540a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:27.037 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 --2- 172.21.15.44:0/2614639989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a9490 0x7f28540a5640 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:27.037 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 -- 172.21.15.44:0/2614639989 >> 172.21.15.44:0/2614639989 conn(0x7f28540aced0 msgr2=0x7f28540ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:27.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 -- 172.21.15.44:0/2614639989 shutdown_connections 2020-11-20T12:07:27.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.022+0000 7f285abd2700 1 -- 172.21.15.44:0/2614639989 wait complete. 2020-11-20T12:07:27.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 Processor -- start 2020-11-20T12:07:27.038 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 -- start start 2020-11-20T12:07:27.039 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28541335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:27.039 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a7870 0x7f285412cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:27.039 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28540a9490 0x7f285412e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:27.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285412dec0 con 0x7f28540a7870 2020-11-20T12:07:27.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285412e120 con 0x7f28540a9490 2020-11-20T12:07:27.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.023+0000 7f285abd2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285412e3c0 con 0x7f2854060b40 2020-11-20T12:07:27.040 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a7870 0x7f285412cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:27.041 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a7870 0x7f285412cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38410/0 (socket says 172.21.15.44:38410) 2020-11-20T12:07:27.041 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 -- 172.21.15.44:0/49662453 learned_addr learned my addr 172.21.15.44:0/49662453 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:27.041 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f285896e700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28541335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:27.041 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f285916f700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28540a9490 0x7f285412e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:27.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 -- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 msgr2=0x7f28541335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:27.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28541335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:27.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 -- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28540a9490 msgr2=0x7f285412e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:27.042 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28540a9490 0x7f285412e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:27.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 -- 172.21.15.44:0/49662453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285412f7c0 con 0x7f28540a7870 2020-11-20T12:07:27.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2853fff700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a7870 0x7f285412cc20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2848009fd0 tx=0x7f2848015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:27.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284801e460 con 0x7f28540a7870 2020-11-20T12:07:27.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284801e5c0 con 0x7f28540a7870 2020-11-20T12:07:27.043 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285412d110 con 0x7f28540a7870 2020-11-20T12:07:27.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284801e940 con 0x7f28540a7870 2020-11-20T12:07:27.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.024+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2854131df0 con 0x7f28540a7870 2020-11-20T12:07:27.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.025+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285412d410 con 0x7f28540a7870 2020-11-20T12:07:27.044 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.025+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2848025030 con 0x7f28540a7870 2020-11-20T12:07:27.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.026+0000 7f2850ff9700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2830056d20 0x7f28300591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:27.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.026+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f2848071a10 con 0x7f28540a7870 2020-11-20T12:07:27.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.026+0000 7f285896e700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2830056d20 0x7f28300591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:27.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.026+0000 7f285896e700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2830056d20 0x7f28300591e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2844006bd0 tx=0x7f2844008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:27.045 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.029+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2848046cc0 con 0x7f28540a7870 2020-11-20T12:07:27.269 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.264+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2854131fe0 con 0x7f28540a7870 2020-11-20T12:07:27.707 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.704+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v26) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2848023070 con 0x7f28540a7870 2020-11-20T12:07:27.886 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:27.882+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f285412e550 con 0x7f28540a7870 2020-11-20T12:07:28.623 INFO:tasks.ceph.osd.0.smithi044.stderr:2020-11-20T12:07:28.619+0000 7f24e62c5700 -1 osd.0 pg_epoch: 36 pg[4.3s0( v 34'1365 (0'0,34'1365] local-lis/les=23/24 n=2 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) [0,5,7,1]p0(0) r=0 lpr=23 crt=34'1365 lcod 34'1364 mlcod 34'1364 active+clean+snaptrim trimq=[1a6~2,1a9~2] ps=[1a4~2]] removing snap head 2020-11-20T12:07:28.704 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:07:28.705 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:28.713 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.710+0000 7f2850ff9700 1 -- 172.21.15.44:0/49662453 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v27) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2848023020 con 0x7f28540a7870 2020-11-20T12:07:28.716 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2830056d20 msgr2=0x7f28300591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:28.716 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2830056d20 0x7f28300591e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2844006bd0 tx=0x7f2844008040).stop 2020-11-20T12:07:28.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a7870 msgr2=0x7f285412cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:28.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a7870 0x7f285412cc20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f2848009fd0 tx=0x7f2848015000).stop 2020-11-20T12:07:28.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 shutdown_connections 2020-11-20T12:07:28.717 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2830056d20 0x7f28300591e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.718 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28540a9490 0x7f285412e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.718 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2854060b40 0x7f28541335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.718 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 --2- 172.21.15.44:0/49662453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28540a7870 0x7f285412cc20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.719 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 >> 172.21.15.44:0/49662453 conn(0x7f28540aced0 msgr2=0x7f28540aba40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:28.719 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 shutdown_connections 2020-11-20T12:07:28.720 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:28.713+0000 7f285abd2700 1 -- 172.21.15.44:0/49662453 wait complete. 2020-11-20T12:07:28.720 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:07:28.729 INFO:tasks.workunit.client.0.smithi044.stderr:++ seq 1 20 2020-11-20T12:07:28.730 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:28.730 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:28.734 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:28.734 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:28.755 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:28.764 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:28.765 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:28.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde55b3d700 1 Processor -- start 2020-11-20T12:07:28.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde55b3d700 1 -- start start 2020-11-20T12:07:28.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde55b3d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fde500a3e10 0x7fde500a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde55b3d700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500a9770 con 0x7fde500aa400 2020-11-20T12:07:28.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde55b3d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500a6da0 con 0x7fde500aa020 2020-11-20T12:07:28.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde55b3d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500a56c0 con 0x7fde500a3e10 2020-11-20T12:07:28.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde4effd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fde500aa400 0x7fde500a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42166/0 (socket says 172.21.15.44:42166) 2020-11-20T12:07:28.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.847+0000 7fde4effd700 1 -- 172.21.15.44:0/464261634 learned_addr learned my addr 172.21.15.44:0/464261634 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:28.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949819957 0 0) 0x7fde380010d0 con 0x7fde500aa400 2020-11-20T12:07:28.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4ffff700 1 --2- 172.21.15.44:0/464261634 >> v2:172.21.15.79:3300/0 conn(0x7fde500a3e10 0x7fde500a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde30003240 con 0x7fde500aa400 2020-11-20T12:07:28.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2896993619 0 0) 0x7fde30003240 con 0x7fde500aa400 2020-11-20T12:07:28.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 >> v2:172.21.15.79:3300/0 conn(0x7fde500a3e10 msgr2=0x7fde500a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:28.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 --2- 172.21.15.44:0/464261634 >> v2:172.21.15.79:3300/0 conn(0x7fde500a3e10 0x7fde500a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 >> v1:172.21.15.79:6789/0 conn(0x7fde500aa020 legacy=0x7fde500a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:28.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500a7040 con 0x7fde500aa400 2020-11-20T12:07:28.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fde38001670 con 0x7fde500aa400 2020-11-20T12:07:28.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 >> v1:172.21.15.44:6789/0 conn(0x7fde500aa400 legacy=0x7fde500a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:28.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde30003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde300044e0 0x7fde30004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde30004e50 0x7fde3000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500a56c0 con 0x7fde30003730 2020-11-20T12:07:28.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300041d0 con 0x7fde300044e0 2020-11-20T12:07:28.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3000bc20 con 0x7fde30004e50 2020-11-20T12:07:28.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4f7fe700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde30003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4effd700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde300044e0 0x7fde30004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.848+0000 7fde4ffff700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde30004e50 0x7fde3000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4f7fe700 1 -- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde30004e50 msgr2=0x7fde3000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:28.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4f7fe700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde30004e50 0x7fde3000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4f7fe700 1 -- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde300044e0 msgr2=0x7fde30004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:28.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4f7fe700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde300044e0 0x7fde30004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4f7fe700 1 -- 172.21.15.44:0/464261634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30003240 con 0x7fde30003730 2020-11-20T12:07:28.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4ffff700 1 -- 172.21.15.44:0/464261634 reap_dead start 2020-11-20T12:07:28.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4effd700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde300044e0 0x7fde30004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:28.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4f7fe700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde30003b10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fde48001f60 tx=0x7fde48007ef0).ready entity=mon.0 client_cookie=188affaa3902819d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:28.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48001600 con 0x7fde30003730 2020-11-20T12:07:28.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800cb10 con 0x7fde30003730 2020-11-20T12:07:28.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde4e7fc700 1 -- 172.21.15.44:0/464261634 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800ce90 con 0x7fde30003730 2020-11-20T12:07:28.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde55b3d700 1 -- 172.21.15.44:0/464261634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 msgr2=0x7fde30003b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:28.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde55b3d700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde30003b10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fde48001f60 tx=0x7fde48007ef0).stop 2020-11-20T12:07:28.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde55b3d700 1 -- 172.21.15.44:0/464261634 shutdown_connections 2020-11-20T12:07:28.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde55b3d700 1 --2- 172.21.15.44:0/464261634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde30003b10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde55b3d700 1 -- 172.21.15.44:0/464261634 >> 172.21.15.44:0/464261634 conn(0x7fde50062690 msgr2=0x7fde500a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:28.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde55b3d700 1 -- 172.21.15.44:0/464261634 shutdown_connections 2020-11-20T12:07:28.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.849+0000 7fde55b3d700 1 -- 172.21.15.44:0/464261634 wait complete. 2020-11-20T12:07:28.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.850+0000 7fde55b3d700 1 Processor -- start 2020-11-20T12:07:28.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde55b3d700 1 -- start start 2020-11-20T12:07:28.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde55b3d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde500677b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde55b3d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde50065ba0 0x7fde50149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde55b3d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde50149720 0x7fde50149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde55b3d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5014a090 con 0x7fde30003730 2020-11-20T12:07:28.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde55b3d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500add20 con 0x7fde50065ba0 2020-11-20T12:07:28.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde55b3d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde50065460 con 0x7fde50149720 2020-11-20T12:07:28.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde4f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde500677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde4f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde500677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38428/0 (socket says 172.21.15.44:38428) 2020-11-20T12:07:28.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde4f7fe700 1 -- 172.21.15.44:0/4215270064 learned_addr learned my addr 172.21.15.44:0/4215270064 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:28.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde4effd700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde50065ba0 0x7fde50149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde4ffff700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde50149720 0x7fde50149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde4f7fe700 1 -- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde50149720 msgr2=0x7fde50149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:28.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.851+0000 7fde4f7fe700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde50149720 0x7fde50149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde4f7fe700 1 -- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde50065ba0 msgr2=0x7fde50149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:28.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde4f7fe700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde50065ba0 0x7fde50149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:28.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde4f7fe700 1 -- 172.21.15.44:0/4215270064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50065700 con 0x7fde30003730 2020-11-20T12:07:28.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde4f7fe700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde500677b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fde48007f30 tx=0x7fde48016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:28.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde477fe700 1 -- 172.21.15.44:0/4215270064 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4801e420 con 0x7fde30003730 2020-11-20T12:07:28.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde477fe700 1 -- 172.21.15.44:0/4215270064 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4801e580 con 0x7fde30003730 2020-11-20T12:07:28.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde50065050 con 0x7fde30003730 2020-11-20T12:07:28.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde477fe700 1 -- 172.21.15.44:0/4215270064 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4801e900 con 0x7fde30003730 2020-11-20T12:07:28.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.852+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde500ac820 con 0x7fde30003730 2020-11-20T12:07:28.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.853+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde50061650 con 0x7fde30003730 2020-11-20T12:07:28.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.853+0000 7fde477fe700 1 -- 172.21.15.44:0/4215270064 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fde48025030 con 0x7fde30003730 2020-11-20T12:07:28.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.853+0000 7fde477fe700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde2c056d20 0x7fde2c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:28.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.853+0000 7fde477fe700 1 -- 172.21.15.44:0/4215270064 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fde48071930 con 0x7fde30003730 2020-11-20T12:07:28.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.853+0000 7fde4effd700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde2c056d20 0x7fde2c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:28.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.854+0000 7fde4effd700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde2c056d20 0x7fde2c0591e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fde50067320 tx=0x7fde38001e50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:28.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:28.857+0000 7fde477fe700 1 -- 172.21.15.44:0/4215270064 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fde48046c70 con 0x7fde30003730 2020-11-20T12:07:29.075 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:29.090 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:29.090 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:29.090 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:29.091 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:29.091 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:29.091 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:29.091 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:29.091 INFO:tasks.workunit.client.0.smithi044.stdout: 36138 ? 00:00:00 ps 2020-11-20T12:07:29.092 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:29.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.101+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fde500446d0 con 0x7fde30003730 2020-11-20T12:07:29.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.101+0000 7fde477fe700 1 -- 172.21.15.44:0/4215270064 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fde48023020 con 0x7fde30003730 2020-11-20T12:07:29.106 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:29.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde2c056d20 msgr2=0x7fde2c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde2c056d20 0x7fde2c0591e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fde50067320 tx=0x7fde38001e50).stop 2020-11-20T12:07:29.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 msgr2=0x7fde500677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde500677b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fde48007f30 tx=0x7fde48016000).stop 2020-11-20T12:07:29.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 shutdown_connections 2020-11-20T12:07:29.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde2c056d20 0x7fde2c0591e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde50149720 0x7fde50149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde30003730 0x7fde500677b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 --2- 172.21.15.44:0/4215270064 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde50065ba0 0x7fde50149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 >> 172.21.15.44:0/4215270064 conn(0x7fde50062690 msgr2=0x7fde50135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 shutdown_connections 2020-11-20T12:07:29.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.104+0000 7fde55b3d700 1 -- 172.21.15.44:0/4215270064 wait complete. 2020-11-20T12:07:29.111 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:29.112 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:29.118 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:29.118 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:29.121 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:07:29.121 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:07:29.121 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:07:29.122 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:29.123 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:29.135 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:29.143 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:29.144 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:29.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.247+0000 7f8ce036c700 1 Processor -- start 2020-11-20T12:07:29.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.247+0000 7f8ce036c700 1 -- start start 2020-11-20T12:07:29.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.247+0000 7f8ce036c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 0x7f8cd80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.247+0000 7f8ce036c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8cd813ac60 0x7f8cd80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.247+0000 7f8ce036c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80a9770 con 0x7f8cd80aa400 2020-11-20T12:07:29.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8ce036c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd80a56c0 con 0x7f8cd813ac60 2020-11-20T12:07:29.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8ce036c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80a6da0 con 0x7f8cd80aa020 2020-11-20T12:07:29.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd907700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8cd80aa400 0x7f8cd80a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42184/0 (socket says 172.21.15.44:42184) 2020-11-20T12:07:29.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd907700 1 -- 172.21.15.44:0/3129355391 learned_addr learned my addr 172.21.15.44:0/3129355391 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:29.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cde108700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 0x7f8cd80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766133991 0 0) 0x7f8cc80010d0 con 0x7f8cd80aa400 2020-11-20T12:07:29.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc0003240 con 0x7f8cd80aa400 2020-11-20T12:07:29.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cde909700 1 --2- 172.21.15.44:0/3129355391 >> v2:172.21.15.79:3300/0 conn(0x7f8cd813ac60 0x7f8cd80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3043185027 0 0) 0x7f8cc0003240 con 0x7f8cd80aa400 2020-11-20T12:07:29.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 msgr2=0x7f8cd80a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:29.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 0x7f8cd80a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 >> v2:172.21.15.79:3300/0 conn(0x7f8cd813ac60 msgr2=0x7f8cd80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:29.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 --2- 172.21.15.44:0/3129355391 >> v2:172.21.15.79:3300/0 conn(0x7f8cd813ac60 0x7f8cd80a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80a7040 con 0x7f8cd80aa400 2020-11-20T12:07:29.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f8cc8001670 con 0x7f8cd80aa400 2020-11-20T12:07:29.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 >> v1:172.21.15.44:6789/0 conn(0x7f8cd80aa400 legacy=0x7f8cd80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc0003730 0x7f8cc0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc00044e0 0x7f8cc0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 0x7f8cc0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80a56c0 con 0x7f8cc00044e0 2020-11-20T12:07:29.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80a6da0 con 0x7f8cc0003730 2020-11-20T12:07:29.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0004270 con 0x7f8cd80aa020 2020-11-20T12:07:29.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cdd907700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc00044e0 0x7f8cc0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cde909700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 0x7f8cc0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.248+0000 7f8cde108700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc0003730 0x7f8cc0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd907700 1 -- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 msgr2=0x7f8cc0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:29.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd907700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 0x7f8cc0004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd907700 1 -- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc0003730 msgr2=0x7f8cc0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:29.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd907700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc0003730 0x7f8cc0003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd907700 1 -- 172.21.15.44:0/3129355391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0003240 con 0x7f8cc00044e0 2020-11-20T12:07:29.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cde909700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd80aa020 0x7f8cc0004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:29.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cde108700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc0003730 0x7f8cc0003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:29.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd907700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc00044e0 0x7f8cc0004960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8cd8062aa0 tx=0x7f8cd80a5f30).ready entity=mon.0 client_cookie=c57169e29bfaddfd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80072e0 con 0x7f8cc00044e0 2020-11-20T12:07:29.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8007440 con 0x7f8cc00044e0 2020-11-20T12:07:29.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.249+0000 7f8cdd106700 1 -- 172.21.15.44:0/3129355391 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8005820 con 0x7f8cc00044e0 2020-11-20T12:07:29.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.250+0000 7f8ce036c700 1 -- 172.21.15.44:0/3129355391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc00044e0 msgr2=0x7f8cc0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.250+0000 7f8ce036c700 1 --2- 172.21.15.44:0/3129355391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc00044e0 0x7f8cc0004960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8cd8062aa0 tx=0x7f8cd80a5f30).stop 2020-11-20T12:07:29.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.250+0000 7f8cde909700 1 -- 172.21.15.44:0/3129355391 reap_dead start 2020-11-20T12:07:29.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.250+0000 7f8ce036c700 1 -- 172.21.15.44:0/3129355391 shutdown_connections 2020-11-20T12:07:29.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.250+0000 7f8ce036c700 1 -- 172.21.15.44:0/3129355391 >> 172.21.15.44:0/3129355391 conn(0x7f8cd8062690 msgr2=0x7f8cd80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.250+0000 7f8ce036c700 1 -- 172.21.15.44:0/3129355391 shutdown_connections 2020-11-20T12:07:29.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.250+0000 7f8ce036c700 1 -- 172.21.15.44:0/3129355391 wait complete. 2020-11-20T12:07:29.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 Processor -- start 2020-11-20T12:07:29.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 -- start start 2020-11-20T12:07:29.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc00044e0 0x7f8cd80ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd814d7b0 0x7f8cd81168e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134510 0x7f8cd8134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd814e6f0 con 0x7f8cd8134510 2020-11-20T12:07:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80acb60 con 0x7f8cc00044e0 2020-11-20T12:07:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8ce036c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd814dc30 con 0x7f8cd814d7b0 2020-11-20T12:07:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8cde909700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134510 0x7f8cd8134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8cde909700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134510 0x7f8cd8134970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38452/0 (socket says 172.21.15.44:38452) 2020-11-20T12:07:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8cde909700 1 -- 172.21.15.44:0/2189730761 learned_addr learned my addr 172.21.15.44:0/2189730761 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8cdd907700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd814d7b0 0x7f8cd81168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.251+0000 7f8cde108700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc00044e0 0x7f8cd80ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cde909700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd814d7b0 msgr2=0x7f8cd81168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cde909700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd814d7b0 0x7f8cd81168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cde909700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc00044e0 msgr2=0x7f8cd80ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cde909700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc00044e0 0x7f8cd80ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cde909700 1 -- 172.21.15.44:0/2189730761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80acd90 con 0x7f8cd8134510 2020-11-20T12:07:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cdd907700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cd814d7b0 0x7f8cd81168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:29.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cde909700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134510 0x7f8cd8134970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f8cd8137690 tx=0x7f8cd0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd000c070 con 0x7f8cd8134510 2020-11-20T12:07:29.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd000f910 con 0x7f8cd8134510 2020-11-20T12:07:29.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd80ad030 con 0x7f8cd8134510 2020-11-20T12:07:29.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd000fc90 con 0x7f8cd8134510 2020-11-20T12:07:29.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.252+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd8116dd0 con 0x7f8cd8134510 2020-11-20T12:07:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.253+0000 7f8ce036c700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134e60 0x7f8cd814a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.253+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80ac1c0 con 0x7f8cd8134e60 2020-11-20T12:07:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.253+0000 7f8cde108700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134e60 0x7f8cd814a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.253+0000 7f8cde108700 1 -- 172.21.15.44:0/2189730761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cd80ad1c0 con 0x7f8cd8134e60 2020-11-20T12:07:29.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.253+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8cd0012040 con 0x7f8cd8134510 2020-11-20T12:07:29.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.253+0000 7f8cde108700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134e60 0x7f8cd814a200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8cd4006fd0 tx=0x7f8cd4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.253+0000 7f8cc67fc700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8cb0056d50 0x7f8cb0059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f8cd00704d0 con 0x7f8cd8134510 2020-11-20T12:07:29.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cdd907700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8cb0056d50 0x7f8cb0059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4012d40 con 0x7f8cd8134e60 2020-11-20T12:07:29.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f8cd4012ea0 con 0x7f8cd8134e60 2020-11-20T12:07:29.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134e60 msgr2=0x7f8cd814a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cc67fc700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134e60 0x7f8cd814a200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8cd4006fd0 tx=0x7f8cd4006df0).stop 2020-11-20T12:07:29.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134e60 msgr2=0x7f8cd814a200 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:29.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cc67fc700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134e60 0x7f8cd814a200 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0x7f8cd4006fd0 tx=0x7f8cd4006df0).stop 2020-11-20T12:07:29.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.254+0000 7f8cdd907700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8cb0056d50 0x7f8cb0059210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8cc8001e70 tx=0x7f8cc8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.262+0000 7f8ce036c700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd814a6f0 0x7f8cd814ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.262+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd8061650 con 0x7f8cd814a6f0 2020-11-20T12:07:29.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.262+0000 7f8cde108700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd814a6f0 0x7f8cd814ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.262+0000 7f8cde108700 1 -- 172.21.15.44:0/2189730761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8cd80446d0 con 0x7f8cd814a6f0 2020-11-20T12:07:29.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.262+0000 7f8cde108700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd814a6f0 0x7f8cd814ab00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8cd4006fd0 tx=0x7f8cd401d000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.262+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4025070 con 0x7f8cd814a6f0 2020-11-20T12:07:29.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4268 (secure 0 0 0) 0x7f8cd4012ae0 con 0x7f8cd814a6f0 2020-11-20T12:07:29.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd814a6f0 msgr2=0x7f8cd814ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8cc67fc700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd814a6f0 0x7f8cd814ab00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8cd4006fd0 tx=0x7f8cd401d000).stop 2020-11-20T12:07:29.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8cc67fc700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd814a6f0 msgr2=0x7f8cd814ab00 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:29.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8cc67fc700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd814a6f0 0x7f8cd814ab00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8cb0056d50 msgr2=0x7f8cb0059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8cb0056d50 0x7f8cb0059210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8cc8001e70 tx=0x7f8cc8015000).stop 2020-11-20T12:07:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134510 msgr2=0x7f8cd8134970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 --2- 172.21.15.44:0/2189730761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd8134510 0x7f8cd8134970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f8cd8137690 tx=0x7f8cd0008040).stop 2020-11-20T12:07:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8cde909700 1 -- 172.21.15.44:0/2189730761 reap_dead start 2020-11-20T12:07:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8cde909700 1 -- 172.21.15.44:0/2189730761 reap_dead start 2020-11-20T12:07:29.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 shutdown_connections 2020-11-20T12:07:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 >> 172.21.15.44:0/2189730761 conn(0x7f8cd8062690 msgr2=0x7f8cd80a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 shutdown_connections 2020-11-20T12:07:29.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.263+0000 7f8ce036c700 1 -- 172.21.15.44:0/2189730761 wait complete. 2020-11-20T12:07:29.282 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:29.283 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:07:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 38, 2020-11-20T12:07:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:07:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:07:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:07:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:29.314 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:29.316 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:29.316 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:29.316 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:29.316 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:29.316 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:29.317 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:29.317 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:29.317 INFO:tasks.workunit.client.0.smithi044.stdout: 36190 ? 00:00:00 ps 2020-11-20T12:07:29.318 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:07:29.318 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:29.318 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:29.319 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:29.319 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:29.319 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:29.319 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:29.320 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:29.320 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:29.325 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:29.326 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:29.398 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:29.411 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:29.411 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:29.411 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:29.411 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:29.412 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:29.412 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:29.412 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:29.412 INFO:tasks.workunit.client.0.smithi044.stdout: 36221 ? 00:00:00 ps 2020-11-20T12:07:29.413 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:29.431 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:29.433 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:29.441 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:29.441 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:29.444 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:29.444 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:29.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.451+0000 7f935c149700 1 Processor -- start 2020-11-20T12:07:29.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.451+0000 7f935c149700 1 -- start start 2020-11-20T12:07:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f935c149700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93540aa020 0x7f93540a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f935c149700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f935413ac60 0x7f93540a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f935c149700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93540a9770 con 0x7f93540aa400 2020-11-20T12:07:29.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f935c149700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93540a56c0 con 0x7f93540aa020 2020-11-20T12:07:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f935c149700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93540a6da0 con 0x7f935413ac60 2020-11-20T12:07:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f93596e4700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f93540aa400 0x7f93540a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42210/0 (socket says 172.21.15.44:42210) 2020-11-20T12:07:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f93596e4700 1 -- 172.21.15.44:0/3633704381 learned_addr learned my addr 172.21.15.44:0/3633704381 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:29.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f9359ee5700 1 --2- 172.21.15.44:0/3633704381 >> v2:172.21.15.79:3300/0 conn(0x7f93540aa020 0x7f93540a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f935a6e6700 1 --2- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f935413ac60 0x7f93540a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f9358ee3700 1 -- 172.21.15.44:0/3633704381 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18480201 0 0) 0x7f93440010d0 con 0x7f93540aa400 2020-11-20T12:07:29.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f9358ee3700 1 -- 172.21.15.44:0/3633704381 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f933c003240 con 0x7f93540aa400 2020-11-20T12:07:29.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f9359ee5700 1 -- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f935413ac60 msgr2=0x7f93540a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:29.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f9359ee5700 1 --2- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f935413ac60 0x7f93540a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f9359ee5700 1 -- 172.21.15.44:0/3633704381 >> v1:172.21.15.44:6789/0 conn(0x7f93540aa400 legacy=0x7f93540a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.452+0000 7f9359ee5700 1 -- 172.21.15.44:0/3633704381 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540a7040 con 0x7f93540aa020 2020-11-20T12:07:29.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.453+0000 7f935a6e6700 1 --2- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f935413ac60 0x7f93540a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:07:29.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.453+0000 7f9359ee5700 1 --2- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 0x7f93540a9280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f935000b9a0 tx=0x7f9350006df0).ready entity=mon.1 client_cookie=ec82a144364c2e64 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.453+0000 7f9358ee3700 1 -- 172.21.15.44:0/3633704381 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350009680 con 0x7f93540aa020 2020-11-20T12:07:29.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.453+0000 7f9358ee3700 1 -- 172.21.15.44:0/3633704381 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500097e0 con 0x7f93540aa020 2020-11-20T12:07:29.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.453+0000 7f9358ee3700 1 -- 172.21.15.44:0/3633704381 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350009b60 con 0x7f93540aa020 2020-11-20T12:07:29.461 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:29.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 -- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 msgr2=0x7f93540a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 --2- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 0x7f93540a9280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f935000b9a0 tx=0x7f9350006df0).stop 2020-11-20T12:07:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 -- 172.21.15.44:0/3633704381 shutdown_connections 2020-11-20T12:07:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 --2- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f935413ac60 0x7f93540a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 --2- 172.21.15.44:0/3633704381 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 0x7f93540a9280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 -- 172.21.15.44:0/3633704381 >> 172.21.15.44:0/3633704381 conn(0x7f9354062690 msgr2=0x7f93540a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 -- 172.21.15.44:0/3633704381 shutdown_connections 2020-11-20T12:07:29.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.454+0000 7f935c149700 1 -- 172.21.15.44:0/3633704381 wait complete. 2020-11-20T12:07:29.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.455+0000 7f935c149700 1 Processor -- start 2020-11-20T12:07:29.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.455+0000 7f935c149700 1 -- start start 2020-11-20T12:07:29.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.455+0000 7f935c149700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 0x7f93540acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935c149700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93540aa400 0x7f93541323b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935c149700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f935413ac60 0x7f9354149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935c149700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540681e0 con 0x7f935413ac60 2020-11-20T12:07:29.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935c149700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354145b70 con 0x7f93540aa020 2020-11-20T12:07:29.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935c149700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9354145e10 con 0x7f93540aa400 2020-11-20T12:07:29.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f935413ac60 0x7f9354149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f935413ac60 0x7f9354149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38468/0 (socket says 172.21.15.44:38468) 2020-11-20T12:07:29.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 -- 172.21.15.44:0/500211106 learned_addr learned my addr 172.21.15.44:0/500211106 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:29.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f9359ee5700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 0x7f93540acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f93596e4700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93540aa400 0x7f93541323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93540aa400 msgr2=0x7f93541323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93540aa400 0x7f93541323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 msgr2=0x7f93540acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540aa020 0x7f93540acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 -- 172.21.15.44:0/500211106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93541460b0 con 0x7f935413ac60 2020-11-20T12:07:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f93596e4700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f93540aa400 0x7f93541323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935a6e6700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f935413ac60 0x7f9354149140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f934c00e6a0 tx=0x7f934c00b470).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c00ba20 con 0x7f935413ac60 2020-11-20T12:07:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c0117f0 con 0x7f935413ac60 2020-11-20T12:07:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.456+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9354146350 con 0x7f935413ac60 2020-11-20T12:07:29.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.457+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c011bb0 con 0x7f935413ac60 2020-11-20T12:07:29.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.457+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93541467f0 con 0x7f935413ac60 2020-11-20T12:07:29.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.458+0000 7f935c149700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9354149630 0x7f9354149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.458+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354132a60 con 0x7f9354149630 2020-11-20T12:07:29.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.458+0000 7f9359ee5700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9354149630 0x7f9354149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.458+0000 7f9359ee5700 1 -- 172.21.15.44:0/500211106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93540a3d30 con 0x7f9354149630 2020-11-20T12:07:29.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.458+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f934c01a470 con 0x7f935413ac60 2020-11-20T12:07:29.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.458+0000 7f9359ee5700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9354149630 0x7f9354149a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f935000b9a0 tx=0x7f9350015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f932c056d20 0x7f932c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f934c0735d0 con 0x7f935413ac60 2020-11-20T12:07:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f93596e4700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f932c056d20 0x7f932c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350023030 con 0x7f9354149630 2020-11-20T12:07:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f9350011a70 con 0x7f9354149630 2020-11-20T12:07:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9354149630 msgr2=0x7f9354149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9354149630 0x7f9354149a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f935000b9a0 tx=0x7f9350015000).stop 2020-11-20T12:07:29.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9354149630 msgr2=0x7f9354149a40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:29.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f9341ffb700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9354149630 0x7f9354149a40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.459+0000 7f93596e4700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f932c056d20 0x7f932c0591e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f9354131eb0 tx=0x7f9344009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.476 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:29.476 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:29.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.468+0000 7f935c149700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540650a0 0x7f9354065480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.468+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354146aa0 con 0x7f93540650a0 2020-11-20T12:07:29.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9359ee5700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540650a0 0x7f9354065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9359ee5700 1 -- 172.21.15.44:0/500211106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9354061650 con 0x7f93540650a0 2020-11-20T12:07:29.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9359ee5700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540650a0 0x7f9354065480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9350009380 tx=0x7f9350021070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350033060 con 0x7f93540650a0 2020-11-20T12:07:29.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f9350011860 con 0x7f93540650a0 2020-11-20T12:07:29.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540650a0 msgr2=0x7f9354065480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9341ffb700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540650a0 0x7f9354065480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9350009380 tx=0x7f9350021070).stop 2020-11-20T12:07:29.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9341ffb700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540650a0 msgr2=0x7f9354065480 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:29.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.469+0000 7f9341ffb700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93540650a0 0x7f9354065480 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0x7f9350009380 tx=0x7f9350021070).stop 2020-11-20T12:07:29.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f932c056d20 msgr2=0x7f932c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f932c056d20 0x7f932c0591e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f9354131eb0 tx=0x7f9344009040).stop 2020-11-20T12:07:29.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f935413ac60 msgr2=0x7f9354149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 --2- 172.21.15.44:0/500211106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f935413ac60 0x7f9354149140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f934c00e6a0 tx=0x7f934c00b470).stop 2020-11-20T12:07:29.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935a6e6700 1 -- 172.21.15.44:0/500211106 reap_dead start 2020-11-20T12:07:29.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935a6e6700 1 -- 172.21.15.44:0/500211106 reap_dead start 2020-11-20T12:07:29.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 shutdown_connections 2020-11-20T12:07:29.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 >> 172.21.15.44:0/500211106 conn(0x7f9354062690 msgr2=0x7f93540a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 shutdown_connections 2020-11-20T12:07:29.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.470+0000 7f935c149700 1 -- 172.21.15.44:0/500211106 wait complete. 2020-11-20T12:07:29.481 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:07:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 38, 2020-11-20T12:07:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:07:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:07:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:07:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:07:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:07:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:07:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:29.515 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:29.516 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:07:29.566 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:29.577 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:29.577 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:29.577 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:29.578 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:29.578 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:29.578 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:29.578 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:29.578 INFO:tasks.workunit.client.0.smithi044.stdout: 36264 ? 00:00:00 ps 2020-11-20T12:07:29.579 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:29.596 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:29.599 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:29.606 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:29.606 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:29.609 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:29.609 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:29.623 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:29.631 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:29.632 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:29.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.689+0000 7f03e7dca700 1 Processor -- start 2020-11-20T12:07:29.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.689+0000 7f03e7dca700 1 -- start start 2020-11-20T12:07:29.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e7dca700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03e00aa020 0x7f03e00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e7dca700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f03e013ac60 0x7f03e00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e7dca700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00a9770 con 0x7f03e00aa400 2020-11-20T12:07:29.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e7dca700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e00a56c0 con 0x7f03e00aa020 2020-11-20T12:07:29.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e7dca700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e00a6da0 con 0x7f03e013ac60 2020-11-20T12:07:29.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5b66700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03e00aa020 0x7f03e00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5365700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f03e00aa400 0x7f03e00a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42228/0 (socket says 172.21.15.44:42228) 2020-11-20T12:07:29.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5365700 1 -- 172.21.15.44:0/3220487823 learned_addr learned my addr 172.21.15.44:0/3220487823 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:29.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e6367700 1 --2- 172.21.15.44:0/3220487823 >> v2:172.21.15.79:3300/0 conn(0x7f03e013ac60 0x7f03e00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5b66700 1 -- 172.21.15.44:0/3220487823 >> v2:172.21.15.79:3300/0 conn(0x7f03e013ac60 msgr2=0x7f03e00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:29.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5b66700 1 --2- 172.21.15.44:0/3220487823 >> v2:172.21.15.79:3300/0 conn(0x7f03e013ac60 0x7f03e00a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5b66700 1 -- 172.21.15.44:0/3220487823 >> v1:172.21.15.44:6789/0 conn(0x7f03e00aa400 legacy=0x7f03e00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e4b64700 1 -- 172.21.15.44:0/3220487823 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23081230 0 0) 0x7f03dc0010d0 con 0x7f03e00aa400 2020-11-20T12:07:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5b66700 1 -- 172.21.15.44:0/3220487823 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00a7040 con 0x7f03e00aa020 2020-11-20T12:07:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e5b66700 1 --2- 172.21.15.44:0/3220487823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e00aa020 0x7f03e00a9280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f03e00a8f70 tx=0x7f03d0006df0).ready entity=mon.0 client_cookie=fc1afc78af02bab6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e4b64700 1 -- 172.21.15.44:0/3220487823 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d00097b0 con 0x7f03e00aa020 2020-11-20T12:07:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e4b64700 1 -- 172.21.15.44:0/3220487823 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0009910 con 0x7f03e00aa020 2020-11-20T12:07:29.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.690+0000 7f03e4b64700 1 -- 172.21.15.44:0/3220487823 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0009c90 con 0x7f03e00aa020 2020-11-20T12:07:29.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 -- 172.21.15.44:0/3220487823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e00aa020 msgr2=0x7f03e00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 --2- 172.21.15.44:0/3220487823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e00aa020 0x7f03e00a9280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f03e00a8f70 tx=0x7f03d0006df0).stop 2020-11-20T12:07:29.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 -- 172.21.15.44:0/3220487823 shutdown_connections 2020-11-20T12:07:29.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 --2- 172.21.15.44:0/3220487823 >> v2:172.21.15.79:3300/0 conn(0x7f03e013ac60 0x7f03e00a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 --2- 172.21.15.44:0/3220487823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e00aa020 0x7f03e00a9280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 -- 172.21.15.44:0/3220487823 >> 172.21.15.44:0/3220487823 conn(0x7f03e0062690 msgr2=0x7f03e00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 -- 172.21.15.44:0/3220487823 shutdown_connections 2020-11-20T12:07:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.691+0000 7f03e7dca700 1 -- 172.21.15.44:0/3220487823 wait complete. 2020-11-20T12:07:29.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.692+0000 7f03e7dca700 1 Processor -- start 2020-11-20T12:07:29.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.692+0000 7f03e7dca700 1 -- start start 2020-11-20T12:07:29.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e7dca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00aa020 0x7f03e0066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e7dca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03e00aa400 0x7f03e013ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e7dca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e013ac60 0x7f03e014a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e7dca700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00a38a0 con 0x7f03e013ac60 2020-11-20T12:07:29.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e7dca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e014a6c0 con 0x7f03e00aa400 2020-11-20T12:07:29.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e7dca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e014a950 con 0x7f03e00aa020 2020-11-20T12:07:29.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e013ac60 0x7f03e014a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e013ac60 0x7f03e014a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38488/0 (socket says 172.21.15.44:38488) 2020-11-20T12:07:29.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 -- 172.21.15.44:0/1838847673 learned_addr learned my addr 172.21.15.44:0/1838847673 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:29.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e5b66700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00aa020 0x7f03e0066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e5365700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03e00aa400 0x7f03e013ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00aa020 msgr2=0x7f03e0066900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00aa020 0x7f03e0066900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03e00aa400 msgr2=0x7f03e013ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03e00aa400 0x7f03e013ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 -- 172.21.15.44:0/1838847673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e014abf0 con 0x7f03e013ac60 2020-11-20T12:07:29.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e5b66700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00aa020 0x7f03e0066900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:29.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e5365700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03e00aa400 0x7f03e013ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e6367700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e013ac60 0x7f03e014a1d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f03d800ef80 tx=0x7f03d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8009e00 con 0x7f03e013ac60 2020-11-20T12:07:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d801a470 con 0x7f03e013ac60 2020-11-20T12:07:29.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e014ae90 con 0x7f03e013ac60 2020-11-20T12:07:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.693+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d801a810 con 0x7f03e013ac60 2020-11-20T12:07:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.694+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e00ad730 con 0x7f03e013ac60 2020-11-20T12:07:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.694+0000 7f03e7dca700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00ad9b0 0x7f03e0146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.694+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e0136eb0 con 0x7f03e00ad9b0 2020-11-20T12:07:29.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03e5b66700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00ad9b0 0x7f03e0146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f03d8021060 con 0x7f03e013ac60 2020-11-20T12:07:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03e5b66700 1 -- 172.21.15.44:0/1838847673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03e00add90 con 0x7f03e00ad9b0 2020-11-20T12:07:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03cdffb700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03b8056d20 0x7f03b80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f03d8072a70 con 0x7f03e013ac60 2020-11-20T12:07:29.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03e5b66700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00ad9b0 0x7f03e0146c70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f03e00a8f70 tx=0x7f03d0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03e5365700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03b8056d20 0x7f03b80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.695+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011dd0 con 0x7f03e00ad9b0 2020-11-20T12:07:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.696+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f03d001f440 con 0x7f03e00ad9b0 2020-11-20T12:07:29.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.696+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00ad9b0 msgr2=0x7f03e0146c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.696+0000 7f03cdffb700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00ad9b0 0x7f03e0146c70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f03e00a8f70 tx=0x7f03d0015000).stop 2020-11-20T12:07:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.696+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00ad9b0 msgr2=0x7f03e0146c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.696+0000 7f03cdffb700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e00ad9b0 0x7f03e0146c70 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0x7f03e00a8f70 tx=0x7f03d0015000).stop 2020-11-20T12:07:29.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.696+0000 7f03e5365700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03b8056d20 0x7f03b80591e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f03e0132c50 tx=0x7f03dc009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.703+0000 7f03e7dca700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e0147160 0x7f03e00aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.703+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e014b020 con 0x7f03e0147160 2020-11-20T12:07:29.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03e5b66700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e0147160 0x7f03e00aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03e5b66700 1 -- 172.21.15.44:0/1838847673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03e0061650 con 0x7f03e0147160 2020-11-20T12:07:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03e5b66700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e0147160 0x7f03e00aac60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f03e0136ff0 tx=0x7f03d0020000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0028450 con 0x7f03e0147160 2020-11-20T12:07:29.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f03d0011bc0 con 0x7f03e0147160 2020-11-20T12:07:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e0147160 msgr2=0x7f03e00aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03cdffb700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e0147160 0x7f03e00aac60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f03e0136ff0 tx=0x7f03d0020000).stop 2020-11-20T12:07:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03cdffb700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e0147160 msgr2=0x7f03e00aac60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.704+0000 7f03cdffb700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03e0147160 0x7f03e00aac60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03b8056d20 msgr2=0x7f03b80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03b8056d20 0x7f03b80591e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f03e0132c50 tx=0x7f03dc009040).stop 2020-11-20T12:07:29.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e013ac60 msgr2=0x7f03e014a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 --2- 172.21.15.44:0/1838847673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e013ac60 0x7f03e014a1d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f03d800ef80 tx=0x7f03d8008040).stop 2020-11-20T12:07:29.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e6367700 1 -- 172.21.15.44:0/1838847673 reap_dead start 2020-11-20T12:07:29.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e6367700 1 -- 172.21.15.44:0/1838847673 reap_dead start 2020-11-20T12:07:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 shutdown_connections 2020-11-20T12:07:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 >> 172.21.15.44:0/1838847673 conn(0x7f03e0062690 msgr2=0x7f03e00a40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 shutdown_connections 2020-11-20T12:07:29.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.705+0000 7f03e7dca700 1 -- 172.21.15.44:0/1838847673 wait complete. 2020-11-20T12:07:29.719 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:07:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 38, 2020-11-20T12:07:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:07:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:07:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:07:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:07:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:07:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:07:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:07:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:29.755 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:29.756 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:29.757 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:29.757 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:29.757 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:29.757 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:29.758 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:29.758 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:29.758 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:29.758 INFO:tasks.workunit.client.0.smithi044.stdout: 36306 ? 00:00:00 ps 2020-11-20T12:07:29.759 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:07:29.759 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:07:29.759 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:07:29.760 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:07:29.760 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:07:29.760 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:07:29.761 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:29.772 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:07:29.773 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:07:29.773 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:29.778 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:29.781 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:29.787 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:29.787 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:29.790 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:29.790 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:29.804 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:29.812 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:29.814 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:29.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5ada28700 1 Processor -- start 2020-11-20T12:07:29.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5ada28700 1 -- start start 2020-11-20T12:07:29.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5ada28700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc5a8060330 0x7fc5a80a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5ada28700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc5a813b0a0 0x7fc5a8141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5ada28700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80a6260 con 0x7fc5a80a4c30 2020-11-20T12:07:29.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5ada28700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80a52e0 con 0x7fc5a8060330 2020-11-20T12:07:29.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5ada28700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80a6490 con 0x7fc5a813b0a0 2020-11-20T12:07:29.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5a6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc5a8060330 0x7fc5a80a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5a77fe700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7fc5a813b0a0 msgr2=0x7fc5a8141810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:29.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5a67fc700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fc5a80a4c30 legacy=0x7fc5a813a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:29.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5a77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc5a813b0a0 0x7fc5a8141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:29.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5a6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc5a8060330 0x7fc5a80a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38514/0 (socket says 172.21.15.44:38514) 2020-11-20T12:07:29.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.899+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/887703854 learned_addr learned my addr 172.21.15.44:0/887703854 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:29.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/887703854 >> v2:172.21.15.79:3300/0 conn(0x7fc5a813b0a0 msgr2=0x7fc5a8141810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:29.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a6ffd700 1 --2- 172.21.15.44:0/887703854 >> v2:172.21.15.79:3300/0 conn(0x7fc5a813b0a0 0x7fc5a8141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:29.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/887703854 >> v1:172.21.15.79:6789/0 conn(0x7fc5a80a4c30 legacy=0x7fc5a813a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:29.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/887703854 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80a6730 con 0x7fc5a8060330 2020-11-20T12:07:29.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a6ffd700 1 --2- 172.21.15.44:0/887703854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a80a4740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc5a8061250 tx=0x7fc59c006f40).ready entity=mon.0 client_cookie=4f1ccf921c8d0df3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a5ffb700 1 -- 172.21.15.44:0/887703854 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00caf0 con 0x7fc5a8060330 2020-11-20T12:07:29.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a5ffb700 1 -- 172.21.15.44:0/887703854 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00cc50 con 0x7fc5a8060330 2020-11-20T12:07:29.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5a5ffb700 1 -- 172.21.15.44:0/887703854 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c010410 con 0x7fc5a8060330 2020-11-20T12:07:29.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5ada28700 1 -- 172.21.15.44:0/887703854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 msgr2=0x7fc5a80a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5ada28700 1 --2- 172.21.15.44:0/887703854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a80a4740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fc5a8061250 tx=0x7fc59c006f40).stop 2020-11-20T12:07:29.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5ada28700 1 -- 172.21.15.44:0/887703854 shutdown_connections 2020-11-20T12:07:29.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5ada28700 1 --2- 172.21.15.44:0/887703854 >> v2:172.21.15.79:3300/0 conn(0x7fc5a813b0a0 0x7fc5a8141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:29.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.900+0000 7fc5ada28700 1 --2- 172.21.15.44:0/887703854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a80a4740 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.901+0000 7fc5ada28700 1 -- 172.21.15.44:0/887703854 >> 172.21.15.44:0/887703854 conn(0x7fc5a8062690 msgr2=0x7fc5a80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:29.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.901+0000 7fc5ada28700 1 -- 172.21.15.44:0/887703854 shutdown_connections 2020-11-20T12:07:29.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.901+0000 7fc5ada28700 1 -- 172.21.15.44:0/887703854 wait complete. 2020-11-20T12:07:29.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 Processor -- start 2020-11-20T12:07:29.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 -- start start 2020-11-20T12:07:29.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a8066c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5a80a4c30 0x7fc5a80ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a813b0a0 0x7fc5a80aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81482b0 con 0x7fc5a8060330 2020-11-20T12:07:29.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8148550 con 0x7fc5a813b0a0 2020-11-20T12:07:29.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5ada28700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8148c90 con 0x7fc5a80a4c30 2020-11-20T12:07:29.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5a6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a8066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5a6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a8066c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38520/0 (socket says 172.21.15.44:38520) 2020-11-20T12:07:29.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/3012808677 learned_addr learned my addr 172.21.15.44:0/3012808677 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:29.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5a77fe700 1 -- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a813b0a0 msgr2=0x7fc5a80aac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:29.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.902+0000 7fc5a77fe700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a813b0a0 0x7fc5a80aac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:29.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5a67fc700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5a80a4c30 0x7fc5a80ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5a80a4c30 msgr2=0x7fc5a80ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:29.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5a6ffd700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5a80a4c30 0x7fc5a80ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:29.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a813b0a0 msgr2=0x7fc5a80aac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:29.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5a6ffd700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a813b0a0 0x7fc5a80aac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:29.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5a6ffd700 1 -- 172.21.15.44:0/3012808677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80ad790 con 0x7fc5a8060330 2020-11-20T12:07:29.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5a6ffd700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a8066c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fc59c006a10 tx=0x7fc5a8137f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc58effd700 1 -- 172.21.15.44:0/3012808677 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c01a6d0 con 0x7fc5a8060330 2020-11-20T12:07:29.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc58effd700 1 -- 172.21.15.44:0/3012808677 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c01a830 con 0x7fc5a8060330 2020-11-20T12:07:29.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80ab150 con 0x7fc5a8060330 2020-11-20T12:07:29.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc58effd700 1 -- 172.21.15.44:0/3012808677 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c01abb0 con 0x7fc5a8060330 2020-11-20T12:07:29.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.903+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a8147fd0 con 0x7fc5a8060330 2020-11-20T12:07:29.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.904+0000 7fc58effd700 1 -- 172.21.15.44:0/3012808677 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc59c00e040 con 0x7fc5a8060330 2020-11-20T12:07:29.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.904+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8061650 con 0x7fc5a8060330 2020-11-20T12:07:29.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.904+0000 7fc58effd700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc584056d00 0x7fc5840591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:29.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.904+0000 7fc58effd700 1 -- 172.21.15.44:0/3012808677 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fc59c071d20 con 0x7fc5a8060330 2020-11-20T12:07:29.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.905+0000 7fc5a67fc700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc584056d00 0x7fc5840591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:29.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.905+0000 7fc5a67fc700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc584056d00 0x7fc5840591c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc5a80673b0 tx=0x7fc598006c50).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:29.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:29.908+0000 7fc58effd700 1 -- 172.21.15.44:0/3012808677 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc59c046fb0 con 0x7fc5a8060330 2020-11-20T12:07:29.944 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:29.955 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:29.955 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:29.955 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:29.955 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:29.956 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:29.956 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:29.956 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:29.956 INFO:tasks.workunit.client.0.smithi044.stdout: 36356 ? 00:00:00 ps 2020-11-20T12:07:29.957 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:29.974 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:29.977 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:29.983 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:29.983 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:29.985 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:29.986 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:29.999 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:30.006 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:30.007 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:30.109 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:30.120 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:30.120 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:30.120 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:30.120 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:30.121 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:30.121 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:30.121 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:30.121 INFO:tasks.workunit.client.0.smithi044.stdout: 36366 ? 00:00:00 ps 2020-11-20T12:07:30.122 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:30.138 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:30.141 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:30.147 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:30.147 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:30.150 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:30.150 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:30.164 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:30.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.161+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc5a80446d0 con 0x7fc5a8060330 2020-11-20T12:07:30.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.162+0000 7fc58effd700 1 -- 172.21.15.44:0/3012808677 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fc59c04a5d0 con 0x7fc5a8060330 2020-11-20T12:07:30.166 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:30.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc584056d00 msgr2=0x7fc5840591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:30.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc584056d00 0x7fc5840591c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc5a80673b0 tx=0x7fc598006c50).stop 2020-11-20T12:07:30.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 msgr2=0x7fc5a8066c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:30.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a8066c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fc59c006a10 tx=0x7fc5a8137f50).stop 2020-11-20T12:07:30.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 shutdown_connections 2020-11-20T12:07:30.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc584056d00 0x7fc5840591c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:30.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5a813b0a0 0x7fc5a80aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:30.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5a8060330 0x7fc5a8066c40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:30.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 --2- 172.21.15.44:0/3012808677 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc5a80a4c30 0x7fc5a80ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 >> 172.21.15.44:0/3012808677 conn(0x7fc5a8062690 msgr2=0x7fc5a813a760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 shutdown_connections 2020-11-20T12:07:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:30.164+0000 7fc5ada28700 1 -- 172.21.15.44:0/3012808677 wait complete. 2020-11-20T12:07:30.172 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:30.174 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:30.242 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:30.253 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:30.254 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:30.254 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:30.254 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:30.254 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:30.255 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:30.255 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:30.255 INFO:tasks.workunit.client.0.smithi044.stdout: 36378 ? 00:00:00 ps 2020-11-20T12:07:30.255 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:30.271 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:30.274 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:30.280 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:30.280 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:30.282 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:30.282 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:30.295 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:30.303 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:30.305 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:30.394 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:30.405 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:30.406 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:30.406 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:30.406 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:30.406 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:30.407 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:30.407 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:30.407 INFO:tasks.workunit.client.0.smithi044.stdout: 36388 ? 00:00:00 ps 2020-11-20T12:07:30.408 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:30.425 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:30.427 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:30.433 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:30.434 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:30.436 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:30.436 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:30.449 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:30.457 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:30.458 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:30.539 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:30.549 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:30.550 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:30.550 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:30.550 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:30.550 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:30.551 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:30.551 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:30.551 INFO:tasks.workunit.client.0.smithi044.stdout: 36398 ? 00:00:00 ps 2020-11-20T12:07:30.551 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:30.567 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:30.569 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:30.575 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:30.576 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:30.578 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:30.578 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:30.590 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:30.598 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:30.599 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:30.776 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:30.788 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:30.788 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:30.788 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:30.789 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:30.789 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:30.789 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:30.789 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:30.789 INFO:tasks.workunit.client.0.smithi044.stdout: 36408 ? 00:00:00 ps 2020-11-20T12:07:30.790 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:30.808 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:30.811 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:30.817 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:30.817 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:30.819 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:30.820 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:30.834 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:30.842 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:30.844 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:30.911 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:30.922 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:30.922 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:30.922 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:30.922 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:30.923 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:30.923 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:30.923 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:30.923 INFO:tasks.workunit.client.0.smithi044.stdout: 36418 ? 00:00:00 ps 2020-11-20T12:07:30.924 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:30.942 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:30.944 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:30.950 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:30.950 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:30.953 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:30.953 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:30.966 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:30.974 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:30.975 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:31.035 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:31.046 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:31.047 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:31.048 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:31.048 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:31.048 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:31.049 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:31.050 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:31.050 INFO:tasks.workunit.client.0.smithi044.stdout: 36428 ? 00:00:00 ps 2020-11-20T12:07:31.052 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:31.067 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:31.069 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:31.076 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:31.076 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:31.078 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:31.079 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:31.092 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:31.101 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:31.102 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:31.177 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:31.188 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:31.189 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:31.189 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:31.189 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:31.189 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:31.189 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:31.190 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:31.190 INFO:tasks.workunit.client.0.smithi044.stdout: 36438 ? 00:00:00 ps 2020-11-20T12:07:31.190 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:31.206 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:31.209 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:31.214 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:31.215 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:31.218 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:31.218 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:31.230 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:31.238 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:31.239 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:31.319 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:31.330 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:31.331 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:31.331 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:31.331 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:31.332 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:31.332 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:31.332 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:31.333 INFO:tasks.workunit.client.0.smithi044.stdout: 36448 ? 00:00:00 ps 2020-11-20T12:07:31.333 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:31.349 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:31.351 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:31.358 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:31.358 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:31.361 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:31.361 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:31.375 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:31.384 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:31.385 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:31.503 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:31.514 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:31.514 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:31.514 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:31.514 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:31.515 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:31.515 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:31.515 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:31.515 INFO:tasks.workunit.client.0.smithi044.stdout: 36458 ? 00:00:00 ps 2020-11-20T12:07:31.516 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:31.533 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:31.536 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:31.542 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:31.542 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:31.545 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:31.545 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:31.560 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:31.570 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:31.571 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:31.668 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:31.679 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:31.679 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:31.679 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:31.679 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:31.680 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:31.680 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:31.680 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:31.680 INFO:tasks.workunit.client.0.smithi044.stdout: 36468 ? 00:00:00 ps 2020-11-20T12:07:31.681 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:31.698 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:31.700 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:31.706 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:31.707 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:31.709 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:31.709 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:31.722 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:31.730 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:31.731 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:31.812 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:31.823 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:31.823 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:31.824 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:31.824 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:31.824 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:31.825 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:31.825 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:31.825 INFO:tasks.workunit.client.0.smithi044.stdout: 36478 ? 00:00:00 ps 2020-11-20T12:07:31.826 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:31.842 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:31.844 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:31.850 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:31.851 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:31.853 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:31.853 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:31.867 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:31.874 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:31.876 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:31.980 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:31.991 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:31.991 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:31.991 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:31.992 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:31.992 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:31.992 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:31.992 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:31.992 INFO:tasks.workunit.client.0.smithi044.stdout: 36488 ? 00:00:00 ps 2020-11-20T12:07:31.993 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:32.010 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:32.013 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:32.019 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 20` 2020-11-20T12:07:32.019 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a 2020-11-20T12:07:32.021 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/one 2020-11-20T12:07:32.022 INFO:tasks.workunit.client.0.smithi044.stderr:+ cat 2020-11-20T12:07:32.035 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir a/.snap/two 2020-11-20T12:07:32.043 INFO:tasks.workunit.client.0.smithi044.stderr:+ chmod 777 a/foo 2020-11-20T12:07:32.044 INFO:tasks.workunit.client.0.smithi044.stderr:+ sync 2020-11-20T12:07:32.121 INFO:tasks.workunit.client.0.smithi044.stderr:+ ps 2020-11-20T12:07:32.133 INFO:tasks.workunit.client.0.smithi044.stdout: PID TTY TIME CMD 2020-11-20T12:07:32.133 INFO:tasks.workunit.client.0.smithi044.stdout: 7176 ? 00:00:00 systemd 2020-11-20T12:07:32.133 INFO:tasks.workunit.client.0.smithi044.stdout: 7180 ? 00:00:00 (sd-pam) 2020-11-20T12:07:32.133 INFO:tasks.workunit.client.0.smithi044.stdout: 14125 ? 00:00:00 sshd 2020-11-20T12:07:32.134 INFO:tasks.workunit.client.0.smithi044.stdout: 29933 ? 00:00:00 bash 2020-11-20T12:07:32.134 INFO:tasks.workunit.client.0.smithi044.stdout: 36048 ? 00:00:00 timeout 2020-11-20T12:07:32.134 INFO:tasks.workunit.client.0.smithi044.stdout: 36065 ? 00:00:00 snaptest-double 2020-11-20T12:07:32.134 INFO:tasks.workunit.client.0.smithi044.stdout: 36498 ? 00:00:00 ps 2020-11-20T12:07:32.135 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a/.snap/one a/.snap/two 2020-11-20T12:07:32.151 INFO:tasks.workunit.client.0.smithi044.stderr:+ rm a/foo 2020-11-20T12:07:32.154 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir a 2020-11-20T12:07:32.160 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:07:32.160 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:07:32.161 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:07:32.161 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:07:32.199 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2020-11-20T12:07:32.200 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2020-11-20T12:07:32.263 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:07:32.365 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.361+0000 7f12b0cd3700 1 Processor -- start 2020-11-20T12:07:32.365 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.361+0000 7f12b0cd3700 1 -- start start 2020-11-20T12:07:32.368 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12b0cd3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f12ac0a8b40 0x7f12ac13aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:32.368 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12b0cd3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f12ac13b3b0 0x7f12ac1380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:32.368 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12b0cd3700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac064ed0 con 0x7f12ac0a3930 2020-11-20T12:07:32.368 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12b0cd3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac0a6320 con 0x7f12ac13b3b0 2020-11-20T12:07:32.368 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12b0cd3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac0a5be0 con 0x7f12ac0a8b40 2020-11-20T12:07:32.369 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12aa59c700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f12ac0a8b40 msgr2=0x7f12ac13aec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:32.369 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a9d9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f12ac13b3b0 0x7f12ac1380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:32.369 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12aa59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f12ac0a8b40 0x7f12ac13aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:32.369 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12aad9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f12ac0a3930 0x7f12ac1385a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42284/0 (socket says 172.21.15.44:42284) 2020-11-20T12:07:32.370 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12aad9d700 1 -- 172.21.15.44:0/3481626000 learned_addr learned my addr 172.21.15.44:0/3481626000 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:32.370 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a9d9b700 1 -- 172.21.15.44:0/3481626000 >> v2:172.21.15.79:3300/0 conn(0x7f12ac0a8b40 msgr2=0x7f12ac13aec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:32.370 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a9d9b700 1 --2- 172.21.15.44:0/3481626000 >> v2:172.21.15.79:3300/0 conn(0x7f12ac0a8b40 0x7f12ac13aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:32.370 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a9d9b700 1 -- 172.21.15.44:0/3481626000 >> v1:172.21.15.44:6789/0 conn(0x7f12ac0a3930 legacy=0x7f12ac1385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:32.370 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a959a700 1 -- 172.21.15.44:0/3481626000 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148738999 0 0) 0x7f12ac064ed0 con 0x7f12ac0a3930 2020-11-20T12:07:32.371 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a9d9b700 1 -- 172.21.15.44:0/3481626000 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac0a5e80 con 0x7f12ac13b3b0 2020-11-20T12:07:32.371 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a9d9b700 1 --2- 172.21.15.44:0/3481626000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac13b3b0 0x7f12ac1380b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f12ac0a74a0 tx=0x7f129c0091d0).ready entity=mon.0 client_cookie=d06b38e79d204fb1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:32.371 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.362+0000 7f12a959a700 1 -- 172.21.15.44:0/3481626000 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00bbc0 con 0x7f12ac13b3b0 2020-11-20T12:07:32.371 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12a959a700 1 -- 172.21.15.44:0/3481626000 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c00bd20 con 0x7f12ac13b3b0 2020-11-20T12:07:32.371 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12a959a700 1 -- 172.21.15.44:0/3481626000 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c011610 con 0x7f12ac13b3b0 2020-11-20T12:07:32.372 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 -- 172.21.15.44:0/3481626000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac13b3b0 msgr2=0x7f12ac1380b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:32.372 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/3481626000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac13b3b0 0x7f12ac1380b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f12ac0a74a0 tx=0x7f129c0091d0).stop 2020-11-20T12:07:32.372 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 -- 172.21.15.44:0/3481626000 shutdown_connections 2020-11-20T12:07:32.372 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/3481626000 >> v2:172.21.15.79:3300/0 conn(0x7f12ac0a8b40 0x7f12ac13aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:32.372 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/3481626000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac13b3b0 0x7f12ac1380b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:32.373 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 -- 172.21.15.44:0/3481626000 >> 172.21.15.44:0/3481626000 conn(0x7f12ac0aced0 msgr2=0x7f12ac0ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:32.373 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 -- 172.21.15.44:0/3481626000 shutdown_connections 2020-11-20T12:07:32.373 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.363+0000 7f12b0cd3700 1 -- 172.21.15.44:0/3481626000 wait complete. 2020-11-20T12:07:32.373 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.364+0000 7f12b0cd3700 1 Processor -- start 2020-11-20T12:07:32.374 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.364+0000 7f12b0cd3700 1 -- start start 2020-11-20T12:07:32.377 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac0a3930 0x7f12ac0a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:32.377 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12ac0a8b40 0x7f12ac0a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:32.378 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12ac13b3b0 0x7f12ac09d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:32.378 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac09d490 con 0x7f12ac0a3930 2020-11-20T12:07:32.378 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac09e990 con 0x7f12ac0a8b40 2020-11-20T12:07:32.378 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac09ec30 con 0x7f12ac13b3b0 2020-11-20T12:07:32.378 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac0a3930 0x7f12ac0a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:32.379 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12a9d9b700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12ac0a8b40 msgr2=0x7f12ac0a7a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:32.379 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac0a3930 0x7f12ac0a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38542/0 (socket says 172.21.15.44:38542) 2020-11-20T12:07:32.379 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 -- 172.21.15.44:0/2584212033 learned_addr learned my addr 172.21.15.44:0/2584212033 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:32.379 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12a9d9b700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12ac0a8b40 0x7f12ac0a7a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:32.379 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aad9d700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12ac13b3b0 0x7f12ac09d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:32.380 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 -- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12ac13b3b0 msgr2=0x7f12ac09d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:32.380 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12ac13b3b0 0x7f12ac09d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:32.380 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 -- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12ac0a8b40 msgr2=0x7f12ac0a7a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:32.380 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12ac0a8b40 0x7f12ac0a7a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:32.380 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 -- 172.21.15.44:0/2584212033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac0a8630 con 0x7f12ac0a3930 2020-11-20T12:07:32.381 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aad9d700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12ac13b3b0 0x7f12ac09d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:32.381 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12aa59c700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac0a3930 0x7f12ac0a0670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f12a00097c0 tx=0x7f12a0006e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:32.381 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0013070 con 0x7f12ac0a3930 2020-11-20T12:07:32.381 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0006170 con 0x7f12ac0a3930 2020-11-20T12:07:32.381 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac09e4f0 con 0x7f12ac0a3930 2020-11-20T12:07:32.382 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0006510 con 0x7f12ac0a3930 2020-11-20T12:07:32.382 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.365+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac09df70 con 0x7f12ac0a3930 2020-11-20T12:07:32.382 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.366+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac09e7f0 con 0x7f12ac0a3930 2020-11-20T12:07:32.382 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.366+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f12a0003af0 con 0x7f12ac0a3930 2020-11-20T12:07:32.382 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.367+0000 7f129a7fc700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1280056cd0 0x7f1280059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:32.383 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.367+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f12a006fa80 con 0x7f12ac0a3930 2020-11-20T12:07:32.383 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.367+0000 7f12a9d9b700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1280056cd0 0x7f1280059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:32.383 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.367+0000 7f12a9d9b700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1280056cd0 0x7f1280059190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f12ac0a8110 tx=0x7f129c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:32.383 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.370+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f12ac09e7f0 con 0x7f12ac0a3930 2020-11-20T12:07:32.611 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:32.606+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f12ac09e7f0 con 0x7f12ac0a3930 2020-11-20T12:07:33.180 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:33.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.318+0000 7feb651a8700 1 Processor -- start 2020-11-20T12:07:33.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.318+0000 7feb651a8700 1 -- start start 2020-11-20T12:07:33.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb651a8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600ad440 0x7feb600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:33.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb651a8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feb600a4b00 0x7feb600a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:33.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb651a8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb600a70f0 con 0x7feb600a9bb0 2020-11-20T12:07:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb651a8700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb600a5880 con 0x7feb600a4b00 2020-11-20T12:07:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb651a8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600a4170 con 0x7feb600ad440 2020-11-20T12:07:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb649a7700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7feb600a9bb0 legacy=0x7feb60134260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:33.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feb600a4b00 0x7feb600a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:33.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feb600a4b00 0x7feb600a96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38556/0 (socket says 172.21.15.44:38556) 2020-11-20T12:07:33.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 -- 172.21.15.44:0/3886850533 learned_addr learned my addr 172.21.15.44:0/3886850533 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5ffff700 1 --2- 172.21.15.44:0/3886850533 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600ad440 0x7feb600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 -- 172.21.15.44:0/3886850533 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600ad440 msgr2=0x7feb600a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 --2- 172.21.15.44:0/3886850533 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600ad440 0x7feb600a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 -- 172.21.15.44:0/3886850533 >> v1:172.21.15.79:6789/0 conn(0x7feb600a9bb0 legacy=0x7feb60134260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:33.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 -- 172.21.15.44:0/3886850533 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600ad820 con 0x7feb600a4b00 2020-11-20T12:07:33.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5f7fe700 1 --2- 172.21.15.44:0/3886850533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600a4b00 0x7feb600a96c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7feb54006fd0 tx=0x7feb54006eb0).ready entity=mon.0 client_cookie=70c0c88cbe5bdba9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:33.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.319+0000 7feb5effd700 1 -- 172.21.15.44:0/3886850533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54009860 con 0x7feb600a4b00 2020-11-20T12:07:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb5effd700 1 -- 172.21.15.44:0/3886850533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb540099c0 con 0x7feb600a4b00 2020-11-20T12:07:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb5effd700 1 -- 172.21.15.44:0/3886850533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54009d40 con 0x7feb600a4b00 2020-11-20T12:07:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 -- 172.21.15.44:0/3886850533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600a4b00 msgr2=0x7feb600a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 --2- 172.21.15.44:0/3886850533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600a4b00 0x7feb600a96c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7feb54006fd0 tx=0x7feb54006eb0).stop 2020-11-20T12:07:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 -- 172.21.15.44:0/3886850533 shutdown_connections 2020-11-20T12:07:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 --2- 172.21.15.44:0/3886850533 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600ad440 0x7feb600a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 --2- 172.21.15.44:0/3886850533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600a4b00 0x7feb600a96c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 -- 172.21.15.44:0/3886850533 >> 172.21.15.44:0/3886850533 conn(0x7feb60061b60 msgr2=0x7feb60061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 -- 172.21.15.44:0/3886850533 shutdown_connections 2020-11-20T12:07:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.320+0000 7feb651a8700 1 -- 172.21.15.44:0/3886850533 wait complete. 2020-11-20T12:07:33.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.321+0000 7feb651a8700 1 Processor -- start 2020-11-20T12:07:33.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.321+0000 7feb651a8700 1 -- start start 2020-11-20T12:07:33.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb651a8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600a4b00 0x7feb60135e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:33.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb651a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb600a9bb0 0x7feb6012f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:33.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb651a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600ad440 0x7feb6012fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:33.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb651a8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600a3780 con 0x7feb600ad440 2020-11-20T12:07:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb651a8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60066370 con 0x7feb600a9bb0 2020-11-20T12:07:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb651a8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600abd50 con 0x7feb600a4b00 2020-11-20T12:07:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb5f7fe700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb600a9bb0 msgr2=0x7feb6012f920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600ad440 0x7feb6012fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb5f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb600a9bb0 0x7feb6012f920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600ad440 0x7feb6012fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38560/0 (socket says 172.21.15.44:38560) 2020-11-20T12:07:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 -- 172.21.15.44:0/1522391790 learned_addr learned my addr 172.21.15.44:0/1522391790 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb5ffff700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600a4b00 0x7feb60135e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 -- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600a4b00 msgr2=0x7feb60135e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600a4b00 0x7feb60135e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 -- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb600a9bb0 msgr2=0x7feb6012f920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb600a9bb0 0x7feb6012f920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 -- 172.21.15.44:0/1522391790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb60136330 con 0x7feb600ad440 2020-11-20T12:07:33.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.322+0000 7feb649a7700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600ad440 0x7feb6012fe10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7feb58006d40 tx=0x7feb58006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:33.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.323+0000 7feb3ffff700 1 -- 172.21.15.44:0/1522391790 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58003c10 con 0x7feb600ad440 2020-11-20T12:07:33.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.323+0000 7feb3ffff700 1 -- 172.21.15.44:0/1522391790 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb58003d70 con 0x7feb600ad440 2020-11-20T12:07:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.323+0000 7feb3ffff700 1 -- 172.21.15.44:0/1522391790 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5800e830 con 0x7feb600ad440 2020-11-20T12:07:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.323+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb600ac400 con 0x7feb600ad440 2020-11-20T12:07:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.323+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb60066100 con 0x7feb600ad440 2020-11-20T12:07:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.324+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb60057d80 con 0x7feb600ad440 2020-11-20T12:07:33.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.324+0000 7feb3ffff700 1 -- 172.21.15.44:0/1522391790 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7feb5800ec60 con 0x7feb600ad440 2020-11-20T12:07:33.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.324+0000 7feb3ffff700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb38056cd0 0x7feb38059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:33.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.324+0000 7feb3ffff700 1 -- 172.21.15.44:0/1522391790 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7feb58013020 con 0x7feb600ad440 2020-11-20T12:07:33.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.324+0000 7feb5ffff700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb38056cd0 0x7feb38059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:33.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.325+0000 7feb5ffff700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb38056cd0 0x7feb38059190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7feb50006fd0 tx=0x7feb50006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:33.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.328+0000 7feb3ffff700 1 -- 172.21.15.44:0/1522391790 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7feb58046f60 con 0x7feb600ad440 2020-11-20T12:07:33.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.575+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feb600a8af0 con 0x7feb600ad440 2020-11-20T12:07:33.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.575+0000 7feb3ffff700 1 -- 172.21.15.44:0/1522391790 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7feb5804a580 con 0x7feb600ad440 2020-11-20T12:07:33.582 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb38056cd0 msgr2=0x7feb38059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb38056cd0 0x7feb38059190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7feb50006fd0 tx=0x7feb50006df0).stop 2020-11-20T12:07:33.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600ad440 msgr2=0x7feb6012fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:33.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600ad440 0x7feb6012fe10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7feb58006d40 tx=0x7feb58006f90).stop 2020-11-20T12:07:33.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 shutdown_connections 2020-11-20T12:07:33.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb38056cd0 0x7feb38059190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:33.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb600ad440 0x7feb6012fe10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:33.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb600a4b00 0x7feb60135e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:33.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 --2- 172.21.15.44:0/1522391790 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb600a9bb0 0x7feb6012f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:33.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 >> 172.21.15.44:0/1522391790 conn(0x7feb60061b60 msgr2=0x7feb60061f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:33.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 shutdown_connections 2020-11-20T12:07:33.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:33.577+0000 7feb651a8700 1 -- 172.21.15.44:0/1522391790 wait complete. 2020-11-20T12:07:36.601 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:36.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.738+0000 7f28a5e99700 1 Processor -- start 2020-11-20T12:07:36.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.738+0000 7f28a5e99700 1 -- start start 2020-11-20T12:07:36.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f28a5e99700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00aa1e0 0x7f28a0065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:36.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f28a5e99700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28a00a7810 0x7f28a0066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:36.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f28a5e99700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a00a8bf0 con 0x7f28a013c170 2020-11-20T12:07:36.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f28a5e99700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a0066520 con 0x7f28a00a7810 2020-11-20T12:07:36.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f28a5e99700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a00a86b0 con 0x7f28a00aa1e0 2020-11-20T12:07:36.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f289ffff700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f28a013c170 legacy=0x7f28a013c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:36.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f289effd700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f28a00a7810 msgr2=0x7f28a0066030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:36.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f289effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28a00a7810 0x7f28a0066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:36.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00aa1e0 0x7f28a0065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00aa1e0 0x7f28a0065b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50006/0 (socket says 172.21.15.44:50006) 2020-11-20T12:07:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.739+0000 7f289f7fe700 1 -- 172.21.15.44:0/931738802 learned_addr learned my addr 172.21.15.44:0/931738802 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:36.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289f7fe700 1 -- 172.21.15.44:0/931738802 >> v2:172.21.15.79:3300/0 conn(0x7f28a00a7810 msgr2=0x7f28a0066030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:36.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289f7fe700 1 --2- 172.21.15.44:0/931738802 >> v2:172.21.15.79:3300/0 conn(0x7f28a00a7810 0x7f28a0066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:36.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289f7fe700 1 -- 172.21.15.44:0/931738802 >> v1:172.21.15.79:6789/0 conn(0x7f28a013c170 legacy=0x7f28a013c5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:36.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289f7fe700 1 -- 172.21.15.44:0/931738802 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a00a8950 con 0x7f28a00aa1e0 2020-11-20T12:07:36.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289f7fe700 1 --2- 172.21.15.44:0/931738802 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00aa1e0 0x7f28a0065b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f2894006fd0 tx=0x7f2894006eb0).ready entity=mon.2 client_cookie=d03eed7de5bb8685 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289e7fc700 1 -- 172.21.15.44:0/931738802 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894009890 con 0x7f28a00aa1e0 2020-11-20T12:07:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289e7fc700 1 -- 172.21.15.44:0/931738802 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28940099f0 con 0x7f28a00aa1e0 2020-11-20T12:07:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.740+0000 7f289e7fc700 1 -- 172.21.15.44:0/931738802 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894009d70 con 0x7f28a00aa1e0 2020-11-20T12:07:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 -- 172.21.15.44:0/931738802 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00aa1e0 msgr2=0x7f28a0065b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 --2- 172.21.15.44:0/931738802 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00aa1e0 0x7f28a0065b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f2894006fd0 tx=0x7f2894006eb0).stop 2020-11-20T12:07:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 -- 172.21.15.44:0/931738802 shutdown_connections 2020-11-20T12:07:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 --2- 172.21.15.44:0/931738802 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00aa1e0 0x7f28a0065b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 --2- 172.21.15.44:0/931738802 >> v2:172.21.15.79:3300/0 conn(0x7f28a00a7810 0x7f28a0066030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 -- 172.21.15.44:0/931738802 >> 172.21.15.44:0/931738802 conn(0x7f28a0062690 msgr2=0x7f28a00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 -- 172.21.15.44:0/931738802 shutdown_connections 2020-11-20T12:07:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.741+0000 7f28a5e99700 1 -- 172.21.15.44:0/931738802 wait complete. 2020-11-20T12:07:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.742+0000 7f28a5e99700 1 Processor -- start 2020-11-20T12:07:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.742+0000 7f28a5e99700 1 -- start start 2020-11-20T12:07:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.742+0000 7f28a5e99700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00a7810 0x7f28a0115130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.742+0000 7f28a5e99700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28a00aa1e0 0x7f28a01179a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.742+0000 7f28a5e99700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a013c170 0x7f28a00a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f28a5e99700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0115b40 con 0x7f28a013c170 2020-11-20T12:07:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f28a5e99700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0115da0 con 0x7f28a00aa1e0 2020-11-20T12:07:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f28a5e99700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0116040 con 0x7f28a00a7810 2020-11-20T12:07:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289effd700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28a00aa1e0 msgr2=0x7f28a01179a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28a00aa1e0 0x7f28a01179a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a013c170 0x7f28a00a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a013c170 0x7f28a00a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38578/0 (socket says 172.21.15.44:38578) 2020-11-20T12:07:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 -- 172.21.15.44:0/757512915 learned_addr learned my addr 172.21.15.44:0/757512915 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289f7fe700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00a7810 0x7f28a0115130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 -- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00a7810 msgr2=0x7f28a0115130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00a7810 0x7f28a0115130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 -- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28a00aa1e0 msgr2=0x7f28a01179a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28a00aa1e0 0x7f28a01179a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 -- 172.21.15.44:0/757512915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a0115620 con 0x7f28a013c170 2020-11-20T12:07:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f289ffff700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a013c170 0x7f28a00a5340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f289800bae0 tx=0x7f2898006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f287f7fe700 1 -- 172.21.15.44:0/757512915 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898003c10 con 0x7f28a013c170 2020-11-20T12:07:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f287f7fe700 1 -- 172.21.15.44:0/757512915 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2898003d70 con 0x7f28a013c170 2020-11-20T12:07:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a01164e0 con 0x7f28a013c170 2020-11-20T12:07:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f287f7fe700 1 -- 172.21.15.44:0/757512915 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898006170 con 0x7f28a013c170 2020-11-20T12:07:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.743+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a0117e90 con 0x7f28a013c170 2020-11-20T12:07:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.744+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a0061650 con 0x7f28a013c170 2020-11-20T12:07:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.745+0000 7f287f7fe700 1 -- 172.21.15.44:0/757512915 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2898006560 con 0x7f28a013c170 2020-11-20T12:07:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.745+0000 7f287f7fe700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2878056cd0 0x7f2878059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.745+0000 7f287f7fe700 1 -- 172.21.15.44:0/757512915 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f2898011020 con 0x7f28a013c170 2020-11-20T12:07:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.745+0000 7f289f7fe700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2878056cd0 0x7f2878059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.746+0000 7f289f7fe700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2878056cd0 0x7f2878059190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f289400b1b0 tx=0x7f2894015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.749+0000 7f287f7fe700 1 -- 172.21.15.44:0/757512915 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2898044da0 con 0x7f28a013c170 2020-11-20T12:07:36.781 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:07:36.784 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:07:36.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.992+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f28a01161b0 con 0x7f28a013c170 2020-11-20T12:07:36.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.993+0000 7f287f7fe700 1 -- 172.21.15.44:0/757512915 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f2898009270 con 0x7f28a013c170 2020-11-20T12:07:36.997 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2878056cd0 msgr2=0x7f2878059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2878056cd0 0x7f2878059190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f289400b1b0 tx=0x7f2894015000).stop 2020-11-20T12:07:36.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a013c170 msgr2=0x7f28a00a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a013c170 0x7f28a00a5340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f289800bae0 tx=0x7f2898006f90).stop 2020-11-20T12:07:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 shutdown_connections 2020-11-20T12:07:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2878056cd0 0x7f2878059190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:37.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28a013c170 0x7f28a00a5340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28a00a7810 0x7f28a0115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 --2- 172.21.15.44:0/757512915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28a00aa1e0 0x7f28a01179a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 >> 172.21.15.44:0/757512915 conn(0x7f28a0062690 msgr2=0x7f28a00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:37.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 shutdown_connections 2020-11-20T12:07:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:36.994+0000 7f28a5e99700 1 -- 172.21.15.44:0/757512915 wait complete. 2020-11-20T12:07:40.010 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:07:40.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.144+0000 7f1e4b6c2700 1 Processor -- start 2020-11-20T12:07:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e4b6c2700 1 -- start start 2020-11-20T12:07:40.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e4b6c2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:40.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e4b6c2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:40.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e4b6c2700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440a6260 con 0x7f1e4413ae80 2020-11-20T12:07:40.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e4b6c2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440a52e0 con 0x7f1e44060330 2020-11-20T12:07:40.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e4b6c2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440aa730 con 0x7f1e440a4c30 2020-11-20T12:07:40.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e49c5f700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f1e4413ae80 legacy=0x7f1e440aa020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:40.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.145+0000 7f1e4945e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:40.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e4945e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38582/0 (socket says 172.21.15.44:38582) 2020-11-20T12:07:40.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 learned_addr learned my addr 172.21.15.44:0/1291402756 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:40.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:40.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:40.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:40.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:40.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:40.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:07:40.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:40.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:40.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.146+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:40.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.346+0000 7f1e49c5f700 1 -- 172.21.15.44:0/1291402756 >> v1:172.21.15.79:6789/0 conn(0x7f1e4413ae80 legacy=0x7f1e440aa020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:40.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:40.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:07:40.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:40.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:40.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:40.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:07:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:40.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:40.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:40.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.347+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:07:40.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.747+0000 7f1e49c5f700 1 -- 172.21.15.44:0/1291402756 >> v1:172.21.15.79:6789/0 conn(0x7f1e4413ae80 legacy=0x7f1e440aa020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:07:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:07:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:40.748+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:07:41.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.548+0000 7f1e49c5f700 1 -- 172.21.15.44:0/1291402756 >> v1:172.21.15.79:6789/0 conn(0x7f1e4413ae80 legacy=0x7f1e440aa020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:41.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:41.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:41.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:07:41.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:41.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:41.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:07:41.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:07:41.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:07:41.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:07:41.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:41.549+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:07:43.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 msgr2=0x7f1e4413a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:43.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e440a4c30 0x7f1e4413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:43.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 msgr2=0x7f1e440a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:43.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.44:3300/0 conn(0x7f1e44060330 0x7f1e440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:43.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 -- 172.21.15.44:0/1291402756 >> v1:172.21.15.79:6789/0 conn(0x7f1e4413ae80 legacy=0x7f1e440aa020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:43.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 0x7f1e24007a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:43.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 -- 172.21.15.44:0/1291402756 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440a6490 con 0x7f1e24008050 2020-11-20T12:07:43.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e36ffd700 1 -- 172.21.15.44:0/1291402756 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440a52e0 con 0x7f1e240055f0 2020-11-20T12:07:43.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 msgr2=0x7f1e24007a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:43.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.146+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 0x7f1e24007a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:43.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.347+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 msgr2=0x7f1e24007a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:43.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.347+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 0x7f1e24007a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:07:43.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.748+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 msgr2=0x7f1e24007a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:43.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:43.748+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 0x7f1e24007a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:07:44.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.549+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 msgr2=0x7f1e24007a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:44.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.549+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 0x7f1e24007a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:07:44.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486581390 0 0) 0x7f1e440a6490 con 0x7f1e24008050 2020-11-20T12:07:44.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e2c003240 con 0x7f1e24008050 2020-11-20T12:07:44.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 146219330 0 0) 0x7f1e2c003240 con 0x7f1e24008050 2020-11-20T12:07:44.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 msgr2=0x7f1e24007a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:44.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 --2- 172.21.15.44:0/1291402756 >> v2:172.21.15.79:3300/0 conn(0x7f1e240055f0 0x7f1e24007a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:44.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440a6260 con 0x7f1e24008050 2020-11-20T12:07:44.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f1e4000d3b0 con 0x7f1e24008050 2020-11-20T12:07:44.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 >> v1:172.21.15.44:6789/0 conn(0x7f1e24008050 legacy=0x7f1e2400a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:44.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e49c5f700 1 -- 172.21.15.44:0/1291402756 reap_dead start 2020-11-20T12:07:44.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c003730 0x7f1e2c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:44.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c004040 0x7f1e2c004520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:44.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004bc0 0x7f1e2c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:44.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440a52e0 con 0x7f1e2c004040 2020-11-20T12:07:44.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c005260 con 0x7f1e2c003730 2020-11-20T12:07:44.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.734+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c00bc20 con 0x7f1e2c004bc0 2020-11-20T12:07:44.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e4945e700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c003730 msgr2=0x7f1e2c003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:44.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c003730 0x7f1e2c003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:44.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c004040 0x7f1e2c004520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:44.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e49c5f700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004bc0 0x7f1e2c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:44.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004bc0 msgr2=0x7f1e2c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:44.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004bc0 0x7f1e2c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:44.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c003730 msgr2=0x7f1e2c003b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:44.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c003730 0x7f1e2c003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:44.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e48c5d700 1 -- 172.21.15.44:0/1291402756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c003240 con 0x7f1e2c004040 2020-11-20T12:07:44.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.735+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c004040 0x7f1e2c004520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f1e40006c00 tx=0x7f1e40008040).ready entity=mon.0 client_cookie=2305dfcea13a046a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:44.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.736+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000d8d0 con 0x7f1e2c004040 2020-11-20T12:07:44.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.736+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400145c0 con 0x7f1e2c004040 2020-11-20T12:07:44.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.736+0000 7f1e37fff700 1 -- 172.21.15.44:0/1291402756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40014940 con 0x7f1e2c004040 2020-11-20T12:07:44.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1291402756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c004040 msgr2=0x7f1e2c004520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:44.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c004040 0x7f1e2c004520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f1e40006c00 tx=0x7f1e40008040).stop 2020-11-20T12:07:44.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1291402756 shutdown_connections 2020-11-20T12:07:44.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004bc0 0x7f1e2c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:44.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c004040 0x7f1e2c004520 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:44.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1291402756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c003730 0x7f1e2c003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:44.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1291402756 >> 172.21.15.44:0/1291402756 conn(0x7f1e44062690 msgr2=0x7f1e440a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:44.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1291402756 shutdown_connections 2020-11-20T12:07:44.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.737+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1291402756 wait complete. 2020-11-20T12:07:44.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.739+0000 7f1e4b6c2700 1 Processor -- start 2020-11-20T12:07:44.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.739+0000 7f1e4b6c2700 1 -- start start 2020-11-20T12:07:44.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4b6c2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c003730 0x7f1e440a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:44.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4b6c2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004040 0x7f1e440ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:44.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4b6c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c004bc0 0x7f1e4409f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:44.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4b6c2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44138200 con 0x7f1e2c003730 2020-11-20T12:07:44.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4b6c2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440a2aa0 con 0x7f1e2c004bc0 2020-11-20T12:07:44.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4b6c2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440a0920 con 0x7f1e2c004040 2020-11-20T12:07:44.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e49c5f700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c004bc0 msgr2=0x7f1e4409f4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:44.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4945e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c003730 0x7f1e440a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e49c5f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c004bc0 0x7f1e4409f4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4945e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c003730 0x7f1e440a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38624/0 (socket says 172.21.15.44:38624) 2020-11-20T12:07:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e4945e700 1 -- 172.21.15.44:0/1109946024 learned_addr learned my addr 172.21.15.44:0/1109946024 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.740+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004040 0x7f1e440ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:44.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4945e700 1 -- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004040 msgr2=0x7f1e440ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004040 0x7f1e440ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4945e700 1 -- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c004bc0 msgr2=0x7f1e4409f4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c004bc0 0x7f1e4409f4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:44.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4945e700 1 -- 172.21.15.44:0/1109946024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440a0bc0 con 0x7f1e2c003730 2020-11-20T12:07:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4945e700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c003730 0x7f1e440a02f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f1e38012c50 tx=0x7f1e38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e357fa700 1 -- 172.21.15.44:0/1109946024 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800fa60 con 0x7f1e2c003730 2020-11-20T12:07:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4409f040 con 0x7f1e2c003730 2020-11-20T12:07:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.741+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4409fbe0 con 0x7f1e2c003730 2020-11-20T12:07:44.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.742+0000 7f1e357fa700 1 -- 172.21.15.44:0/1109946024 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800fdd0 con 0x7f1e2c003730 2020-11-20T12:07:44.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.742+0000 7f1e357fa700 1 -- 172.21.15.44:0/1109946024 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38015450 con 0x7f1e2c003730 2020-11-20T12:07:44.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.743+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e44061650 con 0x7f1e2c003730 2020-11-20T12:07:44.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.743+0000 7f1e357fa700 1 -- 172.21.15.44:0/1109946024 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1e38024070 con 0x7f1e2c003730 2020-11-20T12:07:44.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.743+0000 7f1e357fa700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e1c056cd0 0x7f1e1c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:44.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.743+0000 7f1e357fa700 1 -- 172.21.15.44:0/1109946024 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f1e38070490 con 0x7f1e2c003730 2020-11-20T12:07:44.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.744+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e1c056cd0 0x7f1e1c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:44.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.744+0000 7f1e48c5d700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e1c056cd0 0x7f1e1c059190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1e40006ba0 tx=0x7f1e40008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:44.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:44.747+0000 7f1e357fa700 1 -- 172.21.15.44:0/1109946024 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1e38045720 con 0x7f1e2c003730 2020-11-20T12:07:45.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.003+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1e4409f1d0 con 0x7f1e2c003730 2020-11-20T12:07:45.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.003+0000 7f1e357fa700 1 -- 172.21.15.44:0/1109946024 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7f1e38022070 con 0x7f1e2c003730 2020-11-20T12:07:45.008 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":42,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:07:45.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e1c056cd0 msgr2=0x7f1e1c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e1c056cd0 0x7f1e1c059190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1e40006ba0 tx=0x7f1e40008040).stop 2020-11-20T12:07:45.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c003730 msgr2=0x7f1e440a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c003730 0x7f1e440a02f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f1e38012c50 tx=0x7f1e38008040).stop 2020-11-20T12:07:45.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 shutdown_connections 2020-11-20T12:07:45.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e1c056cd0 0x7f1e1c059190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e2c004bc0 0x7f1e4409f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:45.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e2c003730 0x7f1e440a02f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 --2- 172.21.15.44:0/1109946024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e2c004040 0x7f1e440ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.005+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 >> 172.21.15.44:0/1109946024 conn(0x7f1e44062690 msgr2=0x7f1e44067940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:45.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.006+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 shutdown_connections 2020-11-20T12:07:45.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.006+0000 7f1e4b6c2700 1 -- 172.21.15.44:0/1109946024 wait complete. 2020-11-20T12:07:45.020 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:07:45.021 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:07:45.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.147+0000 7fd4d9e8c700 1 Processor -- start 2020-11-20T12:07:45.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.147+0000 7fd4d9e8c700 1 -- start start 2020-11-20T12:07:45.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d9e8c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd4d40aa020 0x7fd4d40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d9e8c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40a9770 con 0x7fd4d413ac60 2020-11-20T12:07:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d9e8c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40a6da0 con 0x7fd4d40aa400 2020-11-20T12:07:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d9e8c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40a56c0 con 0x7fd4d40aa020 2020-11-20T12:07:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d2ffd700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fd4d40aa400 legacy=0x7fd4d40a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:07:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d37fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd4d40aa020 0x7fd4d40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d3fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd4d413ac60 0x7fd4d40a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42380/0 (socket says 172.21.15.44:42380) 2020-11-20T12:07:45.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d3fff700 1 -- 172.21.15.44:0/2273997073 learned_addr learned my addr 172.21.15.44:0/2273997073 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:45.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d37fe700 1 -- 172.21.15.44:0/2273997073 >> v1:172.21.15.79:6789/0 conn(0x7fd4d40aa400 legacy=0x7fd4d40a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:45.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d37fe700 1 -- 172.21.15.44:0/2273997073 >> v1:172.21.15.44:6789/0 conn(0x7fd4d413ac60 legacy=0x7fd4d40a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d27fc700 1 -- 172.21.15.44:0/2273997073 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457980002 0 0) 0x7fd4cc001450 con 0x7fd4d413ac60 2020-11-20T12:07:45.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d37fe700 1 -- 172.21.15.44:0/2273997073 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40a7040 con 0x7fd4d40aa020 2020-11-20T12:07:45.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d37fe700 1 --2- 172.21.15.44:0/2273997073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40a9280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd4c400b9a0 tx=0x7fd4c4006df0).ready entity=mon.0 client_cookie=cea5d19e963617c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d27fc700 1 -- 172.21.15.44:0/2273997073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009770 con 0x7fd4d40aa020 2020-11-20T12:07:45.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d27fc700 1 -- 172.21.15.44:0/2273997073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c40098d0 con 0x7fd4d40aa020 2020-11-20T12:07:45.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.148+0000 7fd4d27fc700 1 -- 172.21.15.44:0/2273997073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009c50 con 0x7fd4d40aa020 2020-11-20T12:07:45.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.149+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/2273997073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 msgr2=0x7fd4d40a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.149+0000 7fd4d9e8c700 1 --2- 172.21.15.44:0/2273997073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40a9280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd4c400b9a0 tx=0x7fd4c4006df0).stop 2020-11-20T12:07:45.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.149+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/2273997073 shutdown_connections 2020-11-20T12:07:45.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.149+0000 7fd4d9e8c700 1 --2- 172.21.15.44:0/2273997073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40a9280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.149+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/2273997073 >> 172.21.15.44:0/2273997073 conn(0x7fd4d4062690 msgr2=0x7fd4d40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:45.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.149+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/2273997073 shutdown_connections 2020-11-20T12:07:45.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.149+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/2273997073 wait complete. 2020-11-20T12:07:45.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.150+0000 7fd4d9e8c700 1 Processor -- start 2020-11-20T12:07:45.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.150+0000 7fd4d9e8c700 1 -- start start 2020-11-20T12:07:45.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.150+0000 7fd4d9e8c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.150+0000 7fd4d9e8c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd4d40aa400 0x7fd4d40672c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.150+0000 7fd4d9e8c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4d413ac60 0x7fd4d4146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d9e8c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40677b0 con 0x7fd4d40aa020 2020-11-20T12:07:45.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d9e8c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4067fa0 con 0x7fd4d413ac60 2020-11-20T12:07:45.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d9e8c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40ac750 con 0x7fd4d40aa400 2020-11-20T12:07:45.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38636/0 (socket says 172.21.15.44:38636) 2020-11-20T12:07:45.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 -- 172.21.15.44:0/1862442110 learned_addr learned my addr 172.21.15.44:0/1862442110 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:45.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d3fff700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4d413ac60 msgr2=0x7fd4d4146e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:45.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d3fff700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4d413ac60 0x7fd4d4146e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:45.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d2ffd700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd4d40aa400 0x7fd4d40672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd4d40aa400 msgr2=0x7fd4d40672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd4d40aa400 0x7fd4d40672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4d413ac60 msgr2=0x7fd4d4146e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd4d413ac60 0x7fd4d4146e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 -- 172.21.15.44:0/1862442110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40ac9f0 con 0x7fd4d40aa020 2020-11-20T12:07:45.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d2ffd700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd4d40aa400 0x7fd4d40672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:07:45.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40ad950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd4c4000dd0 tx=0x7fd4c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c401e460 con 0x7fd4d40aa020 2020-11-20T12:07:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c401e5c0 con 0x7fd4d40aa020 2020-11-20T12:07:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d40acc90 con 0x7fd4d40aa020 2020-11-20T12:07:45.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c401e940 con 0x7fd4d40aa020 2020-11-20T12:07:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.151+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d4147310 con 0x7fd4d40aa020 2020-11-20T12:07:45.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.152+0000 7fd4d9e8c700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d4130e90 0x7fd4d4131270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.152+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40ace50 con 0x7fd4d4130e90 2020-11-20T12:07:45.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.152+0000 7fd4d2ffd700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d4130e90 0x7fd4d4131270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.152+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd4c401a040 con 0x7fd4d40aa020 2020-11-20T12:07:45.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4d2ffd700 1 -- 172.21.15.44:0/1862442110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4d40ac560 con 0x7fd4d4130e90 2020-11-20T12:07:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4d2ffd700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d4130e90 0x7fd4d4131270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd4d4068380 tx=0x7fd4c80060e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd4ac056d00 0x7fd4ac0591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fd4c4071680 con 0x7fd4d40aa020 2020-11-20T12:07:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c800de20 con 0x7fd4d4130e90 2020-11-20T12:07:45.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fd4c800b840 con 0x7fd4d4130e90 2020-11-20T12:07:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4d3fff700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd4ac056d00 0x7fd4ac0591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d4130e90 msgr2=0x7fd4d4131270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d4130e90 0x7fd4d4131270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd4d4068380 tx=0x7fd4c80060e0).stop 2020-11-20T12:07:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d4130e90 msgr2=0x7fd4d4131270 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:45.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.153+0000 7fd4b37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d4130e90 0x7fd4d4131270 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0x7fd4d4068380 tx=0x7fd4c80060e0).stop 2020-11-20T12:07:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.154+0000 7fd4d3fff700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd4ac056d00 0x7fd4ac0591c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd4cc0048b0 tx=0x7fd4cc001570).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.161+0000 7fd4d9e8c700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d414a370 0x7fd4d414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.161+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4061650 con 0x7fd4d414a370 2020-11-20T12:07:45.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.161+0000 7fd4d2ffd700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d414a370 0x7fd4d414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.161+0000 7fd4d2ffd700 1 -- 172.21.15.44:0/1862442110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd4d40446d0 con 0x7fd4d414a370 2020-11-20T12:07:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d2ffd700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d414a370 0x7fd4d414a780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd4c8015000 tx=0x7fd4c8018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8021420 con 0x7fd4d414a370 2020-11-20T12:07:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4766 (secure 0 0 0) 0x7fd4c800b630 con 0x7fd4d414a370 2020-11-20T12:07:45.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d414a370 msgr2=0x7fd4d414a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4b37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d414a370 0x7fd4d414a780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd4c8015000 tx=0x7fd4c8018000).stop 2020-11-20T12:07:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4b37fe700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d414a370 msgr2=0x7fd4d414a780 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4b37fe700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d414a370 0x7fd4d414a780 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd4ac056d00 msgr2=0x7fd4ac0591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd4ac056d00 0x7fd4ac0591c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd4cc0048b0 tx=0x7fd4cc001570).stop 2020-11-20T12:07:45.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 msgr2=0x7fd4d40ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 --2- 172.21.15.44:0/1862442110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d40aa020 0x7fd4d40ad950 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd4c4000dd0 tx=0x7fd4c4015000).stop 2020-11-20T12:07:45.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d3fff700 1 -- 172.21.15.44:0/1862442110 reap_dead start 2020-11-20T12:07:45.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d3fff700 1 -- 172.21.15.44:0/1862442110 reap_dead start 2020-11-20T12:07:45.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 shutdown_connections 2020-11-20T12:07:45.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 >> 172.21.15.44:0/1862442110 conn(0x7fd4d4062690 msgr2=0x7fd4d40a4450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:45.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 shutdown_connections 2020-11-20T12:07:45.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.162+0000 7fd4d9e8c700 1 -- 172.21.15.44:0/1862442110 wait complete. 2020-11-20T12:07:45.189 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:45.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:45.190 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:45.190 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:07:45.190 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 42, 2020-11-20T12:07:45.191 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:45.191 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:45.191 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:45.191 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.191 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:07:45.192 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:45.192 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:45.192 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:45.192 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:45.192 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:45.193 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:45.193 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:45.193 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:45.193 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:45.193 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:45.194 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:45.194 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.194 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:45.194 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:45.194 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:45.195 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:45.195 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:45.195 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:45.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:45.196 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:45.196 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:45.196 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:45.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.197 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:45.197 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:45.197 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:45.197 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:45.197 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:45.198 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:45.198 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:45.198 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:45.198 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:45.198 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:45.199 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:45.199 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:45.199 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:45.199 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:45.199 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:45.200 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:45.200 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:45.200 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:45.200 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:45.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:45.201 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:45.201 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:45.201 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:45.201 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.202 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:45.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.202 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:45.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.202 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:45.203 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:45.203 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:45.203 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:45.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:45.204 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:45.204 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:45.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:45.205 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.205 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:45.206 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:45.206 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:45.206 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:45.206 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:45.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.207 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:45.207 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:45.207 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:45.208 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:45.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:45.208 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:45.208 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:45.209 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:45.210 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.210 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:45.211 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:45.211 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:45.211 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:45.211 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:45.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.212 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:45.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:45.212 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:45.212 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:45.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:45.213 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:45.213 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.214 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:45.214 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:45.214 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.214 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:45.215 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:45.215 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:45.216 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:45.216 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:45.216 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.216 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.217 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:45.217 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:45.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.217 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.217 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.218 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:45.218 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.218 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.218 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:07:45.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.219 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.219 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.219 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:45.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.219 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.220 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:07:45.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.220 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.220 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.220 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:45.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.221 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.221 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:45.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.221 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.222 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.222 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:07:45.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.222 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.222 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:07:45.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.223 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.223 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.223 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:45.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.224 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.224 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:45.224 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:45.226 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:07:45.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.350+0000 7fb1061ea700 1 Processor -- start 2020-11-20T12:07:45.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.350+0000 7fb1061ea700 1 -- start start 2020-11-20T12:07:45.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb1061ea700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb1000aa020 0x7fb1000a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb1061ea700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb1000aa400 0x7fb1000a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb1061ea700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb10013ac60 0x7fb1000a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb1061ea700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1000a9770 con 0x7fb1000aa400 2020-11-20T12:07:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb1061ea700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1000a6da0 con 0x7fb10013ac60 2020-11-20T12:07:45.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb1061ea700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000a7040 con 0x7fb1000aa020 2020-11-20T12:07:45.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb1000aa400 0x7fb1000a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0fffff700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7fb10013ac60 msgr2=0x7fb1000a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:45.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb1000aa400 0x7fb1000a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38648/0 (socket says 172.21.15.44:38648) 2020-11-20T12:07:45.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 -- 172.21.15.44:0/983071232 learned_addr learned my addr 172.21.15.44:0/983071232 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:45.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0fffff700 1 --2- 172.21.15.44:0/983071232 >> v2:172.21.15.79:3300/0 conn(0x7fb10013ac60 0x7fb1000a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:45.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0ff7fe700 1 --2- 172.21.15.44:0/983071232 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb1000aa020 0x7fb1000a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 -- 172.21.15.44:0/983071232 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb1000aa020 msgr2=0x7fb1000a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:07:45.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 --2- 172.21.15.44:0/983071232 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb1000aa020 0x7fb1000a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 -- 172.21.15.44:0/983071232 >> v2:172.21.15.79:3300/0 conn(0x7fb10013ac60 msgr2=0x7fb1000a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:45.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 --2- 172.21.15.44:0/983071232 >> v2:172.21.15.79:3300/0 conn(0x7fb10013ac60 0x7fb1000a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:45.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 -- 172.21.15.44:0/983071232 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000a6a90 con 0x7fb1000aa400 2020-11-20T12:07:45.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0feffd700 1 --2- 172.21.15.44:0/983071232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa400 0x7fb1000a4fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fb1000a6970 tx=0x7fb0f4006eb0).ready entity=mon.0 client_cookie=151504b4597c10f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0fe7fc700 1 -- 172.21.15.44:0/983071232 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40097d0 con 0x7fb1000aa400 2020-11-20T12:07:45.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0fe7fc700 1 -- 172.21.15.44:0/983071232 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4009930 con 0x7fb1000aa400 2020-11-20T12:07:45.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.351+0000 7fb0fe7fc700 1 -- 172.21.15.44:0/983071232 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4009cb0 con 0x7fb1000aa400 2020-11-20T12:07:45.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 -- 172.21.15.44:0/983071232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa400 msgr2=0x7fb1000a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 --2- 172.21.15.44:0/983071232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa400 0x7fb1000a4fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fb1000a6970 tx=0x7fb0f4006eb0).stop 2020-11-20T12:07:45.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 -- 172.21.15.44:0/983071232 shutdown_connections 2020-11-20T12:07:45.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 --2- 172.21.15.44:0/983071232 >> v2:172.21.15.79:3300/0 conn(0x7fb10013ac60 0x7fb1000a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 --2- 172.21.15.44:0/983071232 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb1000aa020 0x7fb1000a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 --2- 172.21.15.44:0/983071232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa400 0x7fb1000a4fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 -- 172.21.15.44:0/983071232 >> 172.21.15.44:0/983071232 conn(0x7fb100062690 msgr2=0x7fb1000a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 -- 172.21.15.44:0/983071232 shutdown_connections 2020-11-20T12:07:45.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.352+0000 7fb1061ea700 1 -- 172.21.15.44:0/983071232 wait complete. 2020-11-20T12:07:45.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.353+0000 7fb1061ea700 1 Processor -- start 2020-11-20T12:07:45.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.353+0000 7fb1061ea700 1 -- start start 2020-11-20T12:07:45.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa020 0x7fb1000abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10013ac60 0x7fb100139c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.366 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb10012bea0 0x7fb10012b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10012b510 con 0x7fb1000aa020 2020-11-20T12:07:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000a2b90 con 0x7fb10012bea0 2020-11-20T12:07:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000ac3e0 con 0x7fb10013ac60 2020-11-20T12:07:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0fffff700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb10012bea0 msgr2=0x7fb10012b020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:45.367 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa020 0x7fb1000abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0fffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb10012bea0 0x7fb10012b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa020 0x7fb1000abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38652/0 (socket says 172.21.15.44:38652) 2020-11-20T12:07:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 -- 172.21.15.44:0/2977450042 learned_addr learned my addr 172.21.15.44:0/2977450042 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0feffd700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10013ac60 0x7fb100139c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.368 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10013ac60 msgr2=0x7fb100139c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10013ac60 0x7fb100139c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb10012bea0 msgr2=0x7fb10012b020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb10012bea0 0x7fb10012b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:45.369 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 -- 172.21.15.44:0/2977450042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10012c4d0 con 0x7fb1000aa020 2020-11-20T12:07:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0ff7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa020 0x7fb1000abd40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fb0f000eba0 tx=0x7fb0f0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000c9b0 con 0x7fb1000aa020 2020-11-20T12:07:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f000cb10 con 0x7fb1000aa020 2020-11-20T12:07:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000ce90 con 0x7fb1000aa020 2020-11-20T12:07:45.370 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10012c970 con 0x7fb1000aa020 2020-11-20T12:07:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.354+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10012cc70 con 0x7fb1000aa020 2020-11-20T12:07:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.355+0000 7fb1061ea700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a420 0x7fb1000a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.355+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10012c350 con 0x7fb10012a420 2020-11-20T12:07:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.355+0000 7fb0feffd700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a420 0x7fb1000a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.371 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.355+0000 7fb0feffd700 1 -- 172.21.15.44:0/2977450042 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1000ac230 con 0x7fb10012a420 2020-11-20T12:07:45.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0feffd700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a420 0x7fb1000a1880 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fb1000a6970 tx=0x7fb0f4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb0f0012600 con 0x7fb1000aa020 2020-11-20T12:07:45.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0d8056d00 0x7fb0d80591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4011c80 con 0x7fb10012a420 2020-11-20T12:07:45.372 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fb0f0070530 con 0x7fb1000aa020 2020-11-20T12:07:45.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb0f4022070 con 0x7fb10012a420 2020-11-20T12:07:45.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0fffff700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0d8056d00 0x7fb0d80591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a420 msgr2=0x7fb1000a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a420 0x7fb1000a1880 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fb1000a6970 tx=0x7fb0f4015000).stop 2020-11-20T12:07:45.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a420 msgr2=0x7fb1000a1880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:45.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.356+0000 7fb0df7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a420 0x7fb1000a1880 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.357+0000 7fb0fffff700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0d8056d00 0x7fb0d80591c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb10012c2e0 tx=0x7fb0f8006de0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.364+0000 7fb1061ea700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a800 0x7fb1000a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:45.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.364+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb100136f20 con 0x7fb10012a800 2020-11-20T12:07:45.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.364+0000 7fb0feffd700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a800 0x7fb1000a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:45.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.365+0000 7fb0feffd700 1 -- 172.21.15.44:0/2977450042 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb100061650 con 0x7fb10012a800 2020-11-20T12:07:45.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.365+0000 7fb0feffd700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a800 0x7fb1000a1d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb0f4006ef0 tx=0x7fb0f4020070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:45.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.365+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4027680 con 0x7fb10012a800 2020-11-20T12:07:45.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4764 (secure 0 0 0) 0x7fb0f4011a70 con 0x7fb10012a800 2020-11-20T12:07:45.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a800 msgr2=0x7fb1000a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb0df7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a800 0x7fb1000a1d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb0f4006ef0 tx=0x7fb0f4020070).stop 2020-11-20T12:07:45.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb0df7fe700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a800 msgr2=0x7fb1000a1d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:07:45.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb0df7fe700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb10012a800 0x7fb1000a1d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0d8056d00 msgr2=0x7fb0d80591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0d8056d00 0x7fb0d80591c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb10012c2e0 tx=0x7fb0f8006de0).stop 2020-11-20T12:07:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa020 msgr2=0x7fb1000abd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa020 0x7fb1000abd40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fb0f000eba0 tx=0x7fb0f0006df0).stop 2020-11-20T12:07:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb0fffff700 1 -- 172.21.15.44:0/2977450042 reap_dead start 2020-11-20T12:07:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 shutdown_connections 2020-11-20T12:07:45.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 --2- 172.21.15.44:0/2977450042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1000aa020 0x7fb1000abd40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 >> 172.21.15.44:0/2977450042 conn(0x7fb100062690 msgr2=0x7fb1000a36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 shutdown_connections 2020-11-20T12:07:45.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:07:45.366+0000 7fb1061ea700 1 -- 172.21.15.44:0/2977450042 wait complete. 2020-11-20T12:07:45.380 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:07:45.381 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:45.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:45.381 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:07:45.381 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 42, 2020-11-20T12:07:45.381 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:07:45.382 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:07:45.382 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:07:45.382 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.382 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:07:45.383 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:45.383 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:07:45.383 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:07:45.383 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:45.383 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:45.384 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:45.384 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:45.384 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:45.384 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:45.384 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:45.385 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:45.385 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.385 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:07:45.385 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:07:45.385 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:45.386 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:45.386 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:45.386 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:45.386 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:45.386 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:45.387 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:45.387 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:45.387 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.387 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:07:45.388 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:07:45.388 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:07:45.388 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:07:45.388 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:07:45.389 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:07:45.389 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:45.389 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:07:45.389 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:07:45.389 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:07:45.390 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:07:45.390 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:07:45.390 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:07:45.390 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:07:45.390 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:07:45.391 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:07:45.391 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:07:45.391 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:07:45.391 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:07:45.391 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:07:45.392 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:07:45.392 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:07:45.392 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:07:45.392 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.392 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:07:45.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.393 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:07:45.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:07:45.393 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:07:45.394 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:45.394 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:45.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:45.394 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:07:45.395 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:45.395 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:07:45.396 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.396 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.396 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.396 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:07:45.397 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:07:45.397 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:45.397 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:45.397 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:45.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.398 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:07:45.398 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:07:45.398 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:45.398 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:45.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.399 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:45.399 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:07:45.399 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.400 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:45.400 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:07:45.400 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.400 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.401 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.401 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:07:45.401 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:07:45.402 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:45.402 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:45.402 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:45.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.403 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:07:45.403 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:07:45.403 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:07:45.403 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:07:45.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.404 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:07:45.404 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:07:45.404 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.405 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:07:45.405 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:07:45.405 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:07:45.405 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.406 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.406 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:07:45.406 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:07:45.407 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:07:45.407 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:07:45.407 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:07:45.407 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.407 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.408 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:07:45.408 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:07:45.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.408 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.409 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.409 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:45.409 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.409 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.410 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:07:45.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.410 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.410 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.410 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:45.411 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.411 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.411 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:07:45.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.411 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.412 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.412 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:07:45.412 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.412 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.412 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:07:45.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.413 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.413 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.413 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:07:45.413 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.414 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:07:45.414 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:07:45.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:07:45.414 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:07:45.414 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:07:45.415 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:07:45.415 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:07:45.415 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:07:45.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:07:45.415 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:07:45.416 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:07:45.417 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:07:45.744 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:45.740+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v28) v1 ==== 167+0+0 (secure 0 0 0) 0x7f12ac09e7f0 con 0x7f12ac0a3930 2020-11-20T12:07:45.919 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:45.915+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f12ac09e7f0 con 0x7f12ac0a3930 2020-11-20T12:07:46.750 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.746+0000 7f129a7fc700 1 -- 172.21.15.44:0/2584212033 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v29) v1 ==== 167+0+0 (secure 0 0 0) 0x7f12ac09e7f0 con 0x7f12ac0a3930 2020-11-20T12:07:46.754 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1280056cd0 msgr2=0x7f1280059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:46.754 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1280056cd0 0x7f1280059190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f12ac0a8110 tx=0x7f129c016000).stop 2020-11-20T12:07:46.755 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac0a3930 msgr2=0x7f12ac0a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:46.755 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac0a3930 0x7f12ac0a0670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f12a00097c0 tx=0x7f12a0006e60).stop 2020-11-20T12:07:46.756 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 shutdown_connections 2020-11-20T12:07:46.756 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1280056cd0 0x7f1280059190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:46.756 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12ac13b3b0 0x7f12ac09d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:46.757 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12ac0a3930 0x7f12ac0a0670 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:46.757 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 --2- 172.21.15.44:0/2584212033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12ac0a8b40 0x7f12ac0a7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:46.757 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 >> 172.21.15.44:0/2584212033 conn(0x7f12ac0aced0 msgr2=0x7f12ac0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:46.758 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 shutdown_connections 2020-11-20T12:07:46.758 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:46.749+0000 7f12b0cd3700 1 -- 172.21.15.44:0/2584212033 wait complete. 2020-11-20T12:07:46.758 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:07:46.765 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/foo 2020-11-20T12:07:46.775 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 'We want ENOENT, not ESTALE, here.' 2020-11-20T12:07:46.776 INFO:tasks.workunit.client.0.smithi044.stderr:++ seq 1 100 2020-11-20T12:07:46.776 INFO:tasks.workunit.client.0.smithi044.stdout:We want ENOENT, not ESTALE, here. 2020-11-20T12:07:46.777 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.777 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/1 2020-11-20T12:07:46.777 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.780 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/1': No such file or directory 2020-11-20T12:07:46.780 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.781 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/2 2020-11-20T12:07:46.781 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.784 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/2': No such file or directory 2020-11-20T12:07:46.784 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.785 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/3 2020-11-20T12:07:46.785 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.786 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/3': No such file or directory 2020-11-20T12:07:46.787 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.787 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/4 2020-11-20T12:07:46.787 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.788 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/4': No such file or directory 2020-11-20T12:07:46.789 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.789 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/5 2020-11-20T12:07:46.789 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.791 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/5': No such file or directory 2020-11-20T12:07:46.792 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.792 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/6 2020-11-20T12:07:46.792 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.794 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/6': No such file or directory 2020-11-20T12:07:46.795 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.795 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/7 2020-11-20T12:07:46.795 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.797 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/7': No such file or directory 2020-11-20T12:07:46.798 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.798 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/8 2020-11-20T12:07:46.798 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.800 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/8': No such file or directory 2020-11-20T12:07:46.801 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.801 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/9 2020-11-20T12:07:46.802 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.803 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/9': No such file or directory 2020-11-20T12:07:46.804 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.804 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/10 2020-11-20T12:07:46.805 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.806 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/10': No such file or directory 2020-11-20T12:07:46.806 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.807 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/11 2020-11-20T12:07:46.807 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.809 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/11': No such file or directory 2020-11-20T12:07:46.810 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.810 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/12 2020-11-20T12:07:46.811 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.812 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/12': No such file or directory 2020-11-20T12:07:46.813 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.813 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/13 2020-11-20T12:07:46.813 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.815 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/13': No such file or directory 2020-11-20T12:07:46.815 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.815 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/14 2020-11-20T12:07:46.816 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.818 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/14': No such file or directory 2020-11-20T12:07:46.818 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.818 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/15 2020-11-20T12:07:46.818 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.820 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/15': No such file or directory 2020-11-20T12:07:46.821 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.821 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/16 2020-11-20T12:07:46.821 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.823 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/16': No such file or directory 2020-11-20T12:07:46.824 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.824 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/17 2020-11-20T12:07:46.824 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.826 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/17': No such file or directory 2020-11-20T12:07:46.827 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.827 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/18 2020-11-20T12:07:46.827 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.829 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/18': No such file or directory 2020-11-20T12:07:46.830 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.830 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/19 2020-11-20T12:07:46.830 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.832 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/19': No such file or directory 2020-11-20T12:07:46.832 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.833 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/20 2020-11-20T12:07:46.833 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.835 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/20': No such file or directory 2020-11-20T12:07:46.835 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.835 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/21 2020-11-20T12:07:46.836 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.838 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/21': No such file or directory 2020-11-20T12:07:46.838 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.838 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/22 2020-11-20T12:07:46.838 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.841 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/22': No such file or directory 2020-11-20T12:07:46.841 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.841 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/23 2020-11-20T12:07:46.842 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.843 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/23': No such file or directory 2020-11-20T12:07:46.844 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.844 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/24 2020-11-20T12:07:46.844 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.846 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/24': No such file or directory 2020-11-20T12:07:46.847 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.847 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/25 2020-11-20T12:07:46.848 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.849 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/25': No such file or directory 2020-11-20T12:07:46.849 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.850 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/26 2020-11-20T12:07:46.850 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.851 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/26': No such file or directory 2020-11-20T12:07:46.852 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.852 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/27 2020-11-20T12:07:46.852 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.854 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/27': No such file or directory 2020-11-20T12:07:46.854 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.855 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/28 2020-11-20T12:07:46.855 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.857 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/28': No such file or directory 2020-11-20T12:07:46.858 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.858 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/29 2020-11-20T12:07:46.858 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.859 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/29': No such file or directory 2020-11-20T12:07:46.860 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.860 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/30 2020-11-20T12:07:46.860 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.862 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/30': No such file or directory 2020-11-20T12:07:46.863 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.863 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/31 2020-11-20T12:07:46.863 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.865 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/31': No such file or directory 2020-11-20T12:07:46.865 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.866 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/32 2020-11-20T12:07:46.866 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.868 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/32': No such file or directory 2020-11-20T12:07:46.868 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.869 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/33 2020-11-20T12:07:46.869 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.871 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/33': No such file or directory 2020-11-20T12:07:46.871 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.871 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/34 2020-11-20T12:07:46.872 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.873 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/34': No such file or directory 2020-11-20T12:07:46.873 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.874 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/35 2020-11-20T12:07:46.874 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.876 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/35': No such file or directory 2020-11-20T12:07:46.876 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.877 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/36 2020-11-20T12:07:46.877 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.879 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/36': No such file or directory 2020-11-20T12:07:46.879 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.879 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/37 2020-11-20T12:07:46.879 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.881 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/37': No such file or directory 2020-11-20T12:07:46.882 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.882 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/38 2020-11-20T12:07:46.882 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.884 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/38': No such file or directory 2020-11-20T12:07:46.884 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.885 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/39 2020-11-20T12:07:46.885 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.886 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/39': No such file or directory 2020-11-20T12:07:46.887 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.887 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/40 2020-11-20T12:07:46.887 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.889 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/40': No such file or directory 2020-11-20T12:07:46.890 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.890 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/41 2020-11-20T12:07:46.891 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.892 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/41': No such file or directory 2020-11-20T12:07:46.893 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.893 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/42 2020-11-20T12:07:46.893 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.895 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/42': No such file or directory 2020-11-20T12:07:46.895 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.895 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/43 2020-11-20T12:07:46.896 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.897 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/43': No such file or directory 2020-11-20T12:07:46.898 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.898 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.899 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/44 2020-11-20T12:07:46.900 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/44': No such file or directory 2020-11-20T12:07:46.901 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.901 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/45 2020-11-20T12:07:46.901 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.903 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/45': No such file or directory 2020-11-20T12:07:46.903 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.903 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/46 2020-11-20T12:07:46.904 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.906 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/46': No such file or directory 2020-11-20T12:07:46.906 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.906 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/47 2020-11-20T12:07:46.907 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.908 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/47': No such file or directory 2020-11-20T12:07:46.909 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.909 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/48 2020-11-20T12:07:46.909 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.911 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/48': No such file or directory 2020-11-20T12:07:46.912 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.912 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/49 2020-11-20T12:07:46.913 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.913 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/49': No such file or directory 2020-11-20T12:07:46.914 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.914 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/50 2020-11-20T12:07:46.915 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.916 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/50': No such file or directory 2020-11-20T12:07:46.917 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.917 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/51 2020-11-20T12:07:46.917 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.919 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/51': No such file or directory 2020-11-20T12:07:46.919 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.920 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/52 2020-11-20T12:07:46.920 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.922 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/52': No such file or directory 2020-11-20T12:07:46.922 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.922 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/53 2020-11-20T12:07:46.922 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.924 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/53': No such file or directory 2020-11-20T12:07:46.924 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.925 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/54 2020-11-20T12:07:46.925 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.926 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/54': No such file or directory 2020-11-20T12:07:46.927 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.927 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/55 2020-11-20T12:07:46.927 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.929 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/55': No such file or directory 2020-11-20T12:07:46.930 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.930 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/56 2020-11-20T12:07:46.930 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.931 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/56': No such file or directory 2020-11-20T12:07:46.932 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.932 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.933 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/57 2020-11-20T12:07:46.934 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/57': No such file or directory 2020-11-20T12:07:46.935 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.935 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/58 2020-11-20T12:07:46.935 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.937 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/58': No such file or directory 2020-11-20T12:07:46.938 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.938 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/59 2020-11-20T12:07:46.938 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.940 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/59': No such file or directory 2020-11-20T12:07:46.941 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.941 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/60 2020-11-20T12:07:46.941 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.943 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/60': No such file or directory 2020-11-20T12:07:46.943 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.943 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/61 2020-11-20T12:07:46.944 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.945 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/61': No such file or directory 2020-11-20T12:07:46.945 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.946 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/62 2020-11-20T12:07:46.946 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.947 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/62': No such file or directory 2020-11-20T12:07:46.948 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.948 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/63 2020-11-20T12:07:46.948 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.950 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/63': No such file or directory 2020-11-20T12:07:46.950 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.951 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/64 2020-11-20T12:07:46.951 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.953 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/64': No such file or directory 2020-11-20T12:07:46.954 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.954 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/65 2020-11-20T12:07:46.955 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.957 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/65': No such file or directory 2020-11-20T12:07:46.957 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.958 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/66 2020-11-20T12:07:46.958 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.959 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/66': No such file or directory 2020-11-20T12:07:46.960 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.960 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/67 2020-11-20T12:07:46.961 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.962 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/67': No such file or directory 2020-11-20T12:07:46.963 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.963 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/68 2020-11-20T12:07:46.963 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.965 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/68': No such file or directory 2020-11-20T12:07:46.966 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.966 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/69 2020-11-20T12:07:46.966 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.968 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/69': No such file or directory 2020-11-20T12:07:46.968 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.969 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/70 2020-11-20T12:07:46.969 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.971 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/70': No such file or directory 2020-11-20T12:07:46.971 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.972 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/71 2020-11-20T12:07:46.972 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.973 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/71': No such file or directory 2020-11-20T12:07:46.974 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.974 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/72 2020-11-20T12:07:46.974 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.975 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/72': No such file or directory 2020-11-20T12:07:46.976 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.976 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/73 2020-11-20T12:07:46.977 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.978 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/73': No such file or directory 2020-11-20T12:07:46.978 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.978 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/74 2020-11-20T12:07:46.979 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.980 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/74': No such file or directory 2020-11-20T12:07:46.980 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.980 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/75 2020-11-20T12:07:46.981 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.982 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/75': No such file or directory 2020-11-20T12:07:46.982 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.983 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/76 2020-11-20T12:07:46.983 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.984 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/76': No such file or directory 2020-11-20T12:07:46.985 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.985 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/77 2020-11-20T12:07:46.986 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.987 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/77': No such file or directory 2020-11-20T12:07:46.988 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.988 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/78 2020-11-20T12:07:46.988 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.990 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/78': No such file or directory 2020-11-20T12:07:46.990 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.991 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/79 2020-11-20T12:07:46.991 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.993 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/79': No such file or directory 2020-11-20T12:07:46.993 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.994 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/80 2020-11-20T12:07:46.994 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.995 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/80': No such file or directory 2020-11-20T12:07:46.996 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.996 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/81 2020-11-20T12:07:46.996 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:46.998 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/81': No such file or directory 2020-11-20T12:07:46.998 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:46.999 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/82 2020-11-20T12:07:46.999 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.000 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/82': No such file or directory 2020-11-20T12:07:47.001 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.001 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/83 2020-11-20T12:07:47.002 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.003 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/83': No such file or directory 2020-11-20T12:07:47.004 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.004 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/84 2020-11-20T12:07:47.005 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.006 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/84': No such file or directory 2020-11-20T12:07:47.007 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.007 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/85 2020-11-20T12:07:47.007 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.009 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/85': No such file or directory 2020-11-20T12:07:47.010 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.010 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/86 2020-11-20T12:07:47.011 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.012 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/86': No such file or directory 2020-11-20T12:07:47.013 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.013 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/87 2020-11-20T12:07:47.013 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.015 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/87': No such file or directory 2020-11-20T12:07:47.016 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.016 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/88 2020-11-20T12:07:47.016 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.018 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/88': No such file or directory 2020-11-20T12:07:47.018 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.019 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/89 2020-11-20T12:07:47.019 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.021 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/89': No such file or directory 2020-11-20T12:07:47.021 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.021 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/90 2020-11-20T12:07:47.021 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.022 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/90': No such file or directory 2020-11-20T12:07:47.023 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.023 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/91 2020-11-20T12:07:47.023 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.024 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/91': No such file or directory 2020-11-20T12:07:47.024 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.024 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/92 2020-11-20T12:07:47.025 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.027 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/92': No such file or directory 2020-11-20T12:07:47.027 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.027 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/93 2020-11-20T12:07:47.028 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.029 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/93': No such file or directory 2020-11-20T12:07:47.030 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.030 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/94 2020-11-20T12:07:47.030 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.032 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/94': No such file or directory 2020-11-20T12:07:47.033 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.033 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/95 2020-11-20T12:07:47.034 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.035 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/95': No such file or directory 2020-11-20T12:07:47.035 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.036 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/96 2020-11-20T12:07:47.036 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.037 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/96': No such file or directory 2020-11-20T12:07:47.037 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.038 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/97 2020-11-20T12:07:47.038 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.040 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/97': No such file or directory 2020-11-20T12:07:47.041 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.041 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/98 2020-11-20T12:07:47.041 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.043 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/98': No such file or directory 2020-11-20T12:07:47.043 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.043 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/99 2020-11-20T12:07:47.044 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.045 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/99': No such file or directory 2020-11-20T12:07:47.046 INFO:tasks.workunit.client.0.smithi044.stderr:+ for f in `seq 1 100` 2020-11-20T12:07:47.046 INFO:tasks.workunit.client.0.smithi044.stderr:+ stat .snap/foo/100 2020-11-20T12:07:47.046 INFO:tasks.workunit.client.0.smithi044.stderr:+ grep 'No such file' 2020-11-20T12:07:47.048 INFO:tasks.workunit.client.0.smithi044.stdout:stat: cannot stat '.snap/foo/100': No such file or directory 2020-11-20T12:07:47.048 INFO:tasks.workunit.client.0.smithi044.stderr:+ rmdir .snap/foo 2020-11-20T12:07:47.056 INFO:tasks.workunit.client.0.smithi044.stdout:OK 2020-11-20T12:07:47.057 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo OK 2020-11-20T12:07:47.057 INFO:teuthology.orchestra.run:Running command with timeout 3600 2020-11-20T12:07:47.058 INFO:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2020-11-20T12:07:47.092 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2020-11-20T12:07:47.093 INFO:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b86e691bf191566922a8f08e2a5abf64fde8cfcb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2020-11-20T12:07:47.156 INFO:tasks.workunit.client.0.smithi044.stderr:+ set -e 2020-11-20T12:07:47.157 INFO:tasks.workunit.client.0.smithi044.stderr:+ ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2020-11-20T12:07:47.257 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.253+0000 7f0fc4c32700 1 Processor -- start 2020-11-20T12:07:47.258 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.253+0000 7f0fc4c32700 1 -- start start 2020-11-20T12:07:47.266 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fc4c32700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0fc00a3930 0x7f0fc01380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:47.267 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fc4c32700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0fc00a3d90 0x7f0fc01385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:47.267 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fc4c32700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc0064ed0 con 0x7f0fc00a8b40 2020-11-20T12:07:47.268 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fc4c32700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc00a6320 con 0x7f0fc00a3d90 2020-11-20T12:07:47.268 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fc4c32700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc00a5be0 con 0x7f0fc00a3930 2020-11-20T12:07:47.269 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbed9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0fc00a3d90 0x7f0fc01385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:47.269 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbdd9b700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f0fc00a3930 msgr2=0x7f0fc01380b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:47.269 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbe59c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0fc00a8b40 0x7f0fc013aec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42412/0 (socket says 172.21.15.44:42412) 2020-11-20T12:07:47.270 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbe59c700 1 -- 172.21.15.44:0/129978174 learned_addr learned my addr 172.21.15.44:0/129978174 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:07:47.270 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbdd9b700 1 --2- 172.21.15.44:0/129978174 >> v2:172.21.15.79:3300/0 conn(0x7f0fc00a3930 0x7f0fc01380b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:47.270 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbed9d700 1 -- 172.21.15.44:0/129978174 >> v2:172.21.15.79:3300/0 conn(0x7f0fc00a3930 msgr2=0x7f0fc01380b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:07:47.271 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbed9d700 1 --2- 172.21.15.44:0/129978174 >> v2:172.21.15.79:3300/0 conn(0x7f0fc00a3930 0x7f0fc01380b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:47.271 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbed9d700 1 -- 172.21.15.44:0/129978174 >> v1:172.21.15.44:6789/0 conn(0x7f0fc00a8b40 legacy=0x7f0fc013aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:47.272 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbed9d700 1 -- 172.21.15.44:0/129978174 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00a5e80 con 0x7f0fc00a3d90 2020-11-20T12:07:47.272 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbd59a700 1 -- 172.21.15.44:0/129978174 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742098021 0 0) 0x7f0fc0064ed0 con 0x7f0fc00a8b40 2020-11-20T12:07:47.272 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbed9d700 1 --2- 172.21.15.44:0/129978174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc01385a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0fc005c300 tx=0x7f0fb8006df0).ready entity=mon.0 client_cookie=13ac7302eeb41db3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:47.273 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbd59a700 1 -- 172.21.15.44:0/129978174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80097b0 con 0x7f0fc00a3d90 2020-11-20T12:07:47.273 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbd59a700 1 -- 172.21.15.44:0/129978174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb8009910 con 0x7f0fc00a3d90 2020-11-20T12:07:47.274 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.254+0000 7f0fbd59a700 1 -- 172.21.15.44:0/129978174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8009c90 con 0x7f0fc00a3d90 2020-11-20T12:07:47.274 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 -- 172.21.15.44:0/129978174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 msgr2=0x7f0fc01385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:47.274 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/129978174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc01385a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0fc005c300 tx=0x7f0fb8006df0).stop 2020-11-20T12:07:47.275 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 -- 172.21.15.44:0/129978174 shutdown_connections 2020-11-20T12:07:47.275 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/129978174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc01385a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:47.276 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/129978174 >> v2:172.21.15.79:3300/0 conn(0x7f0fc00a3930 0x7f0fc01380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:47.276 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 -- 172.21.15.44:0/129978174 >> 172.21.15.44:0/129978174 conn(0x7f0fc00aced0 msgr2=0x7f0fc00ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:47.276 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 -- 172.21.15.44:0/129978174 shutdown_connections 2020-11-20T12:07:47.277 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.255+0000 7f0fc4c32700 1 -- 172.21.15.44:0/129978174 wait complete. 2020-11-20T12:07:47.278 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.256+0000 7f0fc4c32700 1 Processor -- start 2020-11-20T12:07:47.278 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.256+0000 7f0fc4c32700 1 -- start start 2020-11-20T12:07:47.278 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc00a3930 0x7f0fc009ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:47.279 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc00a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:47.279 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0fc00a8b40 0x7f0fc00a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:47.280 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00a0610 con 0x7f0fc00a3d90 2020-11-20T12:07:47.280 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00a08b0 con 0x7f0fc00a3930 2020-11-20T12:07:47.280 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00a0b50 con 0x7f0fc00a8b40 2020-11-20T12:07:47.281 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbe59c700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc00a3930 msgr2=0x7f0fc009ffe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:07:47.281 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc00a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:47.281 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbe59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc00a3930 0x7f0fc009ffe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:07:47.282 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc00a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38672/0 (socket says 172.21.15.44:38672) 2020-11-20T12:07:47.282 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 -- 172.21.15.44:0/769198300 learned_addr learned my addr 172.21.15.44:0/769198300 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:07:47.282 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbed9d700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0fc00a8b40 0x7f0fc00a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:47.283 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 -- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0fc00a8b40 msgr2=0x7f0fc00a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:47.283 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0fc00a8b40 0x7f0fc00a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:47.283 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 -- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc00a3930 msgr2=0x7f0fc009ffe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:07:47.284 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc00a3930 0x7f0fc009ffe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:47.284 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 -- 172.21.15.44:0/769198300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0145890 con 0x7f0fc00a3d90 2020-11-20T12:07:47.284 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fbdd9b700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc00a3090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f0fb000bac0 tx=0x7f0fb0009250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:47.285 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003c60 con 0x7f0fc00a3d90 2020-11-20T12:07:47.285 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb0003de0 con 0x7f0fc00a3d90 2020-11-20T12:07:47.285 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000c780 con 0x7f0fc00a3d90 2020-11-20T12:07:47.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc0145b40 con 0x7f0fc00a3d90 2020-11-20T12:07:47.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.257+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc0146080 con 0x7f0fc00a3d90 2020-11-20T12:07:47.286 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.258+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc00a0d10 con 0x7f0fc00a3d90 2020-11-20T12:07:47.287 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.258+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0fb0006170 con 0x7f0fc00a3d90 2020-11-20T12:07:47.287 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.259+0000 7f0fae7fc700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0f94056d00 0x7f0f940591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:07:47.287 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.259+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f0fb0011020 con 0x7f0fc00a3d90 2020-11-20T12:07:47.288 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.259+0000 7f0fbe59c700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0f94056d00 0x7f0f940591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:07:47.288 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.259+0000 7f0fbe59c700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0f94056d00 0x7f0f940591c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0fb4004530 tx=0x7f0fb4007880).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:07:47.288 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.262+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0fb0044bd0 con 0x7f0fc00a3d90 2020-11-20T12:07:47.500 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.496+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0fc005c590 con 0x7f0fc00a3d90 2020-11-20T12:07:47.750 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.747+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v30) v1 ==== 167+0+0 (secure 0 0 0) 0x7f0fb00481f0 con 0x7f0fc00a3d90 2020-11-20T12:07:47.933 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:47.929+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0fc00a8f20 con 0x7f0fc00a3d90 2020-11-20T12:07:48.764 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.760+0000 7f0fae7fc700 1 -- 172.21.15.44:0/769198300 <== mon.0 v2:172.21.15.44:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v31) v1 ==== 167+0+0 (secure 0 0 0) 0x7f0fb00483d0 con 0x7f0fc00a3d90 2020-11-20T12:07:48.767 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.763+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0f94056d00 msgr2=0x7f0f940591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:48.767 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.763+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0f94056d00 0x7f0f940591c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f0fb4004530 tx=0x7f0fb4007880).stop 2020-11-20T12:07:48.767 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.763+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 msgr2=0x7f0fc00a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:07:48.768 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.763+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc00a3090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f0fb000bac0 tx=0x7f0fb0009250).stop 2020-11-20T12:07:48.768 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 shutdown_connections 2020-11-20T12:07:48.768 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0f94056d00 0x7f0f940591c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:48.769 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0fc00a8b40 0x7f0fc00a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:48.769 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc00a3930 0x7f0fc009ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:07:48.769 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 --2- 172.21.15.44:0/769198300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fc00a3d90 0x7f0fc00a3090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:07:48.769 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 >> 172.21.15.44:0/769198300 conn(0x7f0fc00aced0 msgr2=0x7f0fc0068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:07:48.770 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 shutdown_connections 2020-11-20T12:07:48.770 INFO:tasks.workunit.client.0.smithi044.stderr:2020-11-20T12:07:48.764+0000 7f0fc4c32700 1 -- 172.21.15.44:0/769198300 wait complete. 2020-11-20T12:07:48.770 INFO:tasks.workunit.client.0.smithi044.stderr:enabled new snapshots 2020-11-20T12:07:48.786 INFO:tasks.workunit.client.0.smithi044.stderr:+ git clone git://git.ceph.com/ceph.git 2020-11-20T12:07:48.798 INFO:tasks.workunit.client.0.smithi044.stderr:Cloning into 'ceph'... 2020-11-20T12:08:05.418 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:08:05.419 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:08:05.419 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:08:05.420 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:08:05.427 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:08:05.428 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:08:05.428 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:08:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.565+0000 7fbc0abe2700 1 Processor -- start 2020-11-20T12:08:05.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.565+0000 7fbc0abe2700 1 -- start start 2020-11-20T12:08:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.565+0000 7fbc0abe2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbc040aa020 0x7fbc040a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.565+0000 7fbc0abe2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 0x7fbc040a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:05.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0abe2700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc040a9770 con 0x7fbc040aa400 2020-11-20T12:08:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0abe2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc040a56c0 con 0x7fbc040aa020 2020-11-20T12:08:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0abe2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc040a6da0 con 0x7fbc0413ac60 2020-11-20T12:08:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0897e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbc040aa020 0x7fbc040a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc03fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbc040aa400 0x7fbc040a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42430/0 (socket says 172.21.15.44:42430) 2020-11-20T12:08:05.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc03fff700 1 -- 172.21.15.44:0/3686867172 learned_addr learned my addr 172.21.15.44:0/3686867172 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:05.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0917f700 1 --2- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 0x7fbc040a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:05.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0917f700 1 -- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 msgr2=0x7fbc040a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:05.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0917f700 1 -- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 msgr2=0x7fbc040a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:05.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0917f700 1 --2- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 0x7fbc040a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0917f700 1 --2- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 0x7fbc040a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0897e700 1 -- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 msgr2=0x7fbc040a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0897e700 1 --2- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 0x7fbc040a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0897e700 1 -- 172.21.15.44:0/3686867172 >> v1:172.21.15.44:6789/0 conn(0x7fbc040aa400 legacy=0x7fbc040a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0897e700 1 -- 172.21.15.44:0/3686867172 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc040a7040 con 0x7fbc040aa020 2020-11-20T12:08:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc037fe700 1 -- 172.21.15.44:0/3686867172 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379912566 0 0) 0x7fbbf00010d0 con 0x7fbc040aa400 2020-11-20T12:08:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc0897e700 1 --2- 172.21.15.44:0/3686867172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa020 0x7fbc040a9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fbc040a8f70 tx=0x7fbbf8006df0).ready entity=mon.0 client_cookie=c9544b2ad4d7d8c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc037fe700 1 -- 172.21.15.44:0/3686867172 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80097b0 con 0x7fbc040aa020 2020-11-20T12:08:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc037fe700 1 -- 172.21.15.44:0/3686867172 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf8009910 con 0x7fbc040aa020 2020-11-20T12:08:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.566+0000 7fbc037fe700 1 -- 172.21.15.44:0/3686867172 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8009c90 con 0x7fbc040aa020 2020-11-20T12:08:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 -- 172.21.15.44:0/3686867172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa020 msgr2=0x7fbc040a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/3686867172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa020 0x7fbc040a9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fbc040a8f70 tx=0x7fbbf8006df0).stop 2020-11-20T12:08:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 -- 172.21.15.44:0/3686867172 shutdown_connections 2020-11-20T12:08:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/3686867172 >> v2:172.21.15.79:3300/0 conn(0x7fbc0413ac60 0x7fbc040a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/3686867172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa020 0x7fbc040a9280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 -- 172.21.15.44:0/3686867172 >> 172.21.15.44:0/3686867172 conn(0x7fbc04062690 msgr2=0x7fbc040a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:05.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 -- 172.21.15.44:0/3686867172 shutdown_connections 2020-11-20T12:08:05.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.567+0000 7fbc0abe2700 1 -- 172.21.15.44:0/3686867172 wait complete. 2020-11-20T12:08:05.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.568+0000 7fbc0abe2700 1 Processor -- start 2020-11-20T12:08:05.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.568+0000 7fbc0abe2700 1 -- start start 2020-11-20T12:08:05.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0abe2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 0x7fbc041331a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0abe2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa400 0x7fbc040ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0abe2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc0413ac60 0x7fbc041311f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:05.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0abe2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04133690 con 0x7fbc040aa400 2020-11-20T12:08:05.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0abe2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04066550 con 0x7fbc040aa020 2020-11-20T12:08:05.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0abe2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040667f0 con 0x7fbc0413ac60 2020-11-20T12:08:05.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa400 0x7fbc040ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:05.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa400 0x7fbc040ad630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38688/0 (socket says 172.21.15.44:38688) 2020-11-20T12:08:05.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 -- 172.21.15.44:0/875703327 learned_addr learned my addr 172.21.15.44:0/875703327 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0897e700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 0x7fbc041331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0917f700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc0413ac60 0x7fbc041311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0897e700 1 -- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 msgr2=0x7fbc041331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:08:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0897e700 1 -- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 msgr2=0x7fbc041331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:08:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0897e700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 0x7fbc041331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0897e700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 0x7fbc041331a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 -- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc0413ac60 msgr2=0x7fbc041311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc0413ac60 0x7fbc041311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 -- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 msgr2=0x7fbc041331a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 0x7fbc041331a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 -- 172.21.15.44:0/875703327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04066a90 con 0x7fbc040aa400 2020-11-20T12:08:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc03fff700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa400 0x7fbc040ad630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbbf000c760 tx=0x7fbbf0007920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc00ff9700 1 -- 172.21.15.44:0/875703327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00072e0 con 0x7fbc040aa400 2020-11-20T12:08:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc00ff9700 1 -- 172.21.15.44:0/875703327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf0007440 con 0x7fbc040aa400 2020-11-20T12:08:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc04133b80 con 0x7fbc040aa400 2020-11-20T12:08:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.569+0000 7fbc00ff9700 1 -- 172.21.15.44:0/875703327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0005800 con 0x7fbc040aa400 2020-11-20T12:08:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.570+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc041316e0 con 0x7fbc040aa400 2020-11-20T12:08:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.570+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc04066c20 con 0x7fbc040aa400 2020-11-20T12:08:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.571+0000 7fbc00ff9700 1 -- 172.21.15.44:0/875703327 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbbf001a060 con 0x7fbc040aa400 2020-11-20T12:08:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.571+0000 7fbc00ff9700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbdc056d20 0x7fbbdc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.571+0000 7fbc00ff9700 1 -- 172.21.15.44:0/875703327 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbbf006f770 con 0x7fbc040aa400 2020-11-20T12:08:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.571+0000 7fbc0897e700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbdc056d20 0x7fbbdc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.572+0000 7fbc0897e700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbdc056d20 0x7fbbdc0591e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fbbf800c010 tx=0x7fbbf8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.575+0000 7fbc00ff9700 1 -- 172.21.15.44:0/875703327 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbbf0044a40 con 0x7fbc040aa400 2020-11-20T12:08:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.814+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbc040a9770 con 0x7fbc040aa400 2020-11-20T12:08:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.815+0000 7fbc00ff9700 1 -- 172.21.15.44:0/875703327 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fbc040a9770 con 0x7fbc040aa400 2020-11-20T12:08:05.819 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":42,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:08:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbdc056d20 msgr2=0x7fbbdc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:05.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbdc056d20 0x7fbbdc0591e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fbbf800c010 tx=0x7fbbf8015000).stop 2020-11-20T12:08:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa400 msgr2=0x7fbc040ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa400 0x7fbc040ad630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbbf000c760 tx=0x7fbbf0007920).stop 2020-11-20T12:08:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 shutdown_connections 2020-11-20T12:08:05.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbdc056d20 0x7fbbdc0591e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc0413ac60 0x7fbc041311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc040aa020 0x7fbc041331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 --2- 172.21.15.44:0/875703327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc040aa400 0x7fbc040ad630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:05.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 >> 172.21.15.44:0/875703327 conn(0x7fbc04062690 msgr2=0x7fbc040a41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:05.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 shutdown_connections 2020-11-20T12:08:05.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:05.816+0000 7fbc0abe2700 1 -- 172.21.15.44:0/875703327 wait complete. 2020-11-20T12:08:06.850 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:08:06.853 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:08:08.831 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:08:08.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.967+0000 7fd6b1f26700 1 Processor -- start 2020-11-20T12:08:08.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.967+0000 7fd6b1f26700 1 -- start start 2020-11-20T12:08:08.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.967+0000 7fd6b1f26700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.967+0000 7fd6b1f26700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.967+0000 7fd6b1f26700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac13aea0 con 0x7fd6ac0a3450 2020-11-20T12:08:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.967+0000 7fd6b1f26700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac0a3af0 con 0x7fd6ac13eae0 2020-11-20T12:08:08.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.967+0000 7fd6b1f26700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac13e100 con 0x7fd6ac13b4a0 2020-11-20T12:08:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6abfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6abfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38694/0 (socket says 172.21.15.44:38694) 2020-11-20T12:08:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 learned_addr learned my addr 172.21.15.44:0/535687960 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:08:08.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:08.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:08.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:08.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:08.968+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:09.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:08:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:08:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:09.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:09.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.169+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:08:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:08:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:08:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:09.570+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:08:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:10.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:10.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:10.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:08:10.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:10.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:08:10.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:10.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:10.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:10.371+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:08:11.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 msgr2=0x7fd6ac0a1880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:11.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6ac13b4a0 0x7fd6ac0a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 msgr2=0x7fd6ac13a9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.44:3300/0 conn(0x7fd6ac13eae0 0x7fd6ac13a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 -- 172.21.15.44:0/535687960 >> v1:172.21.15.79:6789/0 conn(0x7fd6ac0a3450 legacy=0x7fd6ac0aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:11.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 -- 172.21.15.44:0/535687960 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac13e3a0 con 0x7fd690003390 2020-11-20T12:08:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a9ffb700 1 -- 172.21.15.44:0/535687960 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac0a3af0 con 0x7fd690007040 2020-11-20T12:08:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:08:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:11.968+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:12.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.169+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:12.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.169+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:08:12.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.169+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:12.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.169+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:12.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.169+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:08:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.570+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.570+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:08:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.570+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.570+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:12.570+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:08:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.372+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.372+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:08:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.372+0000 7fd6a3fff700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.372+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.372+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:08:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.577+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945217090 0 0) 0x7fd6ac13e3a0 con 0x7fd690003390 2020-11-20T12:08:13.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.577+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd68c003240 con 0x7fd690003390 2020-11-20T12:08:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4131128698 0 0) 0x7fd68c003240 con 0x7fd690003390 2020-11-20T12:08:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 msgr2=0x7fd6900094e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 --2- 172.21.15.44:0/535687960 >> v2:172.21.15.79:3300/0 conn(0x7fd690007040 0x7fd6900094e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69000ab90 con 0x7fd690003390 2020-11-20T12:08:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fd69c00d1c0 con 0x7fd690003390 2020-11-20T12:08:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 >> v1:172.21.15.44:6789/0 conn(0x7fd690003390 legacy=0x7fd690005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6abfff700 1 -- 172.21.15.44:0/535687960 reap_dead start 2020-11-20T12:08:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd68c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd68c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd68c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0a3af0 con 0x7fd68c003780 2020-11-20T12:08:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c0053b0 con 0x7fd68c0044e0 2020-11-20T12:08:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.578+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c00bca0 con 0x7fd68c004e50 2020-11-20T12:08:13.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd68c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6abfff700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd68c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd68c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 msgr2=0x7fd68c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:13.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd68c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 msgr2=0x7fd68c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd68c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/535687960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c003240 con 0x7fd68c003780 2020-11-20T12:08:13.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.579+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd68c003b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fd69c006c00 tx=0x7fd69c008040).ready entity=mon.0 client_cookie=63041dc4ba6dd61c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:13.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.580+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c013ea0 con 0x7fd68c003780 2020-11-20T12:08:13.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.580+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c00d700 con 0x7fd68c003780 2020-11-20T12:08:13.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.580+0000 7fd6aaffd700 1 -- 172.21.15.44:0/535687960 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c016690 con 0x7fd68c003780 2020-11-20T12:08:13.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 -- 172.21.15.44:0/535687960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 msgr2=0x7fd68c003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:13.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd68c003b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fd69c006c00 tx=0x7fd69c008040).stop 2020-11-20T12:08:13.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 -- 172.21.15.44:0/535687960 shutdown_connections 2020-11-20T12:08:13.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd68c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd68c004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/535687960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd68c003b80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 -- 172.21.15.44:0/535687960 >> 172.21.15.44:0/535687960 conn(0x7fd6ac062690 msgr2=0x7fd6ac0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:13.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 -- 172.21.15.44:0/535687960 shutdown_connections 2020-11-20T12:08:13.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.581+0000 7fd6b1f26700 1 -- 172.21.15.44:0/535687960 wait complete. 2020-11-20T12:08:13.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.582+0000 7fd6b1f26700 1 Processor -- start 2020-11-20T12:08:13.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.582+0000 7fd6b1f26700 1 -- start start 2020-11-20T12:08:13.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6b1f26700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd6ac139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6b1f26700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd6ac0a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6b1f26700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd6ac0a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6b1f26700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0a4f50 con 0x7fd68c003780 2020-11-20T12:08:13.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6b1f26700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0a70d0 con 0x7fd68c0044e0 2020-11-20T12:08:13.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6b1f26700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0a5320 con 0x7fd68c004e50 2020-11-20T12:08:13.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd6ac139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd6ac139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38730/0 (socket says 172.21.15.44:38730) 2020-11-20T12:08:13.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/2086130511 learned_addr learned my addr 172.21.15.44:0/2086130511 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:13.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6abfff700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd6ac0a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd6ac0a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6a3fff700 1 -- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 msgr2=0x7fd6ac0a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:08:13.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6a3fff700 1 -- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 msgr2=0x7fd6ac0a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:08:13.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd6ac0a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:13.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd6ac0a4570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:13.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 msgr2=0x7fd6ac0a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:13.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd6ac0a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 msgr2=0x7fd6ac0a4570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:13.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.583+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd6ac0a4570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.584+0000 7fd6ab7fe700 1 -- 172.21.15.44:0/2086130511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0669c0 con 0x7fd68c003780 2020-11-20T12:08:13.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.584+0000 7fd6ab7fe700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd6ac139ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fd68c005380 tx=0x7fd69c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:13.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.584+0000 7fd6a37fe700 1 -- 172.21.15.44:0/2086130511 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0206e0 con 0x7fd68c003780 2020-11-20T12:08:13.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.584+0000 7fd6a37fe700 1 -- 172.21.15.44:0/2086130511 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c020840 con 0x7fd68c003780 2020-11-20T12:08:13.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.584+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac066c60 con 0x7fd68c003780 2020-11-20T12:08:13.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.584+0000 7fd6a37fe700 1 -- 172.21.15.44:0/2086130511 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0209d0 con 0x7fd68c003780 2020-11-20T12:08:13.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.584+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac13a6e0 con 0x7fd68c003780 2020-11-20T12:08:13.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.585+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac061650 con 0x7fd68c003780 2020-11-20T12:08:13.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.586+0000 7fd6a37fe700 1 -- 172.21.15.44:0/2086130511 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd69c030070 con 0x7fd68c003780 2020-11-20T12:08:13.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.586+0000 7fd6a37fe700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d20 0x7fd6880591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.586+0000 7fd6a37fe700 1 -- 172.21.15.44:0/2086130511 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd69c0729c0 con 0x7fd68c003780 2020-11-20T12:08:13.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.587+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d20 0x7fd6880591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.587+0000 7fd6a3fff700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d20 0x7fd6880591e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fd6ac13a3d0 tx=0x7fd694009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.590+0000 7fd6a37fe700 1 -- 172.21.15.44:0/2086130511 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd69c04b740 con 0x7fd68c003780 2020-11-20T12:08:13.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.835+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd6ac0446d0 con 0x7fd68c003780 2020-11-20T12:08:13.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.836+0000 7fd6a37fe700 1 -- 172.21.15.44:0/2086130511 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7fd69c02e020 con 0x7fd68c003780 2020-11-20T12:08:13.841 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:08:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d20 msgr2=0x7fd6880591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d20 0x7fd6880591e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fd6ac13a3d0 tx=0x7fd694009040).stop 2020-11-20T12:08:13.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 msgr2=0x7fd6ac139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd6ac139ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fd68c005380 tx=0x7fd69c008040).stop 2020-11-20T12:08:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 shutdown_connections 2020-11-20T12:08:13.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d20 0x7fd6880591e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd68c004e50 0x7fd6ac0a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd68c003780 0x7fd6ac139ab0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 --2- 172.21.15.44:0/2086130511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd68c0044e0 0x7fd6ac0a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 >> 172.21.15.44:0/2086130511 conn(0x7fd6ac062690 msgr2=0x7fd6ac0a8770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 shutdown_connections 2020-11-20T12:08:13.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.837+0000 7fd6b1f26700 1 -- 172.21.15.44:0/2086130511 wait complete. 2020-11-20T12:08:13.852 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:08:13.853 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:08:13.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0aeddab700 1 Processor -- start 2020-11-20T12:08:13.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0aeddab700 1 -- start start 2020-11-20T12:08:13.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0aeddab700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 0x7f0ae813c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:13.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0aeddab700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae80a8bf0 con 0x7f0ae8066250 2020-11-20T12:08:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0aeddab700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae80a86b0 con 0x7f0ae80aa1e0 2020-11-20T12:08:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0aeddab700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae80a8950 con 0x7f0ae80a7810 2020-11-20T12:08:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0ae7fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0ae8066250 0x7f0ae813c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42490/0 (socket says 172.21.15.44:42490) 2020-11-20T12:08:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.978+0000 7f0ae7fff700 1 -- 172.21.15.44:0/300400843 learned_addr learned my addr 172.21.15.44:0/300400843 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae6ffd700 1 --2- 172.21.15.44:0/300400843 >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 0x7f0ae813c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468166924 0 0) 0x7f0ae80a8bf0 con 0x7f0ae8066250 2020-11-20T12:08:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad4003240 con 0x7f0ae8066250 2020-11-20T12:08:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae6ffd700 1 -- 172.21.15.44:0/300400843 >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 msgr2=0x7f0ae813c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:08:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae6ffd700 1 -- 172.21.15.44:0/300400843 >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 msgr2=0x7f0ae813c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae6ffd700 1 --2- 172.21.15.44:0/300400843 >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 0x7f0ae813c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae6ffd700 1 --2- 172.21.15.44:0/300400843 >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 0x7f0ae813c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:13.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 497214394 0 0) 0x7f0ad4003240 con 0x7f0ae8066250 2020-11-20T12:08:13.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 msgr2=0x7f0ae813c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:13.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 --2- 172.21.15.44:0/300400843 >> v2:172.21.15.79:3300/0 conn(0x7f0ae80a7810 0x7f0ae813c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:13.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 >> v1:172.21.15.79:6789/0 conn(0x7f0ae80aa1e0 legacy=0x7f0ae8065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:13.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80aa5c0 con 0x7f0ae8066250 2020-11-20T12:08:13.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f0ae0001860 con 0x7f0ae8066250 2020-11-20T12:08:13.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 >> v1:172.21.15.44:6789/0 conn(0x7f0ae8066250 legacy=0x7f0ae813c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:13.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4003730 0x7f0ad4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ad40044e0 0x7f0ad4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad4004e50 0x7f0ad400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80a8950 con 0x7f0ad4004e50 2020-11-20T12:08:14.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad4004280 con 0x7f0ad4003730 2020-11-20T12:08:14.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad400bc20 con 0x7f0ad40044e0 2020-11-20T12:08:14.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae7fff700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad4004e50 0x7f0ad400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4003730 0x7f0ad4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.979+0000 7f0ae6ffd700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ad40044e0 0x7f0ad4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae7fff700 1 -- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ad40044e0 msgr2=0x7f0ad4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:14.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae7fff700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ad40044e0 0x7f0ad4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae7fff700 1 -- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4003730 msgr2=0x7f0ad4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:14.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae7fff700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4003730 0x7f0ad4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae7fff700 1 -- 172.21.15.44:0/300400843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad4003240 con 0x7f0ad4004e50 2020-11-20T12:08:14.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae7fff700 1 -- 172.21.15.44:0/300400843 reap_dead start 2020-11-20T12:08:14.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae7fff700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad4004e50 0x7f0ad400b730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0ae80656e0 tx=0x7f0ae0007b50).ready entity=mon.0 client_cookie=88b974b16c54e799 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0005b80 con 0x7f0ad4004e50 2020-11-20T12:08:14.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0005ce0 con 0x7f0ad4004e50 2020-11-20T12:08:14.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0ae67fc700 1 -- 172.21.15.44:0/300400843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00072e0 con 0x7f0ad4004e50 2020-11-20T12:08:14.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.980+0000 7f0aeddab700 1 -- 172.21.15.44:0/300400843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad4004e50 msgr2=0x7f0ad400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.981+0000 7f0aeddab700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad4004e50 0x7f0ad400b730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0ae80656e0 tx=0x7f0ae0007b50).stop 2020-11-20T12:08:14.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.981+0000 7f0aeddab700 1 -- 172.21.15.44:0/300400843 shutdown_connections 2020-11-20T12:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.981+0000 7f0aeddab700 1 --2- 172.21.15.44:0/300400843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ad4004e50 0x7f0ad400b730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.981+0000 7f0aeddab700 1 -- 172.21.15.44:0/300400843 >> 172.21.15.44:0/300400843 conn(0x7f0ae8062690 msgr2=0x7f0ae80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.981+0000 7f0aeddab700 1 -- 172.21.15.44:0/300400843 shutdown_connections 2020-11-20T12:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.981+0000 7f0aeddab700 1 -- 172.21.15.44:0/300400843 wait complete. 2020-11-20T12:08:14.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 Processor -- start 2020-11-20T12:08:14.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 -- start start 2020-11-20T12:08:14.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 0x7f0ae809f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae8147cc0 0x7f0ae813f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ae8137cd0 0x7f0ae80a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80a0b50 con 0x7f0ae8147cc0 2020-11-20T12:08:14.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae809f000 con 0x7f0ad4004e50 2020-11-20T12:08:14.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0aeddab700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae809f2a0 con 0x7f0ae8137cd0 2020-11-20T12:08:14.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0ae6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae8147cc0 0x7f0ae813f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.982+0000 7f0ae6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae8147cc0 0x7f0ae813f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38754/0 (socket says 172.21.15.44:38754) 2020-11-20T12:08:14.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae6ffd700 1 -- 172.21.15.44:0/2639349022 learned_addr learned my addr 172.21.15.44:0/2639349022 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:14.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 0x7f0ae809f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae7fff700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ae8137cd0 0x7f0ae80a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae77fe700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 msgr2=0x7f0ae809f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:08:14.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae77fe700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 msgr2=0x7f0ae809f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:08:14.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 0x7f0ae809f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:14.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 0x7f0ae809f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:14.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae6ffd700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ae8137cd0 msgr2=0x7f0ae80a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae6ffd700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ae8137cd0 0x7f0ae80a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae6ffd700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 msgr2=0x7f0ae809f740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:14.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae6ffd700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ad4004e50 0x7f0ae809f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae6ffd700 1 -- 172.21.15.44:0/2639349022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8148b90 con 0x7f0ae8147cc0 2020-11-20T12:08:14.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ae6ffd700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae8147cc0 0x7f0ae813f270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0adc00fa30 tx=0x7f0adc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00e8a0 con 0x7f0ae8147cc0 2020-11-20T12:08:14.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc00ea00 con 0x7f0ae8147cc0 2020-11-20T12:08:14.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00ed80 con 0x7f0ae8147cc0 2020-11-20T12:08:14.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae80a27d0 con 0x7f0ae8147cc0 2020-11-20T12:08:14.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.983+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae813f760 con 0x7f0ae8147cc0 2020-11-20T12:08:14.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.984+0000 7f0aeddab700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae80a0610 0x7f0ae81380b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.984+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80a09f0 con 0x7f0ae80a0610 2020-11-20T12:08:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.984+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae80a0610 0x7f0ae81380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.984+0000 7f0ae77fe700 1 -- 172.21.15.44:0/2639349022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ae809f460 con 0x7f0ae80a0610 2020-11-20T12:08:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.984+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0adc013670 con 0x7f0ae8147cc0 2020-11-20T12:08:14.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae80a0610 0x7f0ae81380b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0ad8007b60 tx=0x7f0ad8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ac0056d20 0x7f0ac00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0adc0703a0 con 0x7f0ae8147cc0 2020-11-20T12:08:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800f930 con 0x7f0ae80a0610 2020-11-20T12:08:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ae7fff700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ac0056d20 0x7f0ac00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f0ad800fa90 con 0x7f0ae80a0610 2020-11-20T12:08:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae80a0610 msgr2=0x7f0ae81380b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae80a0610 0x7f0ae81380b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0ad8007b60 tx=0x7f0ad8009040).stop 2020-11-20T12:08:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae80a0610 msgr2=0x7f0ae81380b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ac77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae80a0610 0x7f0ae81380b0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0x7f0ad8007b60 tx=0x7f0ad8009040).stop 2020-11-20T12:08:14.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.985+0000 7f0ae7fff700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ac0056d20 0x7f0ac00591e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f0ad4005420 tx=0x7f0ae0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.993+0000 7f0aeddab700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae814b290 0x7f0ae814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.993+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae8061650 con 0x7f0ae814b290 2020-11-20T12:08:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.993+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae814b290 0x7f0ae814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.993+0000 7f0ae77fe700 1 -- 172.21.15.44:0/2639349022 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0ae80446d0 con 0x7f0ae814b290 2020-11-20T12:08:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.993+0000 7f0ae77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae814b290 0x7f0ae814b6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0ad8007b60 tx=0x7f0ad801b000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad801d590 con 0x7f0ae814b290 2020-11-20T12:08:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f0ad800f720 con 0x7f0ae814b290 2020-11-20T12:08:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae814b290 msgr2=0x7f0ae814b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ac77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae814b290 0x7f0ae814b6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0ad8007b60 tx=0x7f0ad801b000).stop 2020-11-20T12:08:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ac77fe700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae814b290 msgr2=0x7f0ae814b6a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:14.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ac77fe700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae814b290 0x7f0ae814b6a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ac0056d20 msgr2=0x7f0ac00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ac0056d20 0x7f0ac00591e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f0ad4005420 tx=0x7f0ae0015000).stop 2020-11-20T12:08:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae8147cc0 msgr2=0x7f0ae813f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 --2- 172.21.15.44:0/2639349022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae8147cc0 0x7f0ae813f270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0adc00fa30 tx=0x7f0adc008040).stop 2020-11-20T12:08:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ae7fff700 1 -- 172.21.15.44:0/2639349022 reap_dead start 2020-11-20T12:08:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0ae7fff700 1 -- 172.21.15.44:0/2639349022 reap_dead start 2020-11-20T12:08:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 shutdown_connections 2020-11-20T12:08:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 >> 172.21.15.44:0/2639349022 conn(0x7f0ae8062690 msgr2=0x7f0ae8065a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 shutdown_connections 2020-11-20T12:08:14.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:13.994+0000 7f0aeddab700 1 -- 172.21.15.44:0/2639349022 wait complete. 2020-11-20T12:08:14.031 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:14.031 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:08:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 50, 2020-11-20T12:08:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:14.032 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:08:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:08:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:14.033 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.034 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.035 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.036 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.037 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.038 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:14.039 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:14.040 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:14.041 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.042 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.043 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:14.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:14.045 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:14.046 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:14.047 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.048 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.048 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:14.048 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.052 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.053 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:14.054 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:14.055 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:14.056 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:14.057 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.058 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.059 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:08:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:08:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.062 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:14.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:08:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.064 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:08:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.065 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.066 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:14.066 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:14.067 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:08:14.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5fe117700 1 Processor -- start 2020-11-20T12:08:14.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5fe117700 1 -- start start 2020-11-20T12:08:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5fe117700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd5f80abc30 0x7fd5f80666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5fe117700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd5f80ab620 0x7fd5f8137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5fe117700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f8067350 con 0x7fd5f8066bc0 2020-11-20T12:08:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5fe117700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80a9b40 con 0x7fd5f80ab620 2020-11-20T12:08:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5fe117700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80ab040 con 0x7fd5f80abc30 2020-11-20T12:08:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5f6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd5f80ab620 0x7fd5f8137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5f6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd5f80ab620 0x7fd5f8137bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38762/0 (socket says 172.21.15.44:38762) 2020-11-20T12:08:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.191+0000 7fd5f6ffd700 1 -- 172.21.15.44:0/3655273786 learned_addr learned my addr 172.21.15.44:0/3655273786 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:14.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f77fe700 1 --2- 172.21.15.44:0/3655273786 >> v2:172.21.15.79:3300/0 conn(0x7fd5f80abc30 0x7fd5f80666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f6ffd700 1 -- 172.21.15.44:0/3655273786 >> v2:172.21.15.79:3300/0 conn(0x7fd5f80abc30 msgr2=0x7fd5f80666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:14.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f6ffd700 1 --2- 172.21.15.44:0/3655273786 >> v2:172.21.15.79:3300/0 conn(0x7fd5f80abc30 0x7fd5f80666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f6ffd700 1 -- 172.21.15.44:0/3655273786 >> v1:172.21.15.79:6789/0 conn(0x7fd5f8066bc0 legacy=0x7fd5f81380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f6ffd700 1 -- 172.21.15.44:0/3655273786 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8067880 con 0x7fd5f80ab620 2020-11-20T12:08:14.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f6ffd700 1 --2- 172.21.15.44:0/3655273786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80ab620 0x7fd5f8137bc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd5f8064350 tx=0x7fd5ec006eb0).ready entity=mon.0 client_cookie=34fd67abfd4db55f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f67fc700 1 -- 172.21.15.44:0/3655273786 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec009810 con 0x7fd5f80ab620 2020-11-20T12:08:14.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f67fc700 1 -- 172.21.15.44:0/3655273786 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec009970 con 0x7fd5f80ab620 2020-11-20T12:08:14.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5f67fc700 1 -- 172.21.15.44:0/3655273786 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec009cf0 con 0x7fd5f80ab620 2020-11-20T12:08:14.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5fe117700 1 -- 172.21.15.44:0/3655273786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80ab620 msgr2=0x7fd5f8137bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.192+0000 7fd5fe117700 1 --2- 172.21.15.44:0/3655273786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80ab620 0x7fd5f8137bc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd5f8064350 tx=0x7fd5ec006eb0).stop 2020-11-20T12:08:14.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.193+0000 7fd5fe117700 1 -- 172.21.15.44:0/3655273786 shutdown_connections 2020-11-20T12:08:14.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.193+0000 7fd5fe117700 1 --2- 172.21.15.44:0/3655273786 >> v2:172.21.15.79:3300/0 conn(0x7fd5f80abc30 0x7fd5f80666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.193+0000 7fd5fe117700 1 --2- 172.21.15.44:0/3655273786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80ab620 0x7fd5f8137bc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.193+0000 7fd5fe117700 1 -- 172.21.15.44:0/3655273786 >> 172.21.15.44:0/3655273786 conn(0x7fd5f8065290 msgr2=0x7fd5f8060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.193+0000 7fd5fe117700 1 -- 172.21.15.44:0/3655273786 shutdown_connections 2020-11-20T12:08:14.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.193+0000 7fd5fe117700 1 -- 172.21.15.44:0/3655273786 wait complete. 2020-11-20T12:08:14.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 Processor -- start 2020-11-20T12:08:14.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 -- start start 2020-11-20T12:08:14.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f8066bc0 0x7fd5f81368d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd5f80ab620 0x7fd5f8131c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80abc30 0x7fd5f8132170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80634b0 con 0x7fd5f80abc30 2020-11-20T12:08:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80aebe0 con 0x7fd5f8066bc0 2020-11-20T12:08:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5fe117700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f813cc70 con 0x7fd5f80ab620 2020-11-20T12:08:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5f7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80abc30 0x7fd5f8132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5f7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80abc30 0x7fd5f8132170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38770/0 (socket says 172.21.15.44:38770) 2020-11-20T12:08:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5f7fff700 1 -- 172.21.15.44:0/484334265 learned_addr learned my addr 172.21.15.44:0/484334265 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5f77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f8066bc0 0x7fd5f81368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.209 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.194+0000 7fd5f6ffd700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd5f80ab620 0x7fd5f8131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5f7fff700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd5f80ab620 msgr2=0x7fd5f8131c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5f7fff700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd5f80ab620 0x7fd5f8131c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5f7fff700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f8066bc0 msgr2=0x7fd5f81368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5f7fff700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f8066bc0 0x7fd5f81368d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5f7fff700 1 -- 172.21.15.44:0/484334265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f813bf90 con 0x7fd5f80abc30 2020-11-20T12:08:14.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5f77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f8066bc0 0x7fd5f81368d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:08:14.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5f7fff700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80abc30 0x7fd5f8132170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fd5f000cac0 tx=0x7fd5f0007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.212 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000cea0 con 0x7fd5f80abc30 2020-11-20T12:08:14.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f000a730 con 0x7fd5f80abc30 2020-11-20T12:08:14.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f813c730 con 0x7fd5f80abc30 2020-11-20T12:08:14.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000aab0 con 0x7fd5f80abc30 2020-11-20T12:08:14.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.195+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f80618c0 con 0x7fd5f80abc30 2020-11-20T12:08:14.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.196+0000 7fd5fe117700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80643c0 0x7fd5f8061dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.196+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f813c8c0 con 0x7fd5f80643c0 2020-11-20T12:08:14.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.196+0000 7fd5f77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80643c0 0x7fd5f8061dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.196+0000 7fd5f77fe700 1 -- 172.21.15.44:0/484334265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5f813ca80 con 0x7fd5f80643c0 2020-11-20T12:08:14.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.196+0000 7fd5f77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80643c0 0x7fd5f8061dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd5e800f470 tx=0x7fd5e8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8011ec0 con 0x7fd5f80643c0 2020-11-20T12:08:14.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd5f0010070 con 0x7fd5f80abc30 2020-11-20T12:08:14.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5d0056d60 0x7fd5d0059220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd5f006fd60 con 0x7fd5f80abc30 2020-11-20T12:08:14.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fd5e80135b0 con 0x7fd5f80643c0 2020-11-20T12:08:14.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80643c0 msgr2=0x7fd5f8061dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80643c0 0x7fd5f8061dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd5e800f470 tx=0x7fd5e8006df0).stop 2020-11-20T12:08:14.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5f6ffd700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5d0056d60 0x7fd5d0059220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80643c0 msgr2=0x7fd5f8061dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:14.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5d77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80643c0 0x7fd5f8061dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.197+0000 7fd5f6ffd700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5d0056d60 0x7fd5d0059220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fd5f8064350 tx=0x7fd5ec015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.205+0000 7fd5fe117700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80622b0 0x7fd5f80626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.205+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f805c2a0 con 0x7fd5f80622b0 2020-11-20T12:08:14.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.205+0000 7fd5f77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80622b0 0x7fd5f80626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5f77fe700 1 -- 172.21.15.44:0/484334265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd5f80aedc0 con 0x7fd5f80622b0 2020-11-20T12:08:14.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5f77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80622b0 0x7fd5f80626c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd5e8018000 tx=0x7fd5e801b000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8014e30 con 0x7fd5f80622b0 2020-11-20T12:08:14.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4265 (secure 0 0 0) 0x7fd5e80133a0 con 0x7fd5f80622b0 2020-11-20T12:08:14.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80622b0 msgr2=0x7fd5f80626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5d77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80622b0 0x7fd5f80626c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd5e8018000 tx=0x7fd5e801b000).stop 2020-11-20T12:08:14.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5d77fe700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80622b0 msgr2=0x7fd5f80626c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:14.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.206+0000 7fd5d77fe700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5f80622b0 0x7fd5f80626c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5d0056d60 msgr2=0x7fd5d0059220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5d0056d60 0x7fd5d0059220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fd5f8064350 tx=0x7fd5ec015000).stop 2020-11-20T12:08:14.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80abc30 msgr2=0x7fd5f8132170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 --2- 172.21.15.44:0/484334265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5f80abc30 0x7fd5f8132170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fd5f000cac0 tx=0x7fd5f0007f10).stop 2020-11-20T12:08:14.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5f7fff700 1 -- 172.21.15.44:0/484334265 reap_dead start 2020-11-20T12:08:14.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5f7fff700 1 -- 172.21.15.44:0/484334265 reap_dead start 2020-11-20T12:08:14.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 shutdown_connections 2020-11-20T12:08:14.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 >> 172.21.15.44:0/484334265 conn(0x7fd5f8065290 msgr2=0x7fd5f8137990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 shutdown_connections 2020-11-20T12:08:14.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.207+0000 7fd5fe117700 1 -- 172.21.15.44:0/484334265 wait complete. 2020-11-20T12:08:14.229 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:14.229 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:08:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 50, 2020-11-20T12:08:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:14.230 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:08:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:08:14.231 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.232 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.233 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:14.234 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.235 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.235 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.235 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.235 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:14.237 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.241 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.242 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:14.246 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:14.247 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.249 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.251 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.252 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:14.254 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:14.255 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.256 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.257 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.258 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.259 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.260 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:14.260 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:14.261 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:08:14.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.384+0000 7f069f9cb700 1 Processor -- start 2020-11-20T12:08:14.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069f9cb700 1 -- start start 2020-11-20T12:08:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069f9cb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0698060330 0x7f06980a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069f9cb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f069813b0a0 0x7f0698141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069f9cb700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06980a6260 con 0x7f06980a4c30 2020-11-20T12:08:14.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069f9cb700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06980a52e0 con 0x7f069813b0a0 2020-11-20T12:08:14.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069f9cb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06980a6490 con 0x7f0698060330 2020-11-20T12:08:14.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069df68700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f069813b0a0 0x7f0698141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069cf66700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f06980a4c30 0x7f069813a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42528/0 (socket says 172.21.15.44:42528) 2020-11-20T12:08:14.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.385+0000 7f069cf66700 1 -- 172.21.15.44:0/1608791840 learned_addr learned my addr 172.21.15.44:0/1608791840 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f069d767700 1 --2- 172.21.15.44:0/1608791840 >> v2:172.21.15.79:3300/0 conn(0x7f0698060330 0x7f06980a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f069df68700 1 -- 172.21.15.44:0/1608791840 >> v2:172.21.15.79:3300/0 conn(0x7f0698060330 msgr2=0x7f06980a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f069df68700 1 --2- 172.21.15.44:0/1608791840 >> v2:172.21.15.79:3300/0 conn(0x7f0698060330 0x7f06980a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f069df68700 1 -- 172.21.15.44:0/1608791840 >> v1:172.21.15.44:6789/0 conn(0x7f06980a4c30 legacy=0x7f069813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f0687fff700 1 -- 172.21.15.44:0/1608791840 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933874179 0 0) 0x7f06980a6260 con 0x7f06980a4c30 2020-11-20T12:08:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f069df68700 1 -- 172.21.15.44:0/1608791840 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980a6730 con 0x7f069813b0a0 2020-11-20T12:08:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f069df68700 1 --2- 172.21.15.44:0/1608791840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069813b0a0 0x7f0698141810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f06980adec0 tx=0x7f0690006df0).ready entity=mon.0 client_cookie=eefbbdbfe6e3291 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f0687fff700 1 -- 172.21.15.44:0/1608791840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900097b0 con 0x7f069813b0a0 2020-11-20T12:08:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f0687fff700 1 -- 172.21.15.44:0/1608791840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690009910 con 0x7f069813b0a0 2020-11-20T12:08:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.386+0000 7f0687fff700 1 -- 172.21.15.44:0/1608791840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690009c90 con 0x7f069813b0a0 2020-11-20T12:08:14.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 -- 172.21.15.44:0/1608791840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069813b0a0 msgr2=0x7f0698141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 --2- 172.21.15.44:0/1608791840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069813b0a0 0x7f0698141810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f06980adec0 tx=0x7f0690006df0).stop 2020-11-20T12:08:14.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 -- 172.21.15.44:0/1608791840 shutdown_connections 2020-11-20T12:08:14.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 --2- 172.21.15.44:0/1608791840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069813b0a0 0x7f0698141810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 --2- 172.21.15.44:0/1608791840 >> v2:172.21.15.79:3300/0 conn(0x7f0698060330 0x7f06980a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 -- 172.21.15.44:0/1608791840 >> 172.21.15.44:0/1608791840 conn(0x7f0698062690 msgr2=0x7f06980a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 -- 172.21.15.44:0/1608791840 shutdown_connections 2020-11-20T12:08:14.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.387+0000 7f069f9cb700 1 -- 172.21.15.44:0/1608791840 wait complete. 2020-11-20T12:08:14.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 Processor -- start 2020-11-20T12:08:14.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 -- start start 2020-11-20T12:08:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0698060330 0x7f06980acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06980a4c30 0x7f0698146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069813b0a0 0x7f069814a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698147670 con 0x7f06980a4c30 2020-11-20T12:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698149df0 con 0x7f069813b0a0 2020-11-20T12:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.388+0000 7f069f9cb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069814a090 con 0x7f0698060330 2020-11-20T12:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06980a4c30 0x7f0698146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06980a4c30 0x7f0698146d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38786/0 (socket says 172.21.15.44:38786) 2020-11-20T12:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 -- 172.21.15.44:0/2258910401 learned_addr learned my addr 172.21.15.44:0/2258910401 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069d767700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0698060330 0x7f06980acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069df68700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069813b0a0 0x7f069814a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0698060330 msgr2=0x7f06980acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0698060330 0x7f06980acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069813b0a0 msgr2=0x7f069814a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069813b0a0 0x7f069814a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 -- 172.21.15.44:0/2258910401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06981360c0 con 0x7f06980a4c30 2020-11-20T12:08:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069cf66700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06980a4c30 0x7f0698146d60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f069400cae0 tx=0x7f0694007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694005b80 con 0x7f06980a4c30 2020-11-20T12:08:14.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694005ce0 con 0x7f06980a4c30 2020-11-20T12:08:14.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06980ac8c0 con 0x7f06980a4c30 2020-11-20T12:08:14.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694007320 con 0x7f06980a4c30 2020-11-20T12:08:14.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.389+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069814aa20 con 0x7f06980a4c30 2020-11-20T12:08:14.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.390+0000 7f069f9cb700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06981307a0 0x7f06980ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.390+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069814ac70 con 0x7f06981307a0 2020-11-20T12:08:14.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.390+0000 7f069d767700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06981307a0 0x7f06980ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.390+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0694010030 con 0x7f06980a4c30 2020-11-20T12:08:14.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f069d767700 1 -- 172.21.15.44:0/2258910401 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f069814a250 con 0x7f06981307a0 2020-11-20T12:08:14.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f069d767700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06981307a0 0x7f06980ab660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0688011be0 tx=0x7f0688006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0670056d20 0x7f06700591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0694070120 con 0x7f06980a4c30 2020-11-20T12:08:14.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f069df68700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0670056d20 0x7f06700591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068800fc30 con 0x7f06981307a0 2020-11-20T12:08:14.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f068800fd90 con 0x7f06981307a0 2020-11-20T12:08:14.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06981307a0 msgr2=0x7f06980ab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06981307a0 0x7f06980ab660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0688011be0 tx=0x7f0688006df0).stop 2020-11-20T12:08:14.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06981307a0 msgr2=0x7f06980ab660 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:14.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f06857fa700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06981307a0 0x7f06980ab660 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.391+0000 7f069df68700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0670056d20 0x7f06700591e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f0690006e30 tx=0x7f0690015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.399+0000 7f069f9cb700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06980abb50 0x7f06980abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.399+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0698147490 con 0x7f06980abb50 2020-11-20T12:08:14.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.399+0000 7f069d767700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06980abb50 0x7f06980abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f069d767700 1 -- 172.21.15.44:0/2258910401 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0698061650 con 0x7f06980abb50 2020-11-20T12:08:14.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f069d767700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06980abb50 0x7f06980abf60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f0688011be0 tx=0x7f068801b000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688006290 con 0x7f06980abb50 2020-11-20T12:08:14.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4775 (secure 0 0 0) 0x7f0688027070 con 0x7f06980abb50 2020-11-20T12:08:14.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06980abb50 msgr2=0x7f06980abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f06857fa700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06980abb50 0x7f06980abf60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f0688011be0 tx=0x7f068801b000).stop 2020-11-20T12:08:14.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f06857fa700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06980abb50 msgr2=0x7f06980abf60 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:14.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.400+0000 7f06857fa700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f06980abb50 0x7f06980abf60 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0x7f0688011be0 tx=0x7f068801b000).stop 2020-11-20T12:08:14.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0670056d20 msgr2=0x7f06700591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0670056d20 0x7f06700591e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f0690006e30 tx=0x7f0690015000).stop 2020-11-20T12:08:14.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06980a4c30 msgr2=0x7f0698146d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 --2- 172.21.15.44:0/2258910401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06980a4c30 0x7f0698146d60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f069400cae0 tx=0x7f0694007e40).stop 2020-11-20T12:08:14.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069df68700 1 -- 172.21.15.44:0/2258910401 reap_dead start 2020-11-20T12:08:14.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069df68700 1 -- 172.21.15.44:0/2258910401 reap_dead start 2020-11-20T12:08:14.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 shutdown_connections 2020-11-20T12:08:14.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 >> 172.21.15.44:0/2258910401 conn(0x7f0698062690 msgr2=0x7f06980a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 shutdown_connections 2020-11-20T12:08:14.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.401+0000 7f069f9cb700 1 -- 172.21.15.44:0/2258910401 wait complete. 2020-11-20T12:08:14.416 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:14.416 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:08:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 50, 2020-11-20T12:08:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:14.417 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:08:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:08:14.418 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.419 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.420 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.425 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:14.426 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.427 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:14.429 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:14.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:14.431 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:14.432 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:14.433 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.434 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:14.435 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:14.437 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:14.438 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:14.441 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:14.442 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.443 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:14.445 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:14.447 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:14.448 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:14.449 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:14.450 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.451 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:08:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.452 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.453 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:08:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:08:14.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.455 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:08:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:14.456 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:08:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:14.457 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:14.458 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:14.458 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:14.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:14.458 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:14.459 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:14.460 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:08:14.461 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:08:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.583+0000 7f944cd64700 1 Processor -- start 2020-11-20T12:08:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.583+0000 7f944cd64700 1 -- start start 2020-11-20T12:08:14.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f944cd64700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f94480aa020 0x7f94480a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f944cd64700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94480a9770 con 0x7f94480aa400 2020-11-20T12:08:14.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f944cd64700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94480a6da0 con 0x7f944813ac60 2020-11-20T12:08:14.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f944cd64700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94480a56c0 con 0x7f94480aa020 2020-11-20T12:08:14.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f9445d9b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f94480aa400 0x7f94480a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42544/0 (socket says 172.21.15.44:42544) 2020-11-20T12:08:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f9445d9b700 1 -- 172.21.15.44:0/320326738 learned_addr learned my addr 172.21.15.44:0/320326738 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f944659c700 1 --2- 172.21.15.44:0/320326738 >> v2:172.21.15.79:3300/0 conn(0x7f94480aa020 0x7f94480a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f944559a700 1 -- 172.21.15.44:0/320326738 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781886043 0 0) 0x7f94380010d0 con 0x7f94480aa400 2020-11-20T12:08:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.584+0000 7f944559a700 1 -- 172.21.15.44:0/320326738 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f942c003240 con 0x7f94480aa400 2020-11-20T12:08:14.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944659c700 1 -- 172.21.15.44:0/320326738 >> v1:172.21.15.79:6789/0 conn(0x7f944813ac60 legacy=0x7f94480a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944559a700 1 -- 172.21.15.44:0/320326738 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1781089239 0 0) 0x7f94480a6da0 con 0x7f944813ac60 2020-11-20T12:08:14.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944659c700 1 -- 172.21.15.44:0/320326738 >> v1:172.21.15.44:6789/0 conn(0x7f94480aa400 legacy=0x7f94480a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944659c700 1 -- 172.21.15.44:0/320326738 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480a7040 con 0x7f94480aa020 2020-11-20T12:08:14.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944659c700 1 --2- 172.21.15.44:0/320326738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa020 0x7f94480a9280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f943c00b9a0 tx=0x7f943c006df0).ready entity=mon.1 client_cookie=ac2999530d7f0c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944559a700 1 -- 172.21.15.44:0/320326738 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c0097b0 con 0x7f94480aa020 2020-11-20T12:08:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944559a700 1 -- 172.21.15.44:0/320326738 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c009910 con 0x7f94480aa020 2020-11-20T12:08:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944559a700 1 -- 172.21.15.44:0/320326738 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c009c90 con 0x7f94480aa020 2020-11-20T12:08:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944cd64700 1 -- 172.21.15.44:0/320326738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa020 msgr2=0x7f94480a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944cd64700 1 --2- 172.21.15.44:0/320326738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa020 0x7f94480a9280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f943c00b9a0 tx=0x7f943c006df0).stop 2020-11-20T12:08:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944cd64700 1 -- 172.21.15.44:0/320326738 shutdown_connections 2020-11-20T12:08:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944cd64700 1 --2- 172.21.15.44:0/320326738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa020 0x7f94480a9280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944cd64700 1 -- 172.21.15.44:0/320326738 >> 172.21.15.44:0/320326738 conn(0x7f9448062690 msgr2=0x7f94480a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944cd64700 1 -- 172.21.15.44:0/320326738 shutdown_connections 2020-11-20T12:08:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.585+0000 7f944cd64700 1 -- 172.21.15.44:0/320326738 wait complete. 2020-11-20T12:08:14.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.586+0000 7f944cd64700 1 Processor -- start 2020-11-20T12:08:14.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.586+0000 7f944cd64700 1 -- start start 2020-11-20T12:08:14.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944cd64700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94480aa020 0x7f94480ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944cd64700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa400 0x7f94480a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944cd64700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f944813ac60 0x7f94480aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944cd64700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94480ac860 con 0x7f94480aa020 2020-11-20T12:08:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944cd64700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94480ab730 con 0x7f94480aa400 2020-11-20T12:08:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944cd64700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944809d5f0 con 0x7f944813ac60 2020-11-20T12:08:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94480aa020 0x7f94480ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94480aa020 0x7f94480ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38802/0 (socket says 172.21.15.44:38802) 2020-11-20T12:08:14.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944659c700 1 -- 172.21.15.44:0/1160805241 learned_addr learned my addr 172.21.15.44:0/1160805241 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:14.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f9445d9b700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa400 0x7f94480a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f9446d9d700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f944813ac60 0x7f94480aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944659c700 1 -- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f944813ac60 msgr2=0x7f94480aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944659c700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f944813ac60 0x7f94480aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944659c700 1 -- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa400 msgr2=0x7f94480a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.587+0000 7f944659c700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa400 0x7f94480a2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.588+0000 7f944659c700 1 -- 172.21.15.44:0/1160805241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9448066fb0 con 0x7f94480aa020 2020-11-20T12:08:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.588+0000 7f944659c700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94480aa020 0x7f94480ac370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f943c00be60 tx=0x7f943c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.588+0000 7f94367fc700 1 -- 172.21.15.44:0/1160805241 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c011d50 con 0x7f94480aa020 2020-11-20T12:08:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.588+0000 7f94367fc700 1 -- 172.21.15.44:0/1160805241 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c01e460 con 0x7f94480aa020 2020-11-20T12:08:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.588+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94480677e0 con 0x7f94480aa020 2020-11-20T12:08:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.588+0000 7f94367fc700 1 -- 172.21.15.44:0/1160805241 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c01e7e0 con 0x7f94480aa020 2020-11-20T12:08:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.588+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94480a32c0 con 0x7f94480aa020 2020-11-20T12:08:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.589+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9448061650 con 0x7f94480aa020 2020-11-20T12:08:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.589+0000 7f94367fc700 1 -- 172.21.15.44:0/1160805241 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f943c025030 con 0x7f94480aa020 2020-11-20T12:08:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.589+0000 7f94367fc700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f941c056d20 0x7f941c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.589+0000 7f94367fc700 1 -- 172.21.15.44:0/1160805241 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f943c0716d0 con 0x7f94480aa020 2020-11-20T12:08:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.590+0000 7f9445d9b700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f941c056d20 0x7f941c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.590+0000 7f9445d9b700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f941c056d20 0x7f941c0591e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f9438007fd0 tx=0x7f9438000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.593+0000 7f94367fc700 1 -- 172.21.15.44:0/1160805241 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f943c046a50 con 0x7f94480aa020 2020-11-20T12:08:14.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.837+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f94480446d0 con 0x7f94480aa020 2020-11-20T12:08:14.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.837+0000 7f94367fc700 1 -- 172.21.15.44:0/1160805241 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f943c023070 con 0x7f94480aa020 2020-11-20T12:08:14.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f941c056d20 msgr2=0x7f941c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f941c056d20 0x7f941c0591e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f9438007fd0 tx=0x7f9438000b80).stop 2020-11-20T12:08:14.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94480aa020 msgr2=0x7f94480ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:14.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94480aa020 0x7f94480ac370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f943c00be60 tx=0x7f943c015000).stop 2020-11-20T12:08:14.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 shutdown_connections 2020-11-20T12:08:14.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f941c056d20 0x7f941c0591e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f944813ac60 0x7f94480aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94480aa020 0x7f94480ac370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 --2- 172.21.15.44:0/1160805241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94480aa400 0x7f94480a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:14.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 >> 172.21.15.44:0/1160805241 conn(0x7f9448062690 msgr2=0x7f9448139e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:14.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 shutdown_connections 2020-11-20T12:08:14.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:14.839+0000 7f944cd64700 1 -- 172.21.15.44:0/1160805241 wait complete. 2020-11-20T12:08:14.854 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:08:24.855 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:08:24.856 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:08:24.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.994+0000 7fc438bf6700 1 Processor -- start 2020-11-20T12:08:24.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.994+0000 7fc438bf6700 1 -- start start 2020-11-20T12:08:25.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc438bf6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa020 0x7fc4340a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc438bf6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc4340aa400 0x7fc4340a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc438bf6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc43413ac60 0x7fc4340a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc438bf6700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4340a9770 con 0x7fc43413ac60 2020-11-20T12:08:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc438bf6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4340a6da0 con 0x7fc4340aa400 2020-11-20T12:08:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc438bf6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340a7040 con 0x7fc4340aa020 2020-11-20T12:08:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc432d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc43413ac60 0x7fc4340a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc432d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc43413ac60 0x7fc4340a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38814/0 (socket says 172.21.15.44:38814) 2020-11-20T12:08:25.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc432d9d700 1 -- 172.21.15.44:0/711341039 learned_addr learned my addr 172.21.15.44:0/711341039 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc431d9b700 1 --2- 172.21.15.44:0/711341039 >> v2:172.21.15.79:3300/0 conn(0x7fc4340aa400 0x7fc4340a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.995+0000 7fc43259c700 1 --2- 172.21.15.44:0/711341039 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa020 0x7fc4340a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc432d9d700 1 -- 172.21.15.44:0/711341039 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa020 msgr2=0x7fc4340a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc432d9d700 1 --2- 172.21.15.44:0/711341039 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa020 0x7fc4340a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc432d9d700 1 -- 172.21.15.44:0/711341039 >> v2:172.21.15.79:3300/0 conn(0x7fc4340aa400 msgr2=0x7fc4340a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc432d9d700 1 --2- 172.21.15.44:0/711341039 >> v2:172.21.15.79:3300/0 conn(0x7fc4340aa400 0x7fc4340a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc432d9d700 1 -- 172.21.15.44:0/711341039 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4340a6a90 con 0x7fc43413ac60 2020-11-20T12:08:25.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc432d9d700 1 --2- 172.21.15.44:0/711341039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc4340a3700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc434062b30 tx=0x7fc42c009290).ready entity=mon.0 client_cookie=ba4ebe94264be0b9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc43159a700 1 -- 172.21.15.44:0/711341039 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00bb10 con 0x7fc43413ac60 2020-11-20T12:08:25.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc43159a700 1 -- 172.21.15.44:0/711341039 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c00bc70 con 0x7fc43413ac60 2020-11-20T12:08:25.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc43159a700 1 -- 172.21.15.44:0/711341039 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c010470 con 0x7fc43413ac60 2020-11-20T12:08:25.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc438bf6700 1 -- 172.21.15.44:0/711341039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 msgr2=0x7fc4340a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc438bf6700 1 --2- 172.21.15.44:0/711341039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc4340a3700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc434062b30 tx=0x7fc42c009290).stop 2020-11-20T12:08:25.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc438bf6700 1 -- 172.21.15.44:0/711341039 shutdown_connections 2020-11-20T12:08:25.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc438bf6700 1 --2- 172.21.15.44:0/711341039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc4340a3700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc438bf6700 1 --2- 172.21.15.44:0/711341039 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa020 0x7fc4340a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc438bf6700 1 --2- 172.21.15.44:0/711341039 >> v2:172.21.15.79:3300/0 conn(0x7fc4340aa400 0x7fc4340a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.996+0000 7fc438bf6700 1 -- 172.21.15.44:0/711341039 >> 172.21.15.44:0/711341039 conn(0x7fc434062690 msgr2=0x7fc4340a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.997+0000 7fc438bf6700 1 -- 172.21.15.44:0/711341039 shutdown_connections 2020-11-20T12:08:25.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.997+0000 7fc438bf6700 1 -- 172.21.15.44:0/711341039 wait complete. 2020-11-20T12:08:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.997+0000 7fc438bf6700 1 Processor -- start 2020-11-20T12:08:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc438bf6700 1 -- start start 2020-11-20T12:08:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc438bf6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4340aa020 0x7fc43413a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc438bf6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa400 0x7fc4340abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc438bf6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc43414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc438bf6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340ac6b0 con 0x7fc43413ac60 2020-11-20T12:08:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc438bf6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43414d290 con 0x7fc4340aa020 2020-11-20T12:08:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc438bf6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43414d530 con 0x7fc4340aa400 2020-11-20T12:08:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc432d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc43414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc432d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc43414d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38820/0 (socket says 172.21.15.44:38820) 2020-11-20T12:08:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc432d9d700 1 -- 172.21.15.44:0/972668480 learned_addr learned my addr 172.21.15.44:0/972668480 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc431d9b700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa400 0x7fc4340abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.998+0000 7fc43259c700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4340aa020 0x7fc43413a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc432d9d700 1 -- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa400 msgr2=0x7fc4340abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc432d9d700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa400 0x7fc4340abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc432d9d700 1 -- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4340aa020 msgr2=0x7fc43413a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc432d9d700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4340aa020 0x7fc43413a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc432d9d700 1 -- 172.21.15.44:0/972668480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4340ac3b0 con 0x7fc43413ac60 2020-11-20T12:08:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc432d9d700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc43414d9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc42c0092d0 tx=0x7fc42c018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc42a7fc700 1 -- 172.21.15.44:0/972668480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0197f0 con 0x7fc43413ac60 2020-11-20T12:08:25.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc42a7fc700 1 -- 172.21.15.44:0/972668480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c019950 con 0x7fc43413ac60 2020-11-20T12:08:25.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc434149c20 con 0x7fc43413ac60 2020-11-20T12:08:25.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc42a7fc700 1 -- 172.21.15.44:0/972668480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c019cf0 con 0x7fc43413ac60 2020-11-20T12:08:25.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:24.999+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43414dec0 con 0x7fc43413ac60 2020-11-20T12:08:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.000+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43414d6f0 con 0x7fc43413ac60 2020-11-20T12:08:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.001+0000 7fc42a7fc700 1 -- 172.21.15.44:0/972668480 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc42c022650 con 0x7fc43413ac60 2020-11-20T12:08:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.001+0000 7fc42a7fc700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc410056d20 0x7fc4100591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.001+0000 7fc42a7fc700 1 -- 172.21.15.44:0/972668480 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc42c072120 con 0x7fc43413ac60 2020-11-20T12:08:25.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.001+0000 7fc43259c700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc410056d20 0x7fc4100591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.002+0000 7fc43259c700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc410056d20 0x7fc4100591e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc41c006fd0 tx=0x7fc41c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.004+0000 7fc42a7fc700 1 -- 172.21.15.44:0/972668480 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc42c04ae20 con 0x7fc43413ac60 2020-11-20T12:08:25.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.244+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc43414e0b0 con 0x7fc43413ac60 2020-11-20T12:08:25.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.244+0000 7fc42a7fc700 1 -- 172.21.15.44:0/972668480 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fc42c027020 con 0x7fc43413ac60 2020-11-20T12:08:25.249 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:08:25.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc410056d20 msgr2=0x7fc4100591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc410056d20 0x7fc4100591e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc41c006fd0 tx=0x7fc41c008040).stop 2020-11-20T12:08:25.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 msgr2=0x7fc43414d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc43414d9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc42c0092d0 tx=0x7fc42c018000).stop 2020-11-20T12:08:25.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 shutdown_connections 2020-11-20T12:08:25.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc410056d20 0x7fc4100591e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc43413ac60 0x7fc43414d9d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4340aa020 0x7fc43413a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 --2- 172.21.15.44:0/972668480 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4340aa400 0x7fc4340abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 >> 172.21.15.44:0/972668480 conn(0x7fc434062690 msgr2=0x7fc434136e30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 shutdown_connections 2020-11-20T12:08:25.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.246+0000 7fc438bf6700 1 -- 172.21.15.44:0/972668480 wait complete. 2020-11-20T12:08:25.261 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:08:25.261 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:08:25.262 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:08:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.386+0000 7f03fca2c700 1 Processor -- start 2020-11-20T12:08:25.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.386+0000 7f03fca2c700 1 -- start start 2020-11-20T12:08:25.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.386+0000 7f03fca2c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 0x7f03f80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.386+0000 7f03fca2c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f03f80aa400 0x7f03f80a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.386+0000 7f03fca2c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f80a9770 con 0x7f03f813ac60 2020-11-20T12:08:25.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.386+0000 7f03fca2c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f80a6da0 con 0x7f03f80aa400 2020-11-20T12:08:25.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.386+0000 7f03fca2c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80a7040 con 0x7f03f80aa020 2020-11-20T12:08:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f6d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f03f813ac60 0x7f03f80a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42576/0 (socket says 172.21.15.44:42576) 2020-11-20T12:08:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f6d9d700 1 -- 172.21.15.44:0/3969963345 learned_addr learned my addr 172.21.15.44:0/3969963345 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f659c700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 0x7f03f80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f5d9b700 1 --2- 172.21.15.44:0/3969963345 >> v2:172.21.15.79:3300/0 conn(0x7f03f80aa400 0x7f03f80a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027105786 0 0) 0x7f03f00010d0 con 0x7f03f813ac60 2020-11-20T12:08:25.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03dc003240 con 0x7f03f813ac60 2020-11-20T12:08:25.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2360907509 0 0) 0x7f03dc003240 con 0x7f03f813ac60 2020-11-20T12:08:25.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 msgr2=0x7f03f80a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 0x7f03f80a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 >> v2:172.21.15.79:3300/0 conn(0x7f03f80aa400 msgr2=0x7f03f80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 --2- 172.21.15.44:0/3969963345 >> v2:172.21.15.79:3300/0 conn(0x7f03f80aa400 0x7f03f80a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80a6a90 con 0x7f03f813ac60 2020-11-20T12:08:25.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f03f0001670 con 0x7f03f813ac60 2020-11-20T12:08:25.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 >> v1:172.21.15.44:6789/0 conn(0x7f03f813ac60 legacy=0x7f03f80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc003730 0x7f03dc003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 0x7f03dc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03dc0049d0 0x7f03dc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80a6da0 con 0x7f03dc0049d0 2020-11-20T12:08:25.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80a7040 con 0x7f03dc003730 2020-11-20T12:08:25.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0041d0 con 0x7f03f80aa020 2020-11-20T12:08:25.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f659c700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc003730 0x7f03dc003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f6d9d700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03dc0049d0 0x7f03dc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.387+0000 7f03f5d9b700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 0x7f03dc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f6d9d700 1 -- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 msgr2=0x7f03dc0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f6d9d700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03f80aa020 0x7f03dc0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f6d9d700 1 -- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc003730 msgr2=0x7f03dc003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f6d9d700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc003730 0x7f03dc003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f6d9d700 1 -- 172.21.15.44:0/3969963345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc003240 con 0x7f03dc0049d0 2020-11-20T12:08:25.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f659c700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc003730 0x7f03dc003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:08:25.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f6d9d700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03dc0049d0 0x7f03dc004e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f03f000c470 tx=0x7f03f0007980).ready entity=mon.0 client_cookie=ef547ee5eb60281d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f00072e0 con 0x7f03dc0049d0 2020-11-20T12:08:25.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f0007440 con 0x7f03dc0049d0 2020-11-20T12:08:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f559a700 1 -- 172.21.15.44:0/3969963345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0005840 con 0x7f03dc0049d0 2020-11-20T12:08:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03fca2c700 1 -- 172.21.15.44:0/3969963345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03dc0049d0 msgr2=0x7f03dc004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03fca2c700 1 --2- 172.21.15.44:0/3969963345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03dc0049d0 0x7f03dc004e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f03f000c470 tx=0x7f03f0007980).stop 2020-11-20T12:08:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.388+0000 7f03f6d9d700 1 -- 172.21.15.44:0/3969963345 reap_dead start 2020-11-20T12:08:25.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.389+0000 7f03fca2c700 1 -- 172.21.15.44:0/3969963345 shutdown_connections 2020-11-20T12:08:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.389+0000 7f03fca2c700 1 -- 172.21.15.44:0/3969963345 >> 172.21.15.44:0/3969963345 conn(0x7f03f8062690 msgr2=0x7f03f80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.389+0000 7f03fca2c700 1 -- 172.21.15.44:0/3969963345 shutdown_connections 2020-11-20T12:08:25.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.389+0000 7f03fca2c700 1 -- 172.21.15.44:0/3969963345 wait complete. 2020-11-20T12:08:25.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 Processor -- start 2020-11-20T12:08:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 -- start start 2020-11-20T12:08:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03dc0049d0 0x7f03f8060330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03f809de80 0x7f03f809e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f809ee30 0x7f03f8134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f8138860 con 0x7f03f809ee30 2020-11-20T12:08:25.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f809e300 con 0x7f03f809de80 2020-11-20T12:08:25.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03fca2c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f809f2b0 con 0x7f03dc0049d0 2020-11-20T12:08:25.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f809ee30 0x7f03f8134060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03f659c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03dc0049d0 0x7f03f8060330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03f659c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03dc0049d0 0x7f03f8060330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50274/0 (socket says 172.21.15.44:50274) 2020-11-20T12:08:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f809ee30 0x7f03f8134060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38840/0 (socket says 172.21.15.44:38840) 2020-11-20T12:08:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03f659c700 1 -- 172.21.15.44:0/1911230831 learned_addr learned my addr 172.21.15.44:0/1911230831 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.390+0000 7f03f5d9b700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03f809de80 0x7f03f809e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03f6d9d700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03dc0049d0 msgr2=0x7f03f8060330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03f6d9d700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03dc0049d0 0x7f03f8060330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03f6d9d700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03f809de80 msgr2=0x7f03f809e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03f6d9d700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03f809de80 0x7f03f809e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03f6d9d700 1 -- 172.21.15.44:0/1911230831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80a0850 con 0x7f03f809ee30 2020-11-20T12:08:25.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03f6d9d700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f809ee30 0x7f03f8134060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f03f000c470 tx=0x7f03f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f000ceb0 con 0x7f03f809ee30 2020-11-20T12:08:25.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f0001570 con 0x7f03f809ee30 2020-11-20T12:08:25.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f80a0af0 con 0x7f03f809ee30 2020-11-20T12:08:25.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0005bb0 con 0x7f03f809ee30 2020-11-20T12:08:25.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.391+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f80609c0 con 0x7f03f809ee30 2020-11-20T12:08:25.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.392+0000 7f03fca2c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134550 0x7f03f80acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.392+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80a0f50 con 0x7f03f8134550 2020-11-20T12:08:25.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.392+0000 7f03f659c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134550 0x7f03f80acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.392+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f03f0010070 con 0x7f03f809ee30 2020-11-20T12:08:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.392+0000 7f03f659c700 1 -- 172.21.15.44:0/1911230831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03f8061650 con 0x7f03f8134550 2020-11-20T12:08:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.392+0000 7f03f659c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134550 0x7f03f80acdb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f03ec006a90 tx=0x7f03ec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.392+0000 7f03e67fc700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03cc056d20 0x7f03cc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f03f0070bd0 con 0x7f03f809ee30 2020-11-20T12:08:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec012de0 con 0x7f03f8134550 2020-11-20T12:08:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03f5d9b700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03cc056d20 0x7f03cc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f03ec019070 con 0x7f03f8134550 2020-11-20T12:08:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134550 msgr2=0x7f03f80acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03e67fc700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134550 0x7f03f80acdb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f03ec006a90 tx=0x7f03ec006df0).stop 2020-11-20T12:08:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134550 msgr2=0x7f03f80acdb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03e67fc700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134550 0x7f03f80acdb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.393+0000 7f03f5d9b700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03cc056d20 0x7f03cc0591e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f03f8138a40 tx=0x7f03e80091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03fca2c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134930 0x7f03f80ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80a9770 con 0x7f03f8134930 2020-11-20T12:08:25.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03f659c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134930 0x7f03f80ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03f659c700 1 -- 172.21.15.44:0/1911230831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03f80446d0 con 0x7f03f8134930 2020-11-20T12:08:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03f659c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134930 0x7f03f80ad2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f03ec015010 tx=0x7f03ec01c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec01e880 con 0x7f03f8134930 2020-11-20T12:08:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7f03ec012bd0 con 0x7f03f8134930 2020-11-20T12:08:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134930 msgr2=0x7f03f80ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03e67fc700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134930 0x7f03f80ad2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f03ec015010 tx=0x7f03ec01c000).stop 2020-11-20T12:08:25.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03e67fc700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134930 msgr2=0x7f03f80ad2a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:25.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.401+0000 7f03e67fc700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f8134930 0x7f03f80ad2a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03cc056d20 msgr2=0x7f03cc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f03cc056d20 0x7f03cc0591e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f03f8138a40 tx=0x7f03e80091d0).stop 2020-11-20T12:08:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f809ee30 msgr2=0x7f03f8134060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 --2- 172.21.15.44:0/1911230831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03f809ee30 0x7f03f8134060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f03f000c470 tx=0x7f03f0015000).stop 2020-11-20T12:08:25.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03f6d9d700 1 -- 172.21.15.44:0/1911230831 reap_dead start 2020-11-20T12:08:25.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03f6d9d700 1 -- 172.21.15.44:0/1911230831 reap_dead start 2020-11-20T12:08:25.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 shutdown_connections 2020-11-20T12:08:25.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 >> 172.21.15.44:0/1911230831 conn(0x7f03f8062690 msgr2=0x7f03f80a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 shutdown_connections 2020-11-20T12:08:25.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.402+0000 7f03fca2c700 1 -- 172.21.15.44:0/1911230831 wait complete. 2020-11-20T12:08:25.430 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:25.430 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:25.430 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:25.431 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:08:25.431 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 50, 2020-11-20T12:08:25.431 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:25.431 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:25.431 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:08:25.432 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:25.432 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.432 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 11, 2020-11-20T12:08:25.433 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:25.433 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:25.433 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:25.433 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.434 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.434 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.434 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.435 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.435 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.436 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.436 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.436 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.437 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:25.437 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:25.438 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.438 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.439 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.439 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.440 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.440 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.441 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.441 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.441 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.442 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:25.442 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:25.442 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:25.443 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:25.443 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:25.443 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:25.444 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:25.444 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:25.444 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:25.445 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:25.445 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:25.445 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:25.446 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:25.446 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:25.446 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:25.446 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.446 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.447 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.447 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.447 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.447 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.447 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.448 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.448 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.448 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:25.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.448 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:25.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.449 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:25.449 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:25.449 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.449 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.450 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:25.450 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.451 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.451 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:25.451 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.452 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.452 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.452 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:25.452 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:25.453 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.453 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.453 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.454 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:25.454 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:25.454 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.455 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.455 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.455 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:25.455 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.456 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:25.456 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.457 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.457 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.457 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:25.458 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:25.458 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.458 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.458 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.459 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:25.459 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:25.459 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.459 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.460 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.460 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:25.460 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.461 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:25.461 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.462 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.462 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:25.462 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:25.462 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.462 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.463 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.463 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.463 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.464 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:25.464 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:25.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.464 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.464 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.465 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.465 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.465 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.465 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:08:25.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.466 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.466 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.466 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.466 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.466 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.467 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:08:25.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.467 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.467 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.467 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.468 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.468 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:25.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.468 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.469 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.469 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:08:25.469 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.469 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.469 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:08:25.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.470 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.470 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.470 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.470 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.471 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.471 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:25.471 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:25.472 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:08:25.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.595+0000 7f9e691db700 1 Processor -- start 2020-11-20T12:08:25.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.595+0000 7f9e691db700 1 -- start start 2020-11-20T12:08:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e691db700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9e640aa020 0x7f9e640a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e691db700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640a9770 con 0x7f9e640aa400 2020-11-20T12:08:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e691db700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640a6da0 con 0x7f9e6413ac60 2020-11-20T12:08:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e691db700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e640a56c0 con 0x7f9e640aa020 2020-11-20T12:08:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e6259c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9e640aa400 0x7f9e640a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42596/0 (socket says 172.21.15.44:42596) 2020-11-20T12:08:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e6259c700 1 -- 172.21.15.44:0/679193836 learned_addr learned my addr 172.21.15.44:0/679193836 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e62d9d700 1 --2- 172.21.15.44:0/679193836 >> v2:172.21.15.79:3300/0 conn(0x7f9e640aa020 0x7f9e640a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e61d9b700 1 -- 172.21.15.44:0/679193836 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198368310 0 0) 0x7f9e540010d0 con 0x7f9e640aa400 2020-11-20T12:08:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.596+0000 7f9e61d9b700 1 -- 172.21.15.44:0/679193836 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e44003240 con 0x7f9e640aa400 2020-11-20T12:08:25.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e62d9d700 1 -- 172.21.15.44:0/679193836 >> v1:172.21.15.79:6789/0 conn(0x7f9e6413ac60 legacy=0x7f9e640a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e61d9b700 1 -- 172.21.15.44:0/679193836 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701234120 0 0) 0x7f9e640a6da0 con 0x7f9e6413ac60 2020-11-20T12:08:25.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e62d9d700 1 -- 172.21.15.44:0/679193836 >> v1:172.21.15.44:6789/0 conn(0x7f9e640aa400 legacy=0x7f9e640a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e62d9d700 1 -- 172.21.15.44:0/679193836 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640a7040 con 0x7f9e640aa020 2020-11-20T12:08:25.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e62d9d700 1 --2- 172.21.15.44:0/679193836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 0x7f9e640a9280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9e4c00b9a0 tx=0x7f9e4c006df0).ready entity=mon.1 client_cookie=184df40dea6c10a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e61d9b700 1 -- 172.21.15.44:0/679193836 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0097b0 con 0x7f9e640aa020 2020-11-20T12:08:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e61d9b700 1 -- 172.21.15.44:0/679193836 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c009910 con 0x7f9e640aa020 2020-11-20T12:08:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e61d9b700 1 -- 172.21.15.44:0/679193836 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c009c90 con 0x7f9e640aa020 2020-11-20T12:08:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e691db700 1 -- 172.21.15.44:0/679193836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 msgr2=0x7f9e640a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e691db700 1 --2- 172.21.15.44:0/679193836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 0x7f9e640a9280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9e4c00b9a0 tx=0x7f9e4c006df0).stop 2020-11-20T12:08:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e691db700 1 -- 172.21.15.44:0/679193836 shutdown_connections 2020-11-20T12:08:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e691db700 1 --2- 172.21.15.44:0/679193836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 0x7f9e640a9280 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e691db700 1 -- 172.21.15.44:0/679193836 >> 172.21.15.44:0/679193836 conn(0x7f9e64062690 msgr2=0x7f9e640a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e691db700 1 -- 172.21.15.44:0/679193836 shutdown_connections 2020-11-20T12:08:25.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.597+0000 7f9e691db700 1 -- 172.21.15.44:0/679193836 wait complete. 2020-11-20T12:08:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.598+0000 7f9e691db700 1 Processor -- start 2020-11-20T12:08:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e691db700 1 -- start start 2020-11-20T12:08:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e691db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 0x7f9e64139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e691db700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9e640aa400 0x7f9e640ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e691db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e6413ac60 0x7f9e640ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e691db700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640acb80 con 0x7f9e6413ac60 2020-11-20T12:08:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e691db700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64065a50 con 0x7f9e640aa020 2020-11-20T12:08:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e691db700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640657b0 con 0x7f9e640aa400 2020-11-20T12:08:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e6359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e6413ac60 0x7f9e640ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e6359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e6413ac60 0x7f9e640ab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38856/0 (socket says 172.21.15.44:38856) 2020-11-20T12:08:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e6359e700 1 -- 172.21.15.44:0/1514375518 learned_addr learned my addr 172.21.15.44:0/1514375518 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e62d9d700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 0x7f9e64139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.599+0000 7f9e6259c700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9e640aa400 0x7f9e640ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e6359e700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9e640aa400 msgr2=0x7f9e640ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e6359e700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9e640aa400 0x7f9e640ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e6359e700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 msgr2=0x7f9e64139ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e6359e700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640aa020 0x7f9e64139ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e6359e700 1 -- 172.21.15.44:0/1514375518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640ab020 con 0x7f9e6413ac60 2020-11-20T12:08:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e6359e700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e6413ac60 0x7f9e640ab4c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f9e5c00c860 tx=0x7f9e5c00cc20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00a7b0 con 0x7f9e6413ac60 2020-11-20T12:08:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c00a910 con 0x7f9e6413ac60 2020-11-20T12:08:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00acb0 con 0x7f9e6413ac60 2020-11-20T12:08:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e64149160 con 0x7f9e6413ac60 2020-11-20T12:08:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.600+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e64065230 con 0x7f9e6413ac60 2020-11-20T12:08:25.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.601+0000 7f9e691db700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e641496a0 0x7f9e64149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.601+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641366b0 con 0x7f9e641496a0 2020-11-20T12:08:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.601+0000 7f9e62d9d700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e641496a0 0x7f9e64149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.601+0000 7f9e62d9d700 1 -- 172.21.15.44:0/1514375518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e640ab1b0 con 0x7f9e641496a0 2020-11-20T12:08:25.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.601+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9e5c010070 con 0x7f9e6413ac60 2020-11-20T12:08:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.601+0000 7f9e62d9d700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e641496a0 0x7f9e64149a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f9e4c00b9a0 tx=0x7f9e4c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9e40056d50 0x7f9e40059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9e5c06ff60 con 0x7f9e6413ac60 2020-11-20T12:08:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c011d50 con 0x7f9e641496a0 2020-11-20T12:08:25.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e6259c700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9e40056d50 0x7f9e40059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f9e4c01f440 con 0x7f9e641496a0 2020-11-20T12:08:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e641496a0 msgr2=0x7f9e64149a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e641496a0 0x7f9e64149a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f9e4c00b9a0 tx=0x7f9e4c015000).stop 2020-11-20T12:08:25.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e641496a0 msgr2=0x7f9e64149a80 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e5affd700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e641496a0 0x7f9e64149a80 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0x7f9e4c00b9a0 tx=0x7f9e4c015000).stop 2020-11-20T12:08:25.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.602+0000 7f9e6259c700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9e40056d50 0x7f9e40059210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f9e6413c670 tx=0x7f9e54009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.610+0000 7f9e691db700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640a7740 0x7f9e640a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.610+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64061650 con 0x7f9e640a7740 2020-11-20T12:08:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.610+0000 7f9e62d9d700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640a7740 0x7f9e640a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.610+0000 7f9e62d9d700 1 -- 172.21.15.44:0/1514375518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9e640446d0 con 0x7f9e640a7740 2020-11-20T12:08:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.610+0000 7f9e62d9d700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640a7740 0x7f9e640a7b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9e4c00be60 tx=0x7f9e4c020000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c02eb30 con 0x7f9e640a7740 2020-11-20T12:08:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f9e4c011b40 con 0x7f9e640a7740 2020-11-20T12:08:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640a7740 msgr2=0x7f9e640a7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e5affd700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640a7740 0x7f9e640a7b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9e4c00be60 tx=0x7f9e4c020000).stop 2020-11-20T12:08:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e5affd700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640a7740 msgr2=0x7f9e640a7b50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e5affd700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9e640a7740 0x7f9e640a7b50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9e40056d50 msgr2=0x7f9e40059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9e40056d50 0x7f9e40059210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f9e6413c670 tx=0x7f9e54009040).stop 2020-11-20T12:08:25.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e6413ac60 msgr2=0x7f9e640ab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 --2- 172.21.15.44:0/1514375518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e6413ac60 0x7f9e640ab4c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f9e5c00c860 tx=0x7f9e5c00cc20).stop 2020-11-20T12:08:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e6359e700 1 -- 172.21.15.44:0/1514375518 reap_dead start 2020-11-20T12:08:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e6359e700 1 -- 172.21.15.44:0/1514375518 reap_dead start 2020-11-20T12:08:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 shutdown_connections 2020-11-20T12:08:25.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 >> 172.21.15.44:0/1514375518 conn(0x7f9e64062690 msgr2=0x7f9e64136b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 shutdown_connections 2020-11-20T12:08:25.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.611+0000 7f9e691db700 1 -- 172.21.15.44:0/1514375518 wait complete. 2020-11-20T12:08:25.624 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:25.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:25.625 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:25.625 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:08:25.625 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 50, 2020-11-20T12:08:25.625 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:25.625 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:25.626 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:08:25.626 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:25.626 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.626 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:08:25.626 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:25.627 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:25.627 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:25.627 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.627 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.627 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.628 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.628 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.628 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.628 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.628 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.629 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.629 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:25.629 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:25.629 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.629 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.630 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.630 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.630 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.630 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.630 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.631 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.631 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.631 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:25.631 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:25.632 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:25.632 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:25.632 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:25.632 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:25.632 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:25.633 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:25.633 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:25.633 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:25.633 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:25.633 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:25.634 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:25.634 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:25.634 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:25.634 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.634 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.635 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.635 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.635 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.635 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.635 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.636 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.636 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.636 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:25.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.636 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:25.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.637 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:25.637 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:25.637 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.637 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.638 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:25.638 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.639 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.639 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:25.639 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.639 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.640 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.640 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:25.640 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:25.640 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.640 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.641 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.641 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:25.641 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:25.642 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.642 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.642 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:25.643 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.643 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.643 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:25.644 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.644 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.644 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.644 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:25.645 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:25.645 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.645 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.645 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.646 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:25.646 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:25.646 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.646 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.647 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.647 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:25.647 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.648 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:25.648 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.649 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.649 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:25.649 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:25.649 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.651 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:25.651 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:25.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.651 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.651 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.652 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.652 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.652 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.652 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:25.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.653 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.653 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.653 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.653 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.654 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:25.654 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:25.655 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:08:25.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.779+0000 7f4f18b0a700 1 Processor -- start 2020-11-20T12:08:25.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.779+0000 7f4f18b0a700 1 -- start start 2020-11-20T12:08:25.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.779+0000 7f4f18b0a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f14060330 0x7f4f140a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.779+0000 7f4f18b0a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f140a6260 con 0x7f4f140a4c30 2020-11-20T12:08:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.779+0000 7f4f18b0a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f140a6490 con 0x7f4f1413b0a0 2020-11-20T12:08:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.779+0000 7f4f18b0a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140a52e0 con 0x7f4f14060330 2020-11-20T12:08:25.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f11d9b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f4f140a4c30 0x7f4f1413a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42616/0 (socket says 172.21.15.44:42616) 2020-11-20T12:08:25.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f11d9b700 1 -- 172.21.15.44:0/2586611844 learned_addr learned my addr 172.21.15.44:0/2586611844 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:25.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1259c700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f14060330 0x7f4f140a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407286313 0 0) 0x7f4f140a6260 con 0x7f4f140a4c30 2020-11-20T12:08:25.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ef8003240 con 0x7f4f140a4c30 2020-11-20T12:08:25.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361922056 0 0) 0x7f4f140a6490 con 0x7f4f1413b0a0 2020-11-20T12:08:25.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f140a6260 con 0x7f4f1413b0a0 2020-11-20T12:08:25.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1147051759 0 0) 0x7f4ef8003240 con 0x7f4f140a4c30 2020-11-20T12:08:25.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f14060330 msgr2=0x7f4f140a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f14060330 0x7f4f140a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 >> v1:172.21.15.79:6789/0 conn(0x7f4f1413b0a0 legacy=0x7f4f14141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140a6730 con 0x7f4f140a4c30 2020-11-20T12:08:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f4f040017e0 con 0x7f4f140a4c30 2020-11-20T12:08:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 >> v1:172.21.15.44:6789/0 conn(0x7f4f140a4c30 legacy=0x7f4f1413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ef80038f0 0x7f4ef8003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4ef80044e0 0x7f4ef8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4ef800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140a52e0 con 0x7f4ef8004e50 2020-11-20T12:08:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef800bc20 con 0x7f4ef80038f0 2020-11-20T12:08:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef800bd60 con 0x7f4ef80044e0 2020-11-20T12:08:25.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1259c700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f14060330 0x7f4f140a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:08:25.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f12d9d700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4ef800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f11d9b700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4ef80044e0 0x7f4ef8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.780+0000 7f4f1259c700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ef80038f0 0x7f4ef8003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f12d9d700 1 -- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4ef80044e0 msgr2=0x7f4ef8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f12d9d700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4ef80044e0 0x7f4ef8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f12d9d700 1 -- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ef80038f0 msgr2=0x7f4ef8003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:25.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f12d9d700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ef80038f0 0x7f4ef8003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f12d9d700 1 -- 172.21.15.44:0/2586611844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8003240 con 0x7f4ef8004e50 2020-11-20T12:08:25.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f12d9d700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4ef800b730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4f0c007e50 tx=0x7f4f0c007b90).ready entity=mon.0 client_cookie=20472892be555641 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c007020 con 0x7f4ef8004e50 2020-11-20T12:08:25.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c007180 con 0x7f4ef8004e50 2020-11-20T12:08:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f1159a700 1 -- 172.21.15.44:0/2586611844 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00c830 con 0x7f4ef8004e50 2020-11-20T12:08:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f18b0a700 1 -- 172.21.15.44:0/2586611844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 msgr2=0x7f4ef800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f18b0a700 1 --2- 172.21.15.44:0/2586611844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4ef800b730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f4f0c007e50 tx=0x7f4f0c007b90).stop 2020-11-20T12:08:25.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f12d9d700 1 -- 172.21.15.44:0/2586611844 reap_dead start 2020-11-20T12:08:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f18b0a700 1 -- 172.21.15.44:0/2586611844 shutdown_connections 2020-11-20T12:08:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f18b0a700 1 -- 172.21.15.44:0/2586611844 >> 172.21.15.44:0/2586611844 conn(0x7f4f14062690 msgr2=0x7f4f140a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.781+0000 7f4f18b0a700 1 -- 172.21.15.44:0/2586611844 shutdown_connections 2020-11-20T12:08:25.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.782+0000 7f4f18b0a700 1 -- 172.21.15.44:0/2586611844 wait complete. 2020-11-20T12:08:25.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.782+0000 7f4f18b0a700 1 Processor -- start 2020-11-20T12:08:25.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f18b0a700 1 -- start start 2020-11-20T12:08:25.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f18b0a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4f140ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f18b0a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f140ad110 0x7f4f14115870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f18b0a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414d780 0x7f4f1414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f18b0a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140ad590 con 0x7f4ef8004e50 2020-11-20T12:08:25.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f18b0a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140ad780 con 0x7f4f140ad110 2020-11-20T12:08:25.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f18b0a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140abca0 con 0x7f4f1414d780 2020-11-20T12:08:25.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4f140ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4f140ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38876/0 (socket says 172.21.15.44:38876) 2020-11-20T12:08:25.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 -- 172.21.15.44:0/1408977238 learned_addr learned my addr 172.21.15.44:0/1408977238 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:25.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f11d9b700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f140ad110 0x7f4f14115870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f12d9d700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414d780 0x7f4f1414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414d780 msgr2=0x7f4f1414dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414d780 0x7f4f1414dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f140ad110 msgr2=0x7f4f14115870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f140ad110 0x7f4f14115870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.783+0000 7f4f1259c700 1 -- 172.21.15.44:0/1408977238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140abf40 con 0x7f4ef8004e50 2020-11-20T12:08:25.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.784+0000 7f4f1259c700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4f140ac3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f4f08011c20 tx=0x7f4f08006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.784+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800fb30 con 0x7f4ef8004e50 2020-11-20T12:08:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.784+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0800fc90 con 0x7f4ef8004e50 2020-11-20T12:08:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.784+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f14133a50 con 0x7f4ef8004e50 2020-11-20T12:08:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.784+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08014440 con 0x7f4ef8004e50 2020-11-20T12:08:25.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.784+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f14115d60 con 0x7f4ef8004e50 2020-11-20T12:08:25.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f18b0a700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414e0d0 0x7f4f1414a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140ac0d0 con 0x7f4f1414e0d0 2020-11-20T12:08:25.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f11d9b700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414e0d0 0x7f4f1414a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4f08014870 con 0x7f4ef8004e50 2020-11-20T12:08:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f11d9b700 1 -- 172.21.15.44:0/1408977238 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f14061650 con 0x7f4f1414e0d0 2020-11-20T12:08:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f11d9b700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414e0d0 0x7f4f1414a9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f4f040048e0 tx=0x7f4f140a6620).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f027fc700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ee8056d20 0x7f4ee80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4f08070cf0 con 0x7f4ef8004e50 2020-11-20T12:08:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04012db0 con 0x7f4f1414e0d0 2020-11-20T12:08:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.785+0000 7f4f12d9d700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ee8056d20 0x7f4ee80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.786+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4f040143a0 con 0x7f4f1414e0d0 2020-11-20T12:08:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.786+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414e0d0 msgr2=0x7f4f1414a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.786+0000 7f4f027fc700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414e0d0 0x7f4f1414a9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f4f040048e0 tx=0x7f4f140a6620).stop 2020-11-20T12:08:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.786+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414e0d0 msgr2=0x7f4f1414a9a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.786+0000 7f4f027fc700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414e0d0 0x7f4f1414a9a0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0x7f4f040048e0 tx=0x7f4f140a6620).stop 2020-11-20T12:08:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.786+0000 7f4f12d9d700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ee8056d20 0x7f4ee80591e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f4f14061250 tx=0x7f4f0c018000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.793+0000 7f4f18b0a700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414ae90 0x7f4f14025a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:25.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.793+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140446d0 con 0x7f4f1414ae90 2020-11-20T12:08:25.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.794+0000 7f4f11d9b700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414ae90 0x7f4f14025a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:25.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.794+0000 7f4f11d9b700 1 -- 172.21.15.44:0/1408977238 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4f140680f0 con 0x7f4f1414ae90 2020-11-20T12:08:25.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.794+0000 7f4f11d9b700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414ae90 0x7f4f14025a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4f040048e0 tx=0x7f4f04009030).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:25.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04015b60 con 0x7f4f1414ae90 2020-11-20T12:08:25.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f4f04012ba0 con 0x7f4f1414ae90 2020-11-20T12:08:25.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414ae90 msgr2=0x7f4f14025a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f027fc700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414ae90 0x7f4f14025a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4f040048e0 tx=0x7f4f04009030).stop 2020-11-20T12:08:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f027fc700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414ae90 msgr2=0x7f4f14025a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f027fc700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4f1414ae90 0x7f4f14025a60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ee8056d20 msgr2=0x7f4ee80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f18b0a700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ee8056d20 0x7f4ee80591e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f4f14061250 tx=0x7f4f0c018000).stop 2020-11-20T12:08:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 msgr2=0x7f4f140ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f18b0a700 1 --2- 172.21.15.44:0/1408977238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ef8004e50 0x7f4f140ac3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f4f08011c20 tx=0x7f4f08006df0).stop 2020-11-20T12:08:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f12d9d700 1 -- 172.21.15.44:0/1408977238 reap_dead start 2020-11-20T12:08:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.795+0000 7f4f12d9d700 1 -- 172.21.15.44:0/1408977238 reap_dead start 2020-11-20T12:08:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.796+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 shutdown_connections 2020-11-20T12:08:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.796+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 >> 172.21.15.44:0/1408977238 conn(0x7f4f14062690 msgr2=0x7f4f140a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.796+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 shutdown_connections 2020-11-20T12:08:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.796+0000 7f4f18b0a700 1 -- 172.21.15.44:0/1408977238 wait complete. 2020-11-20T12:08:25.819 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:25.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:25.820 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:25.820 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:08:25.820 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 50, 2020-11-20T12:08:25.821 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:25.821 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:25.821 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:08:25.821 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:25.821 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.822 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:08:25.822 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:25.822 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:25.822 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:25.823 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.823 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.823 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.823 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.823 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.824 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.824 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.824 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.824 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.825 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:25.825 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:25.825 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.825 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.825 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.826 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.826 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.826 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.826 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.826 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.827 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.827 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:25.827 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:25.827 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:25.828 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:25.828 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:25.828 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:25.828 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:25.829 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:25.829 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:25.829 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:25.829 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:25.829 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:25.830 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:25.830 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:25.830 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:25.831 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:25.831 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:25.831 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:25.831 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:25.831 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:25.832 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:25.832 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:25.832 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:25.832 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.833 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:25.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.833 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:25.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.834 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:25.834 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:25.834 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.834 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.835 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.835 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:25.835 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.836 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:25.836 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.837 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.837 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.838 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:25.838 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:25.838 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.838 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.839 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.839 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:25.840 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:25.840 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.840 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.840 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.841 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:25.841 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.842 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:25.842 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.842 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.842 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.843 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:25.843 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:25.843 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.844 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.844 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.845 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:25.845 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:25.845 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:25.845 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:25.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.846 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:25.846 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:25.846 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.847 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:25.847 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:25.847 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:25.848 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.848 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.848 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:25.848 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:25.849 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:25.849 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:25.849 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:25.849 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.849 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.850 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:25.850 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:25.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.851 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.851 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.851 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.852 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.852 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:08:25.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.852 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.852 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.853 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.853 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.853 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.853 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:08:25.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.854 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.854 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.854 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:08:25.855 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.855 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.855 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:25.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.855 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.856 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.856 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:08:25.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.856 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:25.856 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:08:25.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:25.857 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:25.857 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:25.857 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:25.858 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:25.858 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:25.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:25.858 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:25.859 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:25.859 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:08:25.859 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:08:25.860 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:08:25.860 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:08:25.861 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:08:25.861 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:08:25.871 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:08:25.872 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:08:25.872 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:08:25.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f6b54b700 1 Processor -- start 2020-11-20T12:08:25.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f6b54b700 1 -- start start 2020-11-20T12:08:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f6b54b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1f640aa400 0x7f1f640a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f6b54b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1f6413ac60 0x7f1f640a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f6b54b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640a9770 con 0x7f1f640aa020 2020-11-20T12:08:26.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f6b54b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640a56c0 con 0x7f1f6413ac60 2020-11-20T12:08:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f6b54b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640a6da0 con 0x7f1f640aa400 2020-11-20T12:08:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.994+0000 7f1f69ae8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1f6413ac60 0x7f1f640a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f692e7700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1f640aa020 0x7f1f640a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42636/0 (socket says 172.21.15.44:42636) 2020-11-20T12:08:26.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f68ae6700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f1f640aa400 msgr2=0x7f1f640a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:26.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f692e7700 1 -- 172.21.15.44:0/3283689719 learned_addr learned my addr 172.21.15.44:0/3283689719 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:26.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f68ae6700 1 --2- 172.21.15.44:0/3283689719 >> v2:172.21.15.79:3300/0 conn(0x7f1f640aa400 0x7f1f640a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:26.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f69ae8700 1 -- 172.21.15.44:0/3283689719 >> v2:172.21.15.79:3300/0 conn(0x7f1f640aa400 msgr2=0x7f1f640a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:26.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f69ae8700 1 --2- 172.21.15.44:0/3283689719 >> v2:172.21.15.79:3300/0 conn(0x7f1f640aa400 0x7f1f640a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:26.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f69ae8700 1 -- 172.21.15.44:0/3283689719 >> v1:172.21.15.44:6789/0 conn(0x7f1f640aa020 legacy=0x7f1f640a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f57fff700 1 -- 172.21.15.44:0/3283689719 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450643269 0 0) 0x7f1f580010d0 con 0x7f1f640aa020 2020-11-20T12:08:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f69ae8700 1 -- 172.21.15.44:0/3283689719 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640a7040 con 0x7f1f6413ac60 2020-11-20T12:08:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f69ae8700 1 --2- 172.21.15.44:0/3283689719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640a3700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f1f640a4990 tx=0x7f1f5c006df0).ready entity=mon.0 client_cookie=44abebe3d368e74c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f57fff700 1 -- 172.21.15.44:0/3283689719 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0097b0 con 0x7f1f6413ac60 2020-11-20T12:08:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f57fff700 1 -- 172.21.15.44:0/3283689719 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c009910 con 0x7f1f6413ac60 2020-11-20T12:08:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.995+0000 7f1f57fff700 1 -- 172.21.15.44:0/3283689719 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c009c90 con 0x7f1f6413ac60 2020-11-20T12:08:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3283689719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 msgr2=0x7f1f640a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3283689719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640a3700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f1f640a4990 tx=0x7f1f5c006df0).stop 2020-11-20T12:08:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3283689719 shutdown_connections 2020-11-20T12:08:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3283689719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640a3700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3283689719 >> v2:172.21.15.79:3300/0 conn(0x7f1f640aa400 0x7f1f640a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3283689719 >> 172.21.15.44:0/3283689719 conn(0x7f1f64062690 msgr2=0x7f1f640a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3283689719 shutdown_connections 2020-11-20T12:08:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.996+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3283689719 wait complete. 2020-11-20T12:08:26.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 Processor -- start 2020-11-20T12:08:26.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 -- start start 2020-11-20T12:08:26.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f640aa020 0x7f1f640abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:26.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f640aa400 0x7f1f640aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:26.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640675d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:26.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64068240 con 0x7f1f6413ac60 2020-11-20T12:08:26.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640ab9a0 con 0x7f1f640aa020 2020-11-20T12:08:26.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.997+0000 7f1f6b54b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64135300 con 0x7f1f640aa400 2020-11-20T12:08:26.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f692e7700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f640aa020 msgr2=0x7f1f640abe50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640675d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38898/0 (socket says 172.21.15.44:38898) 2020-11-20T12:08:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 -- 172.21.15.44:0/3869453245 learned_addr learned my addr 172.21.15.44:0/3869453245 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f692e7700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f640aa020 0x7f1f640abe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:26.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f68ae6700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f640aa400 0x7f1f640aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:26.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 -- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f640aa400 msgr2=0x7f1f640aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:26.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f640aa400 0x7f1f640aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:26.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 -- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f640aa020 msgr2=0x7f1f640abe50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:26.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f640aa020 0x7f1f640abe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:26.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 -- 172.21.15.44:0/3869453245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640ab700 con 0x7f1f6413ac60 2020-11-20T12:08:26.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f69ae8700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640675d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1f64061280 tx=0x7f1f5c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:26.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f557fa700 1 -- 172.21.15.44:0/3869453245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c01d5d0 con 0x7f1f6413ac60 2020-11-20T12:08:26.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f557fa700 1 -- 172.21.15.44:0/3869453245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c01d730 con 0x7f1f6413ac60 2020-11-20T12:08:26.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f64146df0 con 0x7f1f6413ac60 2020-11-20T12:08:26.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f557fa700 1 -- 172.21.15.44:0/3869453245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c01dab0 con 0x7f1f6413ac60 2020-11-20T12:08:26.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.998+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640ab4a0 con 0x7f1f6413ac60 2020-11-20T12:08:26.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.999+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f64061650 con 0x7f1f6413ac60 2020-11-20T12:08:26.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:25.999+0000 7f1f557fa700 1 -- 172.21.15.44:0/3869453245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1f5c025030 con 0x7f1f6413ac60 2020-11-20T12:08:26.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.000+0000 7f1f557fa700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f3c056cd0 0x7f1f3c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:26.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.000+0000 7f1f557fa700 1 -- 172.21.15.44:0/3869453245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1f5c071bf0 con 0x7f1f6413ac60 2020-11-20T12:08:26.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.000+0000 7f1f692e7700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f3c056cd0 0x7f1f3c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:26.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.000+0000 7f1f692e7700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f3c056cd0 0x7f1f3c059190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f1f58001230 tx=0x7f1f58007700).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:26.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.004+0000 7f1f557fa700 1 -- 172.21.15.44:0/3869453245 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1f5c046f70 con 0x7f1f6413ac60 2020-11-20T12:08:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.246+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f64146f80 con 0x7f1f6413ac60 2020-11-20T12:08:26.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.246+0000 7f1f557fa700 1 -- 172.21.15.44:0/3869453245 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f1f5c023030 con 0x7f1f6413ac60 2020-11-20T12:08:26.251 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:08:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f3c056cd0 msgr2=0x7f1f3c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f3c056cd0 0x7f1f3c059190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f1f58001230 tx=0x7f1f58007700).stop 2020-11-20T12:08:26.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 msgr2=0x7f1f640675d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640675d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f1f64061280 tx=0x7f1f5c015000).stop 2020-11-20T12:08:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 shutdown_connections 2020-11-20T12:08:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1f3c056cd0 0x7f1f3c059190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f6413ac60 0x7f1f640675d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:26.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f640aa020 0x7f1f640abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 --2- 172.21.15.44:0/3869453245 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1f640aa400 0x7f1f640aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 >> 172.21.15.44:0/3869453245 conn(0x7f1f64062690 msgr2=0x7f1f640a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 shutdown_connections 2020-11-20T12:08:26.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:26.248+0000 7f1f6b54b700 1 -- 172.21.15.44:0/3869453245 wait complete. 2020-11-20T12:08:29.264 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:08:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.403+0000 7fc6d8c4e700 1 Processor -- start 2020-11-20T12:08:29.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.403+0000 7fc6d8c4e700 1 -- start start 2020-11-20T12:08:29.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d8c4e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d40aa020 0x7fc6d40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d8c4e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc6d40aa400 0x7fc6d40a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d8c4e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc6d413ac60 0x7fc6d40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d8c4e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d40a9770 con 0x7fc6d413ac60 2020-11-20T12:08:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d8c4e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d40a6da0 con 0x7fc6d40aa400 2020-11-20T12:08:29.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d8c4e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40a7040 con 0x7fc6d40aa020 2020-11-20T12:08:29.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d1d9b700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7fc6d40aa400 msgr2=0x7fc6d40a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:29.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d1d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc6d40aa400 0x7fc6d40a4fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:29.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc6d413ac60 0x7fc6d40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:29.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc6d413ac60 0x7fc6d40a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38910/0 (socket says 172.21.15.44:38910) 2020-11-20T12:08:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 -- 172.21.15.44:0/1297177614 learned_addr learned my addr 172.21.15.44:0/1297177614 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d259c700 1 --2- 172.21.15.44:0/1297177614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d40aa020 0x7fc6d40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 -- 172.21.15.44:0/1297177614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d40aa020 msgr2=0x7fc6d40a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 --2- 172.21.15.44:0/1297177614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d40aa020 0x7fc6d40a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:29.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 -- 172.21.15.44:0/1297177614 >> v2:172.21.15.79:3300/0 conn(0x7fc6d40aa400 msgr2=0x7fc6d40a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:29.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 --2- 172.21.15.44:0/1297177614 >> v2:172.21.15.79:3300/0 conn(0x7fc6d40aa400 0x7fc6d40a4fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:29.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.404+0000 7fc6d2d9d700 1 -- 172.21.15.44:0/1297177614 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d40a6a90 con 0x7fc6d413ac60 2020-11-20T12:08:29.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d2d9d700 1 --2- 172.21.15.44:0/1297177614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d413ac60 0x7fc6d40a3700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fc6d4062b30 tx=0x7fc6cc006eb0).ready entity=mon.0 client_cookie=1ae6a3374c30f1ca server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:29.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d159a700 1 -- 172.21.15.44:0/1297177614 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc009810 con 0x7fc6d413ac60 2020-11-20T12:08:29.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d159a700 1 -- 172.21.15.44:0/1297177614 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc009970 con 0x7fc6d413ac60 2020-11-20T12:08:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d159a700 1 -- 172.21.15.44:0/1297177614 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc009cf0 con 0x7fc6d413ac60 2020-11-20T12:08:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1297177614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d413ac60 msgr2=0x7fc6d40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1297177614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d413ac60 0x7fc6d40a3700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fc6d4062b30 tx=0x7fc6cc006eb0).stop 2020-11-20T12:08:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1297177614 shutdown_connections 2020-11-20T12:08:29.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1297177614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d413ac60 0x7fc6d40a3700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1297177614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d40aa020 0x7fc6d40a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1297177614 >> v2:172.21.15.79:3300/0 conn(0x7fc6d40aa400 0x7fc6d40a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1297177614 >> 172.21.15.44:0/1297177614 conn(0x7fc6d4062690 msgr2=0x7fc6d40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:29.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1297177614 shutdown_connections 2020-11-20T12:08:29.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.405+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1297177614 wait complete. 2020-11-20T12:08:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.406+0000 7fc6d8c4e700 1 Processor -- start 2020-11-20T12:08:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.406+0000 7fc6d8c4e700 1 -- start start 2020-11-20T12:08:29.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d8c4e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6d40aa020 0x7fc6d40acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d8c4e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d40aa400 0x7fc6d409f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d8c4e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d413ac60 0x7fc6d409ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d8c4e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40a0430 con 0x7fc6d40aa400 2020-11-20T12:08:29.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d8c4e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d409fcd0 con 0x7fc6d40aa020 2020-11-20T12:08:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d8c4e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d409fef0 con 0x7fc6d413ac60 2020-11-20T12:08:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d40aa400 0x7fc6d409f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d259c700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6d40aa020 msgr2=0x7fc6d40acf70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d40aa400 0x7fc6d409f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38916/0 (socket says 172.21.15.44:38916) 2020-11-20T12:08:29.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 -- 172.21.15.44:0/1511481635 learned_addr learned my addr 172.21.15.44:0/1511481635 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:29.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d259c700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6d40aa020 0x7fc6d40acf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:29.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d2d9d700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d413ac60 0x7fc6d409ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:29.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 -- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d413ac60 msgr2=0x7fc6d409ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:29.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d413ac60 0x7fc6d409ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:29.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 -- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6d40aa020 msgr2=0x7fc6d40acf70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:29.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6d40aa020 0x7fc6d40acf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:29.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 -- 172.21.15.44:0/1511481635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d40a0190 con 0x7fc6d40aa400 2020-11-20T12:08:29.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.407+0000 7fc6d1d9b700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d40aa400 0x7fc6d409f7e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc6c400bab0 tx=0x7fc6c4009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:29.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.408+0000 7fc6ca7fc700 1 -- 172.21.15.44:0/1511481635 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4003c50 con 0x7fc6d40aa400 2020-11-20T12:08:29.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.408+0000 7fc6ca7fc700 1 -- 172.21.15.44:0/1511481635 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c4003db0 con 0x7fc6d40aa400 2020-11-20T12:08:29.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.408+0000 7fc6ca7fc700 1 -- 172.21.15.44:0/1511481635 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4006170 con 0x7fc6d40aa400 2020-11-20T12:08:29.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.408+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d40a1c30 con 0x7fc6d40aa400 2020-11-20T12:08:29.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.408+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d409efc0 con 0x7fc6d40aa400 2020-11-20T12:08:29.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.409+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d4137470 con 0x7fc6d40aa400 2020-11-20T12:08:29.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.409+0000 7fc6ca7fc700 1 -- 172.21.15.44:0/1511481635 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc6c40064e0 con 0x7fc6d40aa400 2020-11-20T12:08:29.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.409+0000 7fc6ca7fc700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc6b0056cd0 0x7fc6b0059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:29.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.409+0000 7fc6ca7fc700 1 -- 172.21.15.44:0/1511481635 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc6c4011020 con 0x7fc6d40aa400 2020-11-20T12:08:29.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.409+0000 7fc6d259c700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc6b0056cd0 0x7fc6b0059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:29.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.410+0000 7fc6d259c700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc6b0056cd0 0x7fc6b0059190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fc6bc006fd0 tx=0x7fc6bc006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:29.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.413+0000 7fc6ca7fc700 1 -- 172.21.15.44:0/1511481635 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc6c4044bc0 con 0x7fc6d40aa400 2020-11-20T12:08:29.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.656+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc6d40446d0 con 0x7fc6d40aa400 2020-11-20T12:08:29.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.657+0000 7fc6ca7fc700 1 -- 172.21.15.44:0/1511481635 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fc6c40481e0 con 0x7fc6d40aa400 2020-11-20T12:08:29.661 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:08:29.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.658+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc6b0056cd0 msgr2=0x7fc6b0059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:29.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.658+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc6b0056cd0 0x7fc6b0059190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fc6bc006fd0 tx=0x7fc6bc006df0).stop 2020-11-20T12:08:29.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.658+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d40aa400 msgr2=0x7fc6d409f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:29.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.658+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d40aa400 0x7fc6d409f7e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc6c400bab0 tx=0x7fc6c4009220).stop 2020-11-20T12:08:29.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 shutdown_connections 2020-11-20T12:08:29.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc6b0056cd0 0x7fc6b0059190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:29.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc6d413ac60 0x7fc6d409ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:29.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc6d40aa020 0x7fc6d40acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:29.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 --2- 172.21.15.44:0/1511481635 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6d40aa400 0x7fc6d409f7e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:29.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 >> 172.21.15.44:0/1511481635 conn(0x7fc6d4062690 msgr2=0x7fc6d4136ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:29.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 shutdown_connections 2020-11-20T12:08:29.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:29.659+0000 7fc6d8c4e700 1 -- 172.21.15.44:0/1511481635 wait complete. 2020-11-20T12:08:32.674 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:08:32.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.810+0000 7f92a6874700 1 Processor -- start 2020-11-20T12:08:32.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.810+0000 7f92a6874700 1 -- start start 2020-11-20T12:08:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f92a6874700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f92a00a7810 0x7f92a013c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f92a6874700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a00a8bf0 con 0x7f92a0066250 2020-11-20T12:08:32.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f92a6874700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a00a86b0 con 0x7f92a00aa1e0 2020-11-20T12:08:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f92a6874700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a00a8950 con 0x7f92a00a7810 2020-11-20T12:08:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f9297fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f92a00a7810 0x7f92a013c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f92a4e11700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f92a0066250 0x7f92a013c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42674/0 (socket says 172.21.15.44:42674) 2020-11-20T12:08:32.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f929ffff700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f92a00aa1e0 legacy=0x7f92a0065b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:08:32.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f92a4e11700 1 -- 172.21.15.44:0/4167587515 learned_addr learned my addr 172.21.15.44:0/4167587515 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:32.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f9297fff700 1 -- 172.21.15.44:0/4167587515 >> v1:172.21.15.79:6789/0 conn(0x7f92a00aa1e0 legacy=0x7f92a0065b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:32.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f9297fff700 1 -- 172.21.15.44:0/4167587515 >> v1:172.21.15.44:6789/0 conn(0x7f92a0066250 legacy=0x7f92a013c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:32.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f9297fff700 1 -- 172.21.15.44:0/4167587515 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a00aa5c0 con 0x7f92a00a7810 2020-11-20T12:08:32.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.811+0000 7f9297fff700 1 --2- 172.21.15.44:0/4167587515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a00a7810 0x7f92a013c170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f92a00a8880 tx=0x7f9288006df0).ready entity=mon.0 client_cookie=b121fb761782d4fe server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:32.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f929f7fe700 1 -- 172.21.15.44:0/4167587515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288009770 con 0x7f92a00a7810 2020-11-20T12:08:32.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f929f7fe700 1 -- 172.21.15.44:0/4167587515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92880098d0 con 0x7f92a00a7810 2020-11-20T12:08:32.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f929f7fe700 1 -- 172.21.15.44:0/4167587515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288009c50 con 0x7f92a00a7810 2020-11-20T12:08:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f92a6874700 1 -- 172.21.15.44:0/4167587515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a00a7810 msgr2=0x7f92a013c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f92a6874700 1 --2- 172.21.15.44:0/4167587515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a00a7810 0x7f92a013c170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f92a00a8880 tx=0x7f9288006df0).stop 2020-11-20T12:08:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f92a6874700 1 -- 172.21.15.44:0/4167587515 shutdown_connections 2020-11-20T12:08:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f92a6874700 1 --2- 172.21.15.44:0/4167587515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a00a7810 0x7f92a013c170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:32.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f92a6874700 1 -- 172.21.15.44:0/4167587515 >> 172.21.15.44:0/4167587515 conn(0x7f92a0062690 msgr2=0x7f92a00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:32.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f92a6874700 1 -- 172.21.15.44:0/4167587515 shutdown_connections 2020-11-20T12:08:32.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.812+0000 7f92a6874700 1 -- 172.21.15.44:0/4167587515 wait complete. 2020-11-20T12:08:32.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.813+0000 7f92a6874700 1 Processor -- start 2020-11-20T12:08:32.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.813+0000 7f92a6874700 1 -- start start 2020-11-20T12:08:32.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a6874700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a0066250 0x7f92a01486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a6874700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92a00a7810 0x7f92a00a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a6874700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92a00aa1e0 0x7f92a0147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a6874700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a00a1b90 con 0x7f92a0066250 2020-11-20T12:08:32.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a6874700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a00a1e30 con 0x7f92a00aa1e0 2020-11-20T12:08:32.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a6874700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a00a20d0 con 0x7f92a00a7810 2020-11-20T12:08:32.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a0066250 0x7f92a01486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:32.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a0066250 0x7f92a01486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38930/0 (socket says 172.21.15.44:38930) 2020-11-20T12:08:32.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 -- 172.21.15.44:0/3679698294 learned_addr learned my addr 172.21.15.44:0/3679698294 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:32.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a4e11700 1 -- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92a00aa1e0 msgr2=0x7f92a0147d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:32.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a4e11700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92a00aa1e0 0x7f92a0147d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:32.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f9297fff700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92a00a7810 0x7f92a00a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 -- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92a00a7810 msgr2=0x7f92a00a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92a00a7810 0x7f92a00a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:32.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 -- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92a00aa1e0 msgr2=0x7f92a0147d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92a00aa1e0 0x7f92a0147d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 -- 172.21.15.44:0/3679698294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a0139ae0 con 0x7f92a0066250 2020-11-20T12:08:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f9297fff700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92a00a7810 0x7f92a00a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:08:32.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929ffff700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a0066250 0x7f92a01486d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f92900097c0 tx=0x7f9290006e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929cff9700 1 -- 172.21.15.44:0/3679698294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290006170 con 0x7f92a0066250 2020-11-20T12:08:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929cff9700 1 -- 172.21.15.44:0/3679698294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92900062f0 con 0x7f92a0066250 2020-11-20T12:08:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a00a0f30 con 0x7f92a0066250 2020-11-20T12:08:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.814+0000 7f929cff9700 1 -- 172.21.15.44:0/3679698294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92900038e0 con 0x7f92a0066250 2020-11-20T12:08:32.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.815+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a00a3500 con 0x7f92a0066250 2020-11-20T12:08:32.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.815+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a0061650 con 0x7f92a0066250 2020-11-20T12:08:32.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.816+0000 7f929cff9700 1 -- 172.21.15.44:0/3679698294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9290017030 con 0x7f92a0066250 2020-11-20T12:08:32.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.816+0000 7f929cff9700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f927c056cd0 0x7f927c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.816+0000 7f929cff9700 1 -- 172.21.15.44:0/3679698294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f929006f470 con 0x7f92a0066250 2020-11-20T12:08:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.816+0000 7f9297fff700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f927c056cd0 0x7f927c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.817+0000 7f9297fff700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f927c056cd0 0x7f927c059190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f92a0067be0 tx=0x7f9288015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:32.820+0000 7f929cff9700 1 -- 172.21.15.44:0/3679698294 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9290044740 con 0x7f92a0066250 2020-11-20T12:08:33.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.064+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f92a00446d0 con 0x7f92a0066250 2020-11-20T12:08:33.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.064+0000 7f929cff9700 1 -- 172.21.15.44:0/3679698294 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f9290015080 con 0x7f92a0066250 2020-11-20T12:08:33.069 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:08:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f927c056cd0 msgr2=0x7f927c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f927c056cd0 0x7f927c059190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f92a0067be0 tx=0x7f9288015000).stop 2020-11-20T12:08:33.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a0066250 msgr2=0x7f92a01486d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:33.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a0066250 0x7f92a01486d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f92900097c0 tx=0x7f9290006e60).stop 2020-11-20T12:08:33.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 shutdown_connections 2020-11-20T12:08:33.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f927c056cd0 0x7f927c059190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:33.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92a00aa1e0 0x7f92a0147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92a0066250 0x7f92a01486d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 --2- 172.21.15.44:0/3679698294 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f92a00a7810 0x7f92a00a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 >> 172.21.15.44:0/3679698294 conn(0x7f92a0062690 msgr2=0x7f92a013c560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 shutdown_connections 2020-11-20T12:08:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:33.066+0000 7f92a6874700 1 -- 172.21.15.44:0/3679698294 wait complete. 2020-11-20T12:08:36.082 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:08:36.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f167dd0f700 1 Processor -- start 2020-11-20T12:08:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f167dd0f700 1 -- start start 2020-11-20T12:08:36.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f167dd0f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:36.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f167dd0f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 0x7f1678066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f167dd0f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780a8bf0 con 0x7f16780a7810 2020-11-20T12:08:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f167dd0f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1678066520 con 0x7f16780aa1e0 2020-11-20T12:08:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f167dd0f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780a86b0 con 0x7f167813c880 2020-11-20T12:08:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f16777fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f16777fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38940/0 (socket says 172.21.15.44:38940) 2020-11-20T12:08:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f1677fff700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 msgr2=0x7f1678066030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f1676ffd700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f16780a7810 legacy=0x7f167813c170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:08:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 learned_addr learned my addr 172.21.15.44:0/2163892642 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.220+0000 7f1677fff700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 0x7f1678066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.221+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.221+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.221+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.221+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.421+0000 7f1676ffd700 1 -- 172.21.15.44:0/2163892642 >> v1:172.21.15.79:6789/0 conn(0x7f16780a7810 legacy=0x7f167813c170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:08:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.421+0000 7f1677fff700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 msgr2=0x7f1678066030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:36.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.421+0000 7f1677fff700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 0x7f1678066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:08:36.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.422+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:36.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.422+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.422+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.422+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:36.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.422+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:08:36.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.822+0000 7f1676ffd700 1 -- 172.21.15.44:0/2163892642 >> v1:172.21.15.79:6789/0 conn(0x7f16780a7810 legacy=0x7f167813c170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:08:36.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.822+0000 7f1677fff700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 msgr2=0x7f1678066030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:36.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.822+0000 7f1677fff700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 0x7f1678066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:08:36.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.823+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:36.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.823+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:36.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.823+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:36.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.823+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:36.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:36.823+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:08:36.921 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:08:36.924 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:08:37.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.623+0000 7f1677fff700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 msgr2=0x7f1678066030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:37.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.623+0000 7f1676ffd700 1 -- 172.21.15.44:0/2163892642 >> v1:172.21.15.79:6789/0 conn(0x7f16780a7810 legacy=0x7f167813c170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:08:37.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.623+0000 7f1677fff700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 0x7f1678066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:08:37.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.624+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:37.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.624+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:37.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.624+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:37.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.624+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:37.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:37.624+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:08:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 msgr2=0x7f1678066030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.79:3300/0 conn(0x7f167813c880 0x7f1678066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:39.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 -- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 msgr2=0x7f1678065b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 --2- 172.21.15.44:0/2163892642 >> v2:172.21.15.44:3300/0 conn(0x7f16780aa1e0 0x7f1678065b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 -- 172.21.15.44:0/2163892642 >> v1:172.21.15.79:6789/0 conn(0x7f16780a7810 legacy=0x7f167813c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:39.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 -- 172.21.15.44:0/2163892642 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780a8950 con 0x7f1658005e50 2020-11-20T12:08:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.220+0000 7f16757fa700 1 -- 172.21.15.44:0/2163892642 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1678066520 con 0x7f1658003380 2020-11-20T12:08:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.221+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.221+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:39.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.221+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.221+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.221+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.422+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:39.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.422+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.422+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.422+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:39.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.422+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:08:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.823+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.823+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.823+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.823+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:39.823+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:08:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.624+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.624+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:08:40.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.624+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:08:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.624+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:08:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.625+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:08:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.718+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108586323 0 0) 0x7f16780a8950 con 0x7f1658005e50 2020-11-20T12:08:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.718+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1664003240 con 0x7f1658005e50 2020-11-20T12:08:40.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3873593904 0 0) 0x7f1664003240 con 0x7f1658005e50 2020-11-20T12:08:40.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 msgr2=0x7f16580057d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:40.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1658003380 0x7f16580057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780a8bf0 con 0x7f1658005e50 2020-11-20T12:08:40.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f166c003d80 con 0x7f1658005e50 2020-11-20T12:08:40.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 >> v1:172.21.15.44:6789/0 conn(0x7f1658005e50 legacy=0x7f16580082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f1677fff700 1 -- 172.21.15.44:0/2163892642 reap_dead start 2020-11-20T12:08:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664003730 0x7f1664003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f1664004520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664004bc0 0x7f166400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:40.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678066520 con 0x7f1664003730 2020-11-20T12:08:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1664005260 con 0x7f1664004bc0 2020-11-20T12:08:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166400bc20 con 0x7f1664004040 2020-11-20T12:08:40.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664003730 0x7f1664003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f1677fff700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664004bc0 msgr2=0x7f166400b730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.719+0000 7f1677fff700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664004bc0 0x7f166400b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.720+0000 7f1676ffd700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f1664004520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.720+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 msgr2=0x7f1664004520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.720+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f1664004520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.720+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664004bc0 msgr2=0x7f166400b730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.720+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664004bc0 0x7f166400b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:40.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.720+0000 7f16777fe700 1 -- 172.21.15.44:0/2163892642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664003240 con 0x7f1664003730 2020-11-20T12:08:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.720+0000 7f16777fe700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664003730 0x7f1664003b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f1668019cd0 tx=0x7f1668008040).ready entity=mon.0 client_cookie=72b91192dd7f0b6a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166801e070 con 0x7f1664003730 2020-11-20T12:08:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668017db0 con 0x7f1664003730 2020-11-20T12:08:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f16767fc700 1 -- 172.21.15.44:0/2163892642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668023620 con 0x7f1664003730 2020-11-20T12:08:40.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f167dd0f700 1 -- 172.21.15.44:0/2163892642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664003730 msgr2=0x7f1664003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:40.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f167dd0f700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664003730 0x7f1664003b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f1668019cd0 tx=0x7f1668008040).stop 2020-11-20T12:08:40.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f167dd0f700 1 -- 172.21.15.44:0/2163892642 shutdown_connections 2020-11-20T12:08:40.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f167dd0f700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664004bc0 0x7f166400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:40.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f167dd0f700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f1664004520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f167dd0f700 1 --2- 172.21.15.44:0/2163892642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664003730 0x7f1664003b30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.721+0000 7f167dd0f700 1 -- 172.21.15.44:0/2163892642 >> 172.21.15.44:0/2163892642 conn(0x7f1678062690 msgr2=0x7f16780a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.722+0000 7f167dd0f700 1 -- 172.21.15.44:0/2163892642 shutdown_connections 2020-11-20T12:08:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.722+0000 7f167dd0f700 1 -- 172.21.15.44:0/2163892642 wait complete. 2020-11-20T12:08:40.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.723+0000 7f167dd0f700 1 Processor -- start 2020-11-20T12:08:40.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.723+0000 7f167dd0f700 1 -- start start 2020-11-20T12:08:40.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f167dd0f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664003730 0x7f167813f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:40.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f167dd0f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f16781497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:40.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f167dd0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664004bc0 0x7f1678138020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:40.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f167dd0f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780a3280 con 0x7f1664004bc0 2020-11-20T12:08:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f167dd0f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167814d1d0 con 0x7f1664003730 2020-11-20T12:08:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f167dd0f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167814d470 con 0x7f1664004040 2020-11-20T12:08:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f16777fe700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664003730 msgr2=0x7f167813f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f16777fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664003730 0x7f167813f510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:40.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1676ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f16781497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1677fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664004bc0 0x7f1678138020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1676ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f16781497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50420/0 (socket says 172.21.15.44:50420) 2020-11-20T12:08:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1676ffd700 1 -- 172.21.15.44:0/3622924587 learned_addr learned my addr 172.21.15.44:0/3622924587 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:40.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1676ffd700 1 -- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664003730 msgr2=0x7f167813f510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1676ffd700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664003730 0x7f167813f510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1676ffd700 1 -- 172.21.15.44:0/3622924587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664004bc0 msgr2=0x7f1678138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.724+0000 7f1676ffd700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664004bc0 0x7f1678138020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.725+0000 7f1676ffd700 1 -- 172.21.15.44:0/3622924587 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167814d710 con 0x7f1664004040 2020-11-20T12:08:40.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.725+0000 7f1676ffd700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f16781497d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f166c010ec0 tx=0x7f166c007840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.725+0000 7f16577fe700 1 -- 172.21.15.44:0/3622924587 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00c380 con 0x7f1664004040 2020-11-20T12:08:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.725+0000 7f16577fe700 1 -- 172.21.15.44:0/3622924587 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c00c4e0 con 0x7f1664004040 2020-11-20T12:08:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.725+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167814dbb0 con 0x7f1664004040 2020-11-20T12:08:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.725+0000 7f16577fe700 1 -- 172.21.15.44:0/3622924587 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00c860 con 0x7f1664004040 2020-11-20T12:08:40.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.725+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167814cda0 con 0x7f1664004040 2020-11-20T12:08:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.726+0000 7f16577fe700 1 -- 172.21.15.44:0/3622924587 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f166c00cc10 con 0x7f1664004040 2020-11-20T12:08:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.726+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678062c70 con 0x7f1664004040 2020-11-20T12:08:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.727+0000 7f16577fe700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1650056cd0 0x7f1650059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.727+0000 7f16577fe700 1 -- 172.21.15.44:0/3622924587 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f166c071020 con 0x7f1664004040 2020-11-20T12:08:40.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.727+0000 7f16777fe700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1650056cd0 0x7f1650059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.727+0000 7f16777fe700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1650056cd0 0x7f1650059190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f16680196f0 tx=0x7f1668008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:40.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.731+0000 7f16577fe700 1 -- 172.21.15.44:0/3622924587 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f166c046290 con 0x7f1664004040 2020-11-20T12:08:40.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.976+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f167814d8a0 con 0x7f1664004040 2020-11-20T12:08:40.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.976+0000 7f16577fe700 1 -- 172.21.15.44:0/3622924587 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7f166c0498b0 con 0x7f1664004040 2020-11-20T12:08:40.981 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":54,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:08:40.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1650056cd0 msgr2=0x7f1650059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:40.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1650056cd0 0x7f1650059190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f16680196f0 tx=0x7f1668008040).stop 2020-11-20T12:08:40.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 msgr2=0x7f16781497d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:40.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f16781497d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f166c010ec0 tx=0x7f166c007840).stop 2020-11-20T12:08:40.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 shutdown_connections 2020-11-20T12:08:40.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1650056cd0 0x7f1650059190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1664004bc0 0x7f1678138020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1664003730 0x7f167813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:40.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 --2- 172.21.15.44:0/3622924587 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1664004040 0x7f16781497d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:40.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 >> 172.21.15.44:0/3622924587 conn(0x7f1678062690 msgr2=0x7f16780a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:40.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 shutdown_connections 2020-11-20T12:08:40.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:40.978+0000 7f167dd0f700 1 -- 172.21.15.44:0/3622924587 wait complete. 2020-11-20T12:08:40.993 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:08:40.994 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:08:41.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.119+0000 7ff5e83c2700 1 Processor -- start 2020-11-20T12:08:41.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.119+0000 7ff5e83c2700 1 -- start start 2020-11-20T12:08:41.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e83c2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff5e00aa020 0x7ff5e00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e83c2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff5e013ac60 0x7ff5e00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e83c2700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e00a9770 con 0x7ff5e00aa400 2020-11-20T12:08:41.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e83c2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e00a56c0 con 0x7ff5e00aa020 2020-11-20T12:08:41.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e83c2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e00a6da0 con 0x7ff5e013ac60 2020-11-20T12:08:41.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e695f700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7ff5e013ac60 msgr2=0x7ff5e00a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:41.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e695f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff5e013ac60 0x7ff5e00a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:41.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e615e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff5e00aa020 0x7ff5e00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e595d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff5e00aa400 0x7ff5e00a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42738/0 (socket says 172.21.15.44:42738) 2020-11-20T12:08:41.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e595d700 1 -- 172.21.15.44:0/3681679809 learned_addr learned my addr 172.21.15.44:0/3681679809 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:08:41.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e615e700 1 -- 172.21.15.44:0/3681679809 >> v2:172.21.15.79:3300/0 conn(0x7ff5e013ac60 msgr2=0x7ff5e00a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:41.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e615e700 1 --2- 172.21.15.44:0/3681679809 >> v2:172.21.15.79:3300/0 conn(0x7ff5e013ac60 0x7ff5e00a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:41.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e515c700 1 -- 172.21.15.44:0/3681679809 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656325825 0 0) 0x7ff5d00010d0 con 0x7ff5e00aa400 2020-11-20T12:08:41.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e615e700 1 -- 172.21.15.44:0/3681679809 >> v1:172.21.15.44:6789/0 conn(0x7ff5e00aa400 legacy=0x7ff5e00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e615e700 1 -- 172.21.15.44:0/3681679809 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00a7040 con 0x7ff5e00aa020 2020-11-20T12:08:41.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.120+0000 7ff5e615e700 1 --2- 172.21.15.44:0/3681679809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e00a9280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7ff5e00a8f70 tx=0x7ff5dc006df0).ready entity=mon.0 client_cookie=1621006d48c28081 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e515c700 1 -- 172.21.15.44:0/3681679809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc0097b0 con 0x7ff5e00aa020 2020-11-20T12:08:41.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e515c700 1 -- 172.21.15.44:0/3681679809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc009910 con 0x7ff5e00aa020 2020-11-20T12:08:41.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e515c700 1 -- 172.21.15.44:0/3681679809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc009c90 con 0x7ff5e00aa020 2020-11-20T12:08:41.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 -- 172.21.15.44:0/3681679809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 msgr2=0x7ff5e00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 --2- 172.21.15.44:0/3681679809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e00a9280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7ff5e00a8f70 tx=0x7ff5dc006df0).stop 2020-11-20T12:08:41.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 -- 172.21.15.44:0/3681679809 shutdown_connections 2020-11-20T12:08:41.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 --2- 172.21.15.44:0/3681679809 >> v2:172.21.15.79:3300/0 conn(0x7ff5e013ac60 0x7ff5e00a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:41.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 --2- 172.21.15.44:0/3681679809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e00a9280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 -- 172.21.15.44:0/3681679809 >> 172.21.15.44:0/3681679809 conn(0x7ff5e0062690 msgr2=0x7ff5e00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:41.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 -- 172.21.15.44:0/3681679809 shutdown_connections 2020-11-20T12:08:41.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.121+0000 7ff5e83c2700 1 -- 172.21.15.44:0/3681679809 wait complete. 2020-11-20T12:08:41.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.122+0000 7ff5e83c2700 1 Processor -- start 2020-11-20T12:08:41.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.122+0000 7ff5e83c2700 1 -- start start 2020-11-20T12:08:41.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e0145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5e00aa400 0x7ff5e01327a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5e013ac60 0x7ff5e0149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0146240 con 0x7ff5e00aa020 2020-11-20T12:08:41.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0067f50 con 0x7ff5e00aa400 2020-11-20T12:08:41.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0146460 con 0x7ff5e013ac60 2020-11-20T12:08:41.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e0145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e0145d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38994/0 (socket says 172.21.15.44:38994) 2020-11-20T12:08:41.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 -- 172.21.15.44:0/1546014028 learned_addr learned my addr 172.21.15.44:0/1546014028 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:41.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e595d700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5e00aa400 msgr2=0x7ff5e01327a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:41.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e595d700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5e00aa400 0x7ff5e01327a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:41.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e695f700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5e013ac60 0x7ff5e0149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5e013ac60 msgr2=0x7ff5e0149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5e013ac60 0x7ff5e0149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5e00aa400 msgr2=0x7ff5e01327a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5e00aa400 0x7ff5e01327a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:41.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 -- 172.21.15.44:0/1546014028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0146700 con 0x7ff5e00aa020 2020-11-20T12:08:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e695f700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5e013ac60 0x7ff5e0149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:08:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e615e700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e0145d50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff5dc006e30 tx=0x7ff5dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc01e460 con 0x7ff5e00aa020 2020-11-20T12:08:41.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc01e5c0 con 0x7ff5e00aa020 2020-11-20T12:08:41.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e01469a0 con 0x7ff5e00aa020 2020-11-20T12:08:41.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc01e940 con 0x7ff5e00aa020 2020-11-20T12:08:41.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.123+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e0149810 con 0x7ff5e00aa020 2020-11-20T12:08:41.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.124+0000 7ff5e83c2700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0149a60 0x7ff5e0149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.124+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00a37a0 con 0x7ff5e0149a60 2020-11-20T12:08:41.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.124+0000 7ff5e595d700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0149a60 0x7ff5e0149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff5dc025030 con 0x7ff5e00aa020 2020-11-20T12:08:41.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5e595d700 1 -- 172.21.15.44:0/1546014028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5e0061650 con 0x7ff5e0149a60 2020-11-20T12:08:41.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5e595d700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0149a60 0x7ff5e0149e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7ff5e0067ac0 tx=0x7ff5d00079d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5b8056cd0 0x7ff5b8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff5dc0718e0 con 0x7ff5e00aa020 2020-11-20T12:08:41.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00072e0 con 0x7ff5e0149a60 2020-11-20T12:08:41.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7ff5d0007440 con 0x7ff5e0149a60 2020-11-20T12:08:41.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0149a60 msgr2=0x7ff5e0149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0149a60 0x7ff5e0149e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7ff5e0067ac0 tx=0x7ff5d00079d0).stop 2020-11-20T12:08:41.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0149a60 msgr2=0x7ff5e0149e40 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:41.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5ce7fc700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0149a60 0x7ff5e0149e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.125+0000 7ff5e695f700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5b8056cd0 0x7ff5b8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.126+0000 7ff5e695f700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5b8056cd0 0x7ff5b8059190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff5d8006fd0 tx=0x7ff5d8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.133+0000 7ff5e83c2700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0065150 0x7ff5e0065560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.133+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00446d0 con 0x7ff5e0065150 2020-11-20T12:08:41.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.133+0000 7ff5e595d700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0065150 0x7ff5e0065560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.133+0000 7ff5e595d700 1 -- 172.21.15.44:0/1546014028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff5e0069560 con 0x7ff5e0065150 2020-11-20T12:08:41.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.133+0000 7ff5e595d700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0065150 0x7ff5e0065560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff5e005ef40 tx=0x7ff5d0013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.133+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0017070 con 0x7ff5e0065150 2020-11-20T12:08:41.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4766 (secure 0 0 0) 0x7ff5d00070d0 con 0x7ff5e0065150 2020-11-20T12:08:41.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0065150 msgr2=0x7ff5e0065560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5ce7fc700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0065150 0x7ff5e0065560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff5e005ef40 tx=0x7ff5d0013000).stop 2020-11-20T12:08:41.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5ce7fc700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0065150 msgr2=0x7ff5e0065560 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:41.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5ce7fc700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e0065150 0x7ff5e0065560 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0x7ff5e005ef40 tx=0x7ff5d0013000).stop 2020-11-20T12:08:41.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5b8056cd0 msgr2=0x7ff5b8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5b8056cd0 0x7ff5b8059190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff5d8006fd0 tx=0x7ff5d8008040).stop 2020-11-20T12:08:41.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 msgr2=0x7ff5e0145d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 --2- 172.21.15.44:0/1546014028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5e00aa020 0x7ff5e0145d50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff5dc006e30 tx=0x7ff5dc015000).stop 2020-11-20T12:08:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e695f700 1 -- 172.21.15.44:0/1546014028 reap_dead start 2020-11-20T12:08:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e695f700 1 -- 172.21.15.44:0/1546014028 reap_dead start 2020-11-20T12:08:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 shutdown_connections 2020-11-20T12:08:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 >> 172.21.15.44:0/1546014028 conn(0x7ff5e0062690 msgr2=0x7ff5e01311d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:41.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 shutdown_connections 2020-11-20T12:08:41.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.134+0000 7ff5e83c2700 1 -- 172.21.15.44:0/1546014028 wait complete. 2020-11-20T12:08:41.150 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:41.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:41.151 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:41.151 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:08:41.151 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 54, 2020-11-20T12:08:41.151 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:41.151 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:41.152 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:41.152 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.152 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:08:41.152 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:41.152 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:41.153 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:41.153 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:41.153 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:41.154 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:41.154 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:41.154 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:41.154 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:41.154 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:41.155 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:41.155 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.155 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:41.155 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:41.156 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:41.156 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:41.156 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:41.156 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:41.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:41.157 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:41.157 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:41.157 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:41.157 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.158 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:41.158 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:41.158 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:41.158 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:41.158 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:41.159 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:41.159 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:41.159 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:41.159 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:41.159 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:41.160 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:41.160 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:41.160 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:41.160 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:41.160 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:41.161 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:41.161 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:41.161 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:41.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:41.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:41.162 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:41.162 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:41.162 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:41.162 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.162 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:41.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.163 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:41.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.163 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:41.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:41.164 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:41.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:41.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:41.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:41.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:41.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:41.166 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.166 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:41.167 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:41.167 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:41.167 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:41.167 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:41.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.168 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:41.168 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:41.168 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:41.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:41.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:41.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:41.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:41.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:41.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.171 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:41.171 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:41.172 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:41.172 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:41.172 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:41.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.173 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:41.173 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:41.173 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:41.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:41.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:41.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:41.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:41.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:41.175 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.176 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:41.176 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:41.176 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:41.176 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:41.177 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:41.177 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.177 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.177 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:41.178 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:41.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.178 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.178 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.178 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:08:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:08:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.182 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:41.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.182 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.182 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.183 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:08:41.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.183 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.183 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:08:41.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.184 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.184 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.184 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:41.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.184 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.185 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:41.185 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:41.186 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:08:41.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.312+0000 7f2ead328700 1 Processor -- start 2020-11-20T12:08:41.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.312+0000 7f2ead328700 1 -- start start 2020-11-20T12:08:41.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.312+0000 7f2ead328700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2ea8060330 0x7f2ea80a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.312+0000 7f2ead328700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea8141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.312+0000 7f2ead328700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80a6260 con 0x7f2ea80a4c30 2020-11-20T12:08:41.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.312+0000 7f2ead328700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea80a52e0 con 0x7f2ea8060330 2020-11-20T12:08:41.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.312+0000 7f2ead328700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80a6490 con 0x7f2ea813b0a0 2020-11-20T12:08:41.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea67fc700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f2ea80a4c30 legacy=0x7f2ea813a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:08:41.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2ea8060330 0x7f2ea80a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2ea8060330 0x7f2ea80a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39006/0 (socket says 172.21.15.44:39006) 2020-11-20T12:08:41.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3652785230 learned_addr learned my addr 172.21.15.44:0/3652785230 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:41.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea77fe700 1 --2- 172.21.15.44:0/3652785230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea8141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3652785230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 msgr2=0x7f2ea8141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:08:41.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 --2- 172.21.15.44:0/3652785230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea8141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3652785230 >> v1:172.21.15.79:6789/0 conn(0x7f2ea80a4c30 legacy=0x7f2ea813a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:08:41.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3652785230 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80a6730 con 0x7f2ea8060330 2020-11-20T12:08:41.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea77fe700 1 --2- 172.21.15.44:0/3652785230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea8141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:08:41.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea6ffd700 1 --2- 172.21.15.44:0/3652785230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80a4740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f2ea0009d50 tx=0x7f2ea0009290).ready entity=mon.0 client_cookie=c9bea370dec17118 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea5ffb700 1 -- 172.21.15.44:0/3652785230 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000bc40 con 0x7f2ea8060330 2020-11-20T12:08:41.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea5ffb700 1 -- 172.21.15.44:0/3652785230 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea000bda0 con 0x7f2ea8060330 2020-11-20T12:08:41.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.313+0000 7f2ea5ffb700 1 -- 172.21.15.44:0/3652785230 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0011610 con 0x7f2ea8060330 2020-11-20T12:08:41.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 -- 172.21.15.44:0/3652785230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 msgr2=0x7f2ea80a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 --2- 172.21.15.44:0/3652785230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80a4740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f2ea0009d50 tx=0x7f2ea0009290).stop 2020-11-20T12:08:41.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 -- 172.21.15.44:0/3652785230 shutdown_connections 2020-11-20T12:08:41.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 --2- 172.21.15.44:0/3652785230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea8141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 --2- 172.21.15.44:0/3652785230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80a4740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 -- 172.21.15.44:0/3652785230 >> 172.21.15.44:0/3652785230 conn(0x7f2ea8062690 msgr2=0x7f2ea80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:41.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 -- 172.21.15.44:0/3652785230 shutdown_connections 2020-11-20T12:08:41.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.314+0000 7f2ead328700 1 -- 172.21.15.44:0/3652785230 wait complete. 2020-11-20T12:08:41.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 Processor -- start 2020-11-20T12:08:41.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 -- start start 2020-11-20T12:08:41.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ea80a4c30 0x7f2ea81334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea81339a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea812fb90 con 0x7f2ea8060330 2020-11-20T12:08:41.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea812fe30 con 0x7f2ea80a4c30 2020-11-20T12:08:41.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.315+0000 7f2ead328700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81300d0 con 0x7f2ea813b0a0 2020-11-20T12:08:41.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea67fc700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ea80a4c30 msgr2=0x7f2ea81334b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:08:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea67fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ea80a4c30 0x7f2ea81334b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:08:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39010/0 (socket says 172.21.15.44:39010) 2020-11-20T12:08:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3389944619 learned_addr learned my addr 172.21.15.44:0/3389944619 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:08:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea77fe700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea81339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 msgr2=0x7f2ea81339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea813b0a0 0x7f2ea81339a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ea80a4c30 msgr2=0x7f2ea81334b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:08:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ea80a4c30 0x7f2ea81334b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:08:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 -- 172.21.15.44:0/3389944619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea812f810 con 0x7f2ea8060330 2020-11-20T12:08:41.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ea6ffd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80ad020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f2ea80aa8b0 tx=0x7f2ea0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea001b6b0 con 0x7f2ea8060330 2020-11-20T12:08:41.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea001b810 con 0x7f2ea8060330 2020-11-20T12:08:41.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea001bb90 con 0x7f2ea8060330 2020-11-20T12:08:41.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea80adad0 con 0x7f2ea8060330 2020-11-20T12:08:41.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.316+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea8132fb0 con 0x7f2ea8060330 2020-11-20T12:08:41.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.317+0000 7f2ead328700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea812d010 0x7f2ea812d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.317+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8130290 con 0x7f2ea812d010 2020-11-20T12:08:41.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.317+0000 7f2ea67fc700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea812d010 0x7f2ea812d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.317+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2ea0023030 con 0x7f2ea8060330 2020-11-20T12:08:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2ea67fc700 1 -- 172.21.15.44:0/3389944619 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ea8133320 con 0x7f2ea812d010 2020-11-20T12:08:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d00 0x7f2e840591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2ea0071500 con 0x7f2ea8060330 2020-11-20T12:08:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2ea67fc700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea812d010 0x7f2ea812d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2ea81340b0 tx=0x7f2e9c006e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2ea77fe700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d00 0x7f2e840591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0050d0 con 0x7f2ea812d010 2020-11-20T12:08:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f2e9c005230 con 0x7f2ea812d010 2020-11-20T12:08:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea812d010 msgr2=0x7f2ea812d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2ea77fe700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d00 0x7f2e840591c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f2e98006bd0 tx=0x7f2e98008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea812d010 0x7f2ea812d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2ea81340b0 tx=0x7f2e9c006e60).stop 2020-11-20T12:08:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea812d010 msgr2=0x7f2ea812d3f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.318+0000 7f2e8effd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea812d010 0x7f2ea812d3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.326+0000 7f2ead328700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea81370c0 0x7f2ea81374d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:08:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.326+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea812fcd0 con 0x7f2ea81370c0 2020-11-20T12:08:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.326+0000 7f2ea67fc700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea81370c0 0x7f2ea81374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:08:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2ea67fc700 1 -- 172.21.15.44:0/3389944619 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2ea8061650 con 0x7f2ea81370c0 2020-11-20T12:08:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2ea67fc700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea81370c0 0x7f2ea81374d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f2e9c005000 tx=0x7f2e9c013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:08:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c015c60 con 0x7f2ea81370c0 2020-11-20T12:08:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4764 (secure 0 0 0) 0x7f2e9c003720 con 0x7f2ea81370c0 2020-11-20T12:08:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea81370c0 msgr2=0x7f2ea81374d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2e8effd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea81370c0 0x7f2ea81374d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f2e9c005000 tx=0x7f2e9c013000).stop 2020-11-20T12:08:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2e8effd700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea81370c0 msgr2=0x7f2ea81374d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:08:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2e8effd700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2ea81370c0 0x7f2ea81374d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:08:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d00 msgr2=0x7f2e840591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2ead328700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2e84056d00 0x7f2e840591c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f2e98006bd0 tx=0x7f2e98008040).stop 2020-11-20T12:08:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 msgr2=0x7f2ea80ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:08:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2ead328700 1 --2- 172.21.15.44:0/3389944619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ea8060330 0x7f2ea80ad020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f2ea80aa8b0 tx=0x7f2ea0016000).stop 2020-11-20T12:08:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.327+0000 7f2ea77fe700 1 -- 172.21.15.44:0/3389944619 reap_dead start 2020-11-20T12:08:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.328+0000 7f2ea77fe700 1 -- 172.21.15.44:0/3389944619 reap_dead start 2020-11-20T12:08:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.328+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 shutdown_connections 2020-11-20T12:08:41.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.328+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 >> 172.21.15.44:0/3389944619 conn(0x7f2ea8062690 msgr2=0x7f2ea8134310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:08:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.328+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 shutdown_connections 2020-11-20T12:08:41.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:08:41.328+0000 7f2ead328700 1 -- 172.21.15.44:0/3389944619 wait complete. 2020-11-20T12:08:41.341 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:08:41.341 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:41.341 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:41.342 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:08:41.342 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 54, 2020-11-20T12:08:41.342 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:08:41.342 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:08:41.343 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:08:41.343 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.343 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:08:41.343 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:41.343 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:08:41.344 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:08:41.344 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:41.344 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:41.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:41.344 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:41.345 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:41.345 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:41.345 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:41.345 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:41.345 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.346 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:08:41.346 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:08:41.346 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:41.346 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:41.346 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:41.347 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:41.347 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:41.347 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:41.347 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:41.347 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:41.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.348 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:08:41.348 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:08:41.348 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:08:41.349 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:08:41.349 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:08:41.349 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:08:41.349 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:41.349 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:08:41.350 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:08:41.350 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:08:41.350 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:08:41.350 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:08:41.350 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:08:41.351 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:08:41.351 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:08:41.351 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:08:41.351 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:08:41.351 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:08:41.352 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:08:41.352 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:08:41.352 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:08:41.352 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:08:41.352 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:08:41.353 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.353 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:08:41.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.353 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:08:41.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.354 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:08:41.354 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:08:41.354 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:41.354 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:41.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:41.355 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:08:41.355 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.356 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:41.356 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:08:41.356 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.356 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.357 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:08:41.357 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:08:41.357 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:41.357 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:41.358 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:41.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:08:41.358 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:08:41.359 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:41.359 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:41.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.359 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:41.359 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:08:41.360 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.360 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:41.360 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:08:41.361 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.361 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.361 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.361 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:08:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:08:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:08:41.363 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:08:41.363 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:08:41.363 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:08:41.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:08:41.364 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:08:41.364 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:08:41.365 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:08:41.365 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:08:41.365 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.365 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.366 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:08:41.366 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:08:41.366 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:08:41.366 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:08:41.367 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:08:41.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.367 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.367 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:08:41.368 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:08:41.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.368 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.368 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.368 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:41.369 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.369 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.369 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:08:41.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.369 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.370 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.370 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:41.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.370 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.370 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:08:41.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.371 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.371 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.371 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:08:41.371 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.372 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.372 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:08:41.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.372 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.372 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.373 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:08:41.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.373 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:08:41.373 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:08:41.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:08:41.374 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:08:41.374 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:08:41.374 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:08:41.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:08:41.374 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:08:41.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:08:41.375 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:08:41.375 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:08:41.376 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:09:01.377 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:09:01.378 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:09:01.378 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:09:01.378 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:09:01.387 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:09:01.388 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:09:01.389 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:01.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.529+0000 7f3e8ddf6700 1 Processor -- start 2020-11-20T12:09:01.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e8ddf6700 1 -- start start 2020-11-20T12:09:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e8ddf6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e880aa400 0x7f3e880a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e8ddf6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 0x7f3e880a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:01.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e8ddf6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e880a9770 con 0x7f3e880aa020 2020-11-20T12:09:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e8ddf6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e880a56c0 con 0x7f3e8813ac60 2020-11-20T12:09:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e8ddf6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880a6da0 con 0x7f3e880aa400 2020-11-20T12:09:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e880aa400 0x7f3e880a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.530+0000 7f3e87fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 0x7f3e880a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:01.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e877fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3e880aa020 0x7f3e880a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:45648/0 (socket says 172.21.15.44:45648) 2020-11-20T12:09:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e880aa400 0x7f3e880a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50464/0 (socket says 172.21.15.44:50464) 2020-11-20T12:09:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e877fe700 1 -- 172.21.15.44:0/1868181214 learned_addr learned my addr 172.21.15.44:0/1868181214 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e87fff700 1 -- 172.21.15.44:0/1868181214 >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 msgr2=0x7f3e880a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:09:01.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e87fff700 1 -- 172.21.15.44:0/1868181214 >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 msgr2=0x7f3e880a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:01.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e87fff700 1 --2- 172.21.15.44:0/1868181214 >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 0x7f3e880a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:01.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e87fff700 1 --2- 172.21.15.44:0/1868181214 >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 0x7f3e880a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:01.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e86ffd700 1 -- 172.21.15.44:0/1868181214 >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 msgr2=0x7f3e880a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:01.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e86ffd700 1 --2- 172.21.15.44:0/1868181214 >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 0x7f3e880a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e86ffd700 1 -- 172.21.15.44:0/1868181214 >> v1:172.21.15.79:6789/0 conn(0x7f3e880aa020 legacy=0x7f3e880a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:01.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e86ffd700 1 -- 172.21.15.44:0/1868181214 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880a7040 con 0x7f3e880aa400 2020-11-20T12:09:01.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e86ffd700 1 --2- 172.21.15.44:0/1868181214 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e880aa400 0x7f3e880a9280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f3e7c00b9a0 tx=0x7f3e7c006df0).ready entity=mon.2 client_cookie=3aa3ee55565964aa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:01.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e867fc700 1 -- 172.21.15.44:0/1868181214 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c0097b0 con 0x7f3e880aa400 2020-11-20T12:09:01.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e867fc700 1 -- 172.21.15.44:0/1868181214 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c009910 con 0x7f3e880aa400 2020-11-20T12:09:01.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.531+0000 7f3e867fc700 1 -- 172.21.15.44:0/1868181214 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c009c90 con 0x7f3e880aa400 2020-11-20T12:09:01.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1868181214 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e880aa400 msgr2=0x7f3e880a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:01.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1868181214 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e880aa400 0x7f3e880a9280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f3e7c00b9a0 tx=0x7f3e7c006df0).stop 2020-11-20T12:09:01.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1868181214 shutdown_connections 2020-11-20T12:09:01.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1868181214 >> v2:172.21.15.79:3300/0 conn(0x7f3e8813ac60 0x7f3e880a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1868181214 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e880aa400 0x7f3e880a9280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1868181214 >> 172.21.15.44:0/1868181214 conn(0x7f3e88062690 msgr2=0x7f3e880a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:01.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1868181214 shutdown_connections 2020-11-20T12:09:01.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.532+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1868181214 wait complete. 2020-11-20T12:09:01.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 Processor -- start 2020-11-20T12:09:01.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 -- start start 2020-11-20T12:09:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e880aa020 0x7f3e88067c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 0x7f3e880aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e8813ac60 0x7f3e880650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:01.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880a2f60 con 0x7f3e880aa020 2020-11-20T12:09:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880a3200 con 0x7f3e880aa400 2020-11-20T12:09:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.533+0000 7f3e8ddf6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880a34a0 con 0x7f3e8813ac60 2020-11-20T12:09:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e880aa020 0x7f3e88067c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e880aa020 0x7f3e88067c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39030/0 (socket says 172.21.15.44:39030) 2020-11-20T12:09:01.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 0x7f3e880aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 -- 172.21.15.44:0/1331789405 learned_addr learned my addr 172.21.15.44:0/1331789405 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e87fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e8813ac60 0x7f3e880650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e86ffd700 1 -- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 msgr2=0x7f3e880aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:09:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e86ffd700 1 -- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 msgr2=0x7f3e880aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:09:01.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e86ffd700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 0x7f3e880aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e86ffd700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 0x7f3e880aca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 -- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e8813ac60 msgr2=0x7f3e880650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e8813ac60 0x7f3e880650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 -- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 msgr2=0x7f3e880aca20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:01.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 0x7f3e880aca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 -- 172.21.15.44:0/1331789405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880a2c20 con 0x7f3e880aa020 2020-11-20T12:09:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e87fff700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e8813ac60 0x7f3e880650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e877fe700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e880aa020 0x7f3e88067c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3e7800c710 tx=0x7f3e78007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e677fe700 1 -- 172.21.15.44:0/1331789405 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78007320 con 0x7f3e880aa020 2020-11-20T12:09:01.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e677fe700 1 -- 172.21.15.44:0/1331789405 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e78007480 con 0x7f3e880aa020 2020-11-20T12:09:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e880677c0 con 0x7f3e880aa020 2020-11-20T12:09:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e677fe700 1 -- 172.21.15.44:0/1331789405 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78005800 con 0x7f3e880aa020 2020-11-20T12:09:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.534+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e880657d0 con 0x7f3e880aa020 2020-11-20T12:09:01.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.535+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e88061650 con 0x7f3e880aa020 2020-11-20T12:09:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.536+0000 7f3e677fe700 1 -- 172.21.15.44:0/1331789405 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3e78010030 con 0x7f3e880aa020 2020-11-20T12:09:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.536+0000 7f3e677fe700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3e60056d20 0x7f3e600591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.536+0000 7f3e677fe700 1 -- 172.21.15.44:0/1331789405 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3e7806fbe0 con 0x7f3e880aa020 2020-11-20T12:09:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.536+0000 7f3e86ffd700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3e60056d20 0x7f3e600591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:01.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.537+0000 7f3e86ffd700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3e60056d20 0x7f3e600591e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f3e7c00b150 tx=0x7f3e7c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:01.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.540+0000 7f3e677fe700 1 -- 172.21.15.44:0/1331789405 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3e78044e50 con 0x7f3e880aa020 2020-11-20T12:09:01.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.782+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3e880a9770 con 0x7f3e880aa020 2020-11-20T12:09:01.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.782+0000 7f3e677fe700 1 -- 172.21.15.44:0/1331789405 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f3e880a9770 con 0x7f3e880aa020 2020-11-20T12:09:01.787 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":54,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3e60056d20 msgr2=0x7f3e600591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3e60056d20 0x7f3e600591e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f3e7c00b150 tx=0x7f3e7c015000).stop 2020-11-20T12:09:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e880aa020 msgr2=0x7f3e88067c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e880aa020 0x7f3e88067c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3e7800c710 tx=0x7f3e78007c40).stop 2020-11-20T12:09:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 shutdown_connections 2020-11-20T12:09:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3e60056d20 0x7f3e600591e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3e8813ac60 0x7f3e880650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e880aa020 0x7f3e88067c40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 --2- 172.21.15.44:0/1331789405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e880aa400 0x7f3e880aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 >> 172.21.15.44:0/1331789405 conn(0x7f3e88062690 msgr2=0x7f3e8813bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 shutdown_connections 2020-11-20T12:09:01.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:01.784+0000 7f3e8ddf6700 1 -- 172.21.15.44:0/1331789405 wait complete. 2020-11-20T12:09:04.800 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.936+0000 7f218dfe5700 1 Processor -- start 2020-11-20T12:09:04.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.936+0000 7f218dfe5700 1 -- start start 2020-11-20T12:09:04.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.936+0000 7f218dfe5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:04.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.936+0000 7f218dfe5700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:04.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.936+0000 7f218dfe5700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21880a6260 con 0x7f21880a4c30 2020-11-20T12:09:04.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f218dfe5700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21880a52e0 con 0x7f218813b0a0 2020-11-20T12:09:04.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f218dfe5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21880a6490 con 0x7f2188060330 2020-11-20T12:09:04.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f2187fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:04.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f2186ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f21880a4c30 0x7f218813a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42790/0 (socket says 172.21.15.44:42790) 2020-11-20T12:09:04.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 learned_addr learned my addr 172.21.15.44:0/3954927849 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:04.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:04.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:04.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:04.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:04.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:04.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:04.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:04.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:04.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:04.937+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:05.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:05.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:05.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:05.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:05.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:05.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:09:05.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:05.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:05.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:05.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.138+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:09:05.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:05.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:05.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:05.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:05.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:09:05.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:05.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:05.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:05.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:05.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:05.539+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:09:06.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:06.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:06.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:06.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:06.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:09:06.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:06.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:06.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f21877fe700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:06.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:06.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:06.340+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:09:06.995 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:09:06.998 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:09:07.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 msgr2=0x7f21880a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:08.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2188060330 0x7f21880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:08.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 msgr2=0x7f2188141810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:08.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.44:3300/0 conn(0x7f218813b0a0 0x7f2188141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:08.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 -- 172.21.15.44:0/3954927849 >> v1:172.21.15.44:6789/0 conn(0x7f21880a4c30 legacy=0x7f218813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:08.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 -- 172.21.15.44:0/3954927849 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21880a6730 con 0x7f216c0055c0 2020-11-20T12:09:08.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f21857fa700 1 -- 172.21.15.44:0/3954927849 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21880a52e0 con 0x7f216c009270 2020-11-20T12:09:08.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:08.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:08.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:08.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:08.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:07.937+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:08.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.138+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:08.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.138+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:08.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.138+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:08.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.138+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:08.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.138+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:09:08.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.539+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:08.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.539+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:08.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.539+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:08.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.539+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:08.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:08.539+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:09:09.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.341+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.341+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:09.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.341+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:09.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.341+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:09.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.341+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:09:09.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.520+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101263418 0 0) 0x7f21880a6730 con 0x7f216c0055c0 2020-11-20T12:09:09.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.520+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2168003240 con 0x7f216c0055c0 2020-11-20T12:09:09.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3042054011 0 0) 0x7f2168003240 con 0x7f216c0055c0 2020-11-20T12:09:09.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 msgr2=0x7f216c00b710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:09.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 --2- 172.21.15.44:0/3954927849 >> v2:172.21.15.79:3300/0 conn(0x7f216c009270 0x7f216c00b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c00ce90 con 0x7f216c0055c0 2020-11-20T12:09:09.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f217800d1c0 con 0x7f216c0055c0 2020-11-20T12:09:09.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 >> v1:172.21.15.79:6789/0 conn(0x7f216c0055c0 legacy=0x7f216c007a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2187fff700 1 -- 172.21.15.44:0/3954927849 reap_dead start 2020-11-20T12:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168003780 0x7f2168003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2168004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168004e50 0x7f216800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21880a52e0 con 0x7f2168004e50 2020-11-20T12:09:09.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21680053b0 con 0x7f21680044e0 2020-11-20T12:09:09.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216800bca0 con 0x7f2168003780 2020-11-20T12:09:09.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168004e50 0x7f216800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2168004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f21877fe700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168003780 0x7f2168003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168003780 msgr2=0x7f2168003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:09:09.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168003780 0x7f2168003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168004e50 msgr2=0x7f216800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:09:09.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168004e50 0x7f216800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2186ffd700 1 -- 172.21.15.44:0/3954927849 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168003240 con 0x7f21680044e0 2020-11-20T12:09:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.521+0000 7f2187fff700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168004e50 0x7f216800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:09.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.522+0000 7f2186ffd700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2168004960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2170007f30 tx=0x7f2170009040).ready entity=mon.1 client_cookie=969ebc7a445790f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.523+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21700165d0 con 0x7f21680044e0 2020-11-20T12:09:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.523+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170016730 con 0x7f21680044e0 2020-11-20T12:09:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.523+0000 7f21867fc700 1 -- 172.21.15.44:0/3954927849 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170016ad0 con 0x7f21680044e0 2020-11-20T12:09:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 -- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 msgr2=0x7f2168004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2168004960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2170007f30 tx=0x7f2170009040).stop 2020-11-20T12:09:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 -- 172.21.15.44:0/3954927849 shutdown_connections 2020-11-20T12:09:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168004e50 0x7f216800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2168004960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 --2- 172.21.15.44:0/3954927849 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168003780 0x7f2168003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 -- 172.21.15.44:0/3954927849 >> 172.21.15.44:0/3954927849 conn(0x7f2188062690 msgr2=0x7f21880a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 -- 172.21.15.44:0/3954927849 shutdown_connections 2020-11-20T12:09:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.524+0000 7f218dfe5700 1 -- 172.21.15.44:0/3954927849 wait complete. 2020-11-20T12:09:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.525+0000 7f218dfe5700 1 Processor -- start 2020-11-20T12:09:09.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.525+0000 7f218dfe5700 1 -- start start 2020-11-20T12:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f218dfe5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168003780 0x7f2188149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f218dfe5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2188133ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f218dfe5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168004e50 0x7f21880aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f218dfe5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2188149a60 con 0x7f2168003780 2020-11-20T12:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f218dfe5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2188148b90 con 0x7f21680044e0 2020-11-20T12:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f218dfe5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2188148e30 con 0x7f2168004e50 2020-11-20T12:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f21877fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168003780 0x7f2188149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f21877fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168003780 0x7f2188149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39074/0 (socket says 172.21.15.44:39074) 2020-11-20T12:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f21877fe700 1 -- 172.21.15.44:0/2341925583 learned_addr learned my addr 172.21.15.44:0/2341925583 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f2186ffd700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2188133ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.526+0000 7f2187fff700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168004e50 0x7f21880aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f21877fe700 1 -- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168004e50 msgr2=0x7f21880aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f21877fe700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168004e50 0x7f21880aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f21877fe700 1 -- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 msgr2=0x7f2188133ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f21877fe700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2188133ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f21877fe700 1 -- 172.21.15.44:0/2341925583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21881490d0 con 0x7f2168003780 2020-11-20T12:09:09.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f2186ffd700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2188133ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:09.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f21877fe700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168003780 0x7f2188149570 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f2178016d80 tx=0x7f2178008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f217f7fe700 1 -- 172.21.15.44:0/2341925583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178014ca0 con 0x7f2168003780 2020-11-20T12:09:09.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f217f7fe700 1 -- 172.21.15.44:0/2341925583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2178014e00 con 0x7f2168003780 2020-11-20T12:09:09.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2188134390 con 0x7f2168003780 2020-11-20T12:09:09.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f217f7fe700 1 -- 172.21.15.44:0/2341925583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178017440 con 0x7f2168003780 2020-11-20T12:09:09.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.527+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218809f010 con 0x7f2168003780 2020-11-20T12:09:09.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.529+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218809f2c0 con 0x7f2168003780 2020-11-20T12:09:09.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.529+0000 7f217f7fe700 1 -- 172.21.15.44:0/2341925583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2178027070 con 0x7f2168003780 2020-11-20T12:09:09.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.529+0000 7f217f7fe700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2164056d20 0x7f21640591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.529+0000 7f217f7fe700 1 -- 172.21.15.44:0/2341925583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2178071080 con 0x7f2168003780 2020-11-20T12:09:09.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.529+0000 7f2186ffd700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2164056d20 0x7f21640591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.530+0000 7f2186ffd700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2164056d20 0x7f21640591e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f21880ac390 tx=0x7f2170009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.533+0000 7f217f7fe700 1 -- 172.21.15.44:0/2341925583 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2178046350 con 0x7f2168003780 2020-11-20T12:09:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.779+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2188061650 con 0x7f2168003780 2020-11-20T12:09:09.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.779+0000 7f217f7fe700 1 -- 172.21.15.44:0/2341925583 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f2178025070 con 0x7f2168003780 2020-11-20T12:09:09.784 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:09.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2164056d20 msgr2=0x7f21640591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2164056d20 0x7f21640591e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f21880ac390 tx=0x7f2170009040).stop 2020-11-20T12:09:09.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168003780 msgr2=0x7f2188149570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168003780 0x7f2188149570 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f2178016d80 tx=0x7f2178008040).stop 2020-11-20T12:09:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 shutdown_connections 2020-11-20T12:09:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2164056d20 0x7f21640591e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2168004e50 0x7f21880aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2168003780 0x7f2188149570 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 --2- 172.21.15.44:0/2341925583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21680044e0 0x7f2188133ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 >> 172.21.15.44:0/2341925583 conn(0x7f2188062690 msgr2=0x7f2188135210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 shutdown_connections 2020-11-20T12:09:09.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.781+0000 7f218dfe5700 1 -- 172.21.15.44:0/2341925583 wait complete. 2020-11-20T12:09:09.796 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:09:09.796 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:09:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.919+0000 7f4831247700 1 Processor -- start 2020-11-20T12:09:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.919+0000 7f4831247700 1 -- start start 2020-11-20T12:09:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f4831247700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa400 0x7f482c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f4831247700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f482c13ac60 0x7f482c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f4831247700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0a9770 con 0x7f482c0aa020 2020-11-20T12:09:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f4831247700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0a56c0 con 0x7f482c13ac60 2020-11-20T12:09:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f4831247700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0a6da0 con 0x7f482c0aa400 2020-11-20T12:09:09.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482a59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa400 0x7f482c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f482c13ac60 0x7f482c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f482c13ac60 0x7f482c0a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59366/0 (socket says 172.21.15.44:59366) 2020-11-20T12:09:09.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 -- 172.21.15.44:0/18617324 learned_addr learned my addr 172.21.15.44:0/18617324 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:09.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 -- 172.21.15.44:0/18617324 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa400 msgr2=0x7f482c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:09:09.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 --2- 172.21.15.44:0/18617324 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa400 0x7f482c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 -- 172.21.15.44:0/18617324 >> v1:172.21.15.79:6789/0 conn(0x7f482c0aa020 legacy=0x7f482c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 -- 172.21.15.44:0/18617324 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0a7040 con 0x7f482c13ac60 2020-11-20T12:09:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.920+0000 7f482b59e700 1 --2- 172.21.15.44:0/18617324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c13ac60 0x7f482c0a3700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4824009d50 tx=0x7f4824009290).ready entity=mon.1 client_cookie=9694c5c828392b76 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4829d9b700 1 -- 172.21.15.44:0/18617324 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400bbc0 con 0x7f482c13ac60 2020-11-20T12:09:09.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4829d9b700 1 -- 172.21.15.44:0/18617324 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482400bd20 con 0x7f482c13ac60 2020-11-20T12:09:09.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4829d9b700 1 -- 172.21.15.44:0/18617324 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824011610 con 0x7f482c13ac60 2020-11-20T12:09:09.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 -- 172.21.15.44:0/18617324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c13ac60 msgr2=0x7f482c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 --2- 172.21.15.44:0/18617324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c13ac60 0x7f482c0a3700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4824009d50 tx=0x7f4824009290).stop 2020-11-20T12:09:09.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 -- 172.21.15.44:0/18617324 shutdown_connections 2020-11-20T12:09:09.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 --2- 172.21.15.44:0/18617324 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c13ac60 0x7f482c0a3700 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 --2- 172.21.15.44:0/18617324 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa400 0x7f482c0a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 -- 172.21.15.44:0/18617324 >> 172.21.15.44:0/18617324 conn(0x7f482c062690 msgr2=0x7f482c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 -- 172.21.15.44:0/18617324 shutdown_connections 2020-11-20T12:09:09.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.921+0000 7f4831247700 1 -- 172.21.15.44:0/18617324 wait complete. 2020-11-20T12:09:09.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.922+0000 7f4831247700 1 Processor -- start 2020-11-20T12:09:09.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.922+0000 7f4831247700 1 -- start start 2020-11-20T12:09:09.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f4831247700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa020 0x7f482c137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f4831247700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c0aa400 0x7f482c065ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f4831247700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c13ac60 0x7f482c0661d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f4831247700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0666c0 con 0x7f482c13ac60 2020-11-20T12:09:09.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f4831247700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c067ca0 con 0x7f482c0aa400 2020-11-20T12:09:09.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f4831247700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c135300 con 0x7f482c0aa020 2020-11-20T12:09:09.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482b59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c13ac60 0x7f482c0661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c0aa400 0x7f482c065ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482b59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c13ac60 0x7f482c0661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39092/0 (socket says 172.21.15.44:39092) 2020-11-20T12:09:09.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482b59e700 1 -- 172.21.15.44:0/1166891300 learned_addr learned my addr 172.21.15.44:0/1166891300 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:09.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482ad9d700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa020 0x7f482c137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482a59c700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa020 msgr2=0x7f482c137660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482a59c700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f482c0aa020 0x7f482c137660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482a59c700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c13ac60 msgr2=0x7f482c0661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482a59c700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c13ac60 0x7f482c0661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482a59c700 1 -- 172.21.15.44:0/1166891300 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0ad970 con 0x7f482c0aa400 2020-11-20T12:09:09.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482b59e700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c13ac60 0x7f482c0661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.923+0000 7f482a59c700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c0aa400 0x7f482c065ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f481c00ebd0 tx=0x7f481c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.924+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c00c8b0 con 0x7f482c0aa400 2020-11-20T12:09:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.924+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c00ca10 con 0x7f482c0aa400 2020-11-20T12:09:09.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.924+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c067760 con 0x7f482c0aa400 2020-11-20T12:09:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.924+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c00cd90 con 0x7f482c0aa400 2020-11-20T12:09:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.924+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c130390 con 0x7f482c0aa400 2020-11-20T12:09:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.924+0000 7f4831247700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0abc30 0x7f482c0ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.924+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0acf20 con 0x7f482c0abc30 2020-11-20T12:09:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f482ad9d700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0abc30 0x7f482c0ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f482ad9d700 1 -- 172.21.15.44:0/1166891300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f482c061650 con 0x7f482c0abc30 2020-11-20T12:09:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f481c012600 con 0x7f482c0aa400 2020-11-20T12:09:09.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f482ad9d700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0abc30 0x7f482c0ac010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f482000f820 tx=0x7f4820007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4808056d20 0x7f48080591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f481c0703b0 con 0x7f482c0aa400 2020-11-20T12:09:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820011650 con 0x7f482c0abc30 2020-11-20T12:09:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f482b59e700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4808056d20 0x7f48080591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f48200117b0 con 0x7f482c0abc30 2020-11-20T12:09:09.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0abc30 msgr2=0x7f482c0ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0abc30 0x7f482c0ac010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f482000f820 tx=0x7f4820007b90).stop 2020-11-20T12:09:09.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0abc30 msgr2=0x7f482c0ac010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:09.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.925+0000 7f4812ffd700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0abc30 0x7f482c0ac010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.926+0000 7f482b59e700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4808056d20 0x7f48080591e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f4824009500 tx=0x7f4824016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.933+0000 7f4831247700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0ac500 0x7f482c12afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:09.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.933+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0446d0 con 0x7f482c0ac500 2020-11-20T12:09:09.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.933+0000 7f482ad9d700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0ac500 0x7f482c12afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f482ad9d700 1 -- 172.21.15.44:0/1166891300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f482c0613b0 con 0x7f482c0ac500 2020-11-20T12:09:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f482ad9d700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0ac500 0x7f482c12afc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f4820007570 tx=0x7f4820018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820014e30 con 0x7f482c0ac500 2020-11-20T12:09:09.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f4820011440 con 0x7f482c0ac500 2020-11-20T12:09:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0ac500 msgr2=0x7f482c12afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4812ffd700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0ac500 0x7f482c12afc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f4820007570 tx=0x7f4820018000).stop 2020-11-20T12:09:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4812ffd700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0ac500 msgr2=0x7f482c12afc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:09.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4812ffd700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f482c0ac500 0x7f482c12afc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4808056d20 msgr2=0x7f48080591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4808056d20 0x7f48080591e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f4824009500 tx=0x7f4824016000).stop 2020-11-20T12:09:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c0aa400 msgr2=0x7f482c065ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:09.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 --2- 172.21.15.44:0/1166891300 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f482c0aa400 0x7f482c065ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f481c00ebd0 tx=0x7f481c006df0).stop 2020-11-20T12:09:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f482b59e700 1 -- 172.21.15.44:0/1166891300 reap_dead start 2020-11-20T12:09:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f482b59e700 1 -- 172.21.15.44:0/1166891300 reap_dead start 2020-11-20T12:09:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 shutdown_connections 2020-11-20T12:09:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 >> 172.21.15.44:0/1166891300 conn(0x7f482c062690 msgr2=0x7f482c0a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:09.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 shutdown_connections 2020-11-20T12:09:09.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:09.934+0000 7f4831247700 1 -- 172.21.15.44:0/1166891300 wait complete. 2020-11-20T12:09:09.952 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:09.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:09.952 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:09.953 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:09:09.953 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 62, 2020-11-20T12:09:09.953 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:09.954 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:09:09.954 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:09.954 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:09.954 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:09.955 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:09:09.955 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:09.955 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:09.955 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:09.955 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:09.956 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:09.956 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:09.956 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:09.956 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:09.956 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:09.957 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:09.957 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:09.957 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:09.957 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:09.957 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:09.958 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:09.958 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:09.958 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:09.958 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:09.959 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:09.959 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:09.959 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:09.959 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:09.959 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:09.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.960 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:09.960 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:09.960 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:09.960 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:09.961 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:09.961 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:09.961 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:09.961 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:09.961 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:09.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:09.962 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:09.962 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:09.962 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:09.963 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:09.963 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:09.963 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:09.963 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:09.963 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:09.964 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:09.964 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:09.964 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:09.964 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:09.964 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:09.965 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:09.965 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:09.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.965 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:09.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.966 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:09.966 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:09.966 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:09.966 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:09.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:09.967 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:09.967 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:09.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:09.968 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:09.968 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:09.968 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.969 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:09.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.969 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:09.969 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:09.969 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:09.970 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:09.970 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:09.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.970 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:09.971 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:09.971 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:09.971 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:09.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:09.972 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:09.972 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:09.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:09.973 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:09.973 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:09.973 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.973 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:09.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.974 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:09.974 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:09.974 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:09.974 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:09.975 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:09.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.975 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:09.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:09.976 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:09.976 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:09.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:09.976 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:09.977 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:09.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:09.977 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:09.978 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:09.978 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.978 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:09.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.978 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:09.979 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:09.979 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:09.979 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:09.979 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:09.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.980 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:09.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.980 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:09.980 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:09.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.981 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:09.981 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:09.981 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:09.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.981 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:09.982 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:09:09.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.982 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:09.982 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:09.982 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:09.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.983 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:09.983 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:09:09.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.983 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:09.984 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:09.984 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:09.984 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.984 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:09.985 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:09.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.985 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:09.985 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:09.985 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:09.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.986 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:09.986 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:09:09.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:09.986 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:09.987 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:09.987 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:09.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:09.987 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:09.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:09.988 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:09.988 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:09.989 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:09:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.112+0000 7fd6b8ba7700 1 Processor -- start 2020-11-20T12:09:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.112+0000 7fd6b8ba7700 1 -- start start 2020-11-20T12:09:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b8ba7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40a91f0 0x7fd6b40aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b8ba7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b4136270 con 0x7fd6b40abad0 2020-11-20T12:09:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b8ba7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b40680d0 con 0x7fd6b40679c0 2020-11-20T12:09:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b8ba7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b40ab030 con 0x7fd6b40a91f0 2020-11-20T12:09:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b259c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd6b40abad0 0x7fd6b40a8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42848/0 (socket says 172.21.15.44:42848) 2020-11-20T12:09:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b259c700 1 -- 172.21.15.44:0/2155297628 learned_addr learned my addr 172.21.15.44:0/2155297628 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b1d9b700 1 --2- 172.21.15.44:0/2155297628 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40a91f0 0x7fd6b40aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b159a700 1 -- 172.21.15.44:0/2155297628 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742633572 0 0) 0x7fd6b4136270 con 0x7fd6b40abad0 2020-11-20T12:09:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b159a700 1 -- 172.21.15.44:0/2155297628 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd698003240 con 0x7fd6b40abad0 2020-11-20T12:09:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b159a700 1 -- 172.21.15.44:0/2155297628 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29492908 0 0) 0x7fd6b40680d0 con 0x7fd6b40679c0 2020-11-20T12:09:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.113+0000 7fd6b159a700 1 -- 172.21.15.44:0/2155297628 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b4136270 con 0x7fd6b40679c0 2020-11-20T12:09:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b1d9b700 1 -- 172.21.15.44:0/2155297628 >> v1:172.21.15.79:6789/0 conn(0x7fd6b40679c0 legacy=0x7fd6b4138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b1d9b700 1 -- 172.21.15.44:0/2155297628 >> v1:172.21.15.44:6789/0 conn(0x7fd6b40abad0 legacy=0x7fd6b40a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b1d9b700 1 -- 172.21.15.44:0/2155297628 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4068370 con 0x7fd6b40a91f0 2020-11-20T12:09:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b1d9b700 1 --2- 172.21.15.44:0/2155297628 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40a91f0 0x7fd6b40aa5e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fd6a400b9d0 tx=0x7fd6a4006df0).ready entity=mon.2 client_cookie=4f956d0077fa1be0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b159a700 1 -- 172.21.15.44:0/2155297628 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4009680 con 0x7fd6b40a91f0 2020-11-20T12:09:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b159a700 1 -- 172.21.15.44:0/2155297628 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a40097e0 con 0x7fd6b40a91f0 2020-11-20T12:09:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b159a700 1 -- 172.21.15.44:0/2155297628 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4009b60 con 0x7fd6b40a91f0 2020-11-20T12:09:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/2155297628 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40a91f0 msgr2=0x7fd6b40aa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b8ba7700 1 --2- 172.21.15.44:0/2155297628 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40a91f0 0x7fd6b40aa5e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fd6a400b9d0 tx=0x7fd6a4006df0).stop 2020-11-20T12:09:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/2155297628 shutdown_connections 2020-11-20T12:09:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b8ba7700 1 --2- 172.21.15.44:0/2155297628 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40a91f0 0x7fd6b40aa5e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/2155297628 >> 172.21.15.44:0/2155297628 conn(0x7fd6b4065290 msgr2=0x7fd6b4060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/2155297628 shutdown_connections 2020-11-20T12:09:10.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.114+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/2155297628 wait complete. 2020-11-20T12:09:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.115+0000 7fd6b8ba7700 1 Processor -- start 2020-11-20T12:09:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.115+0000 7fd6b8ba7700 1 -- start start 2020-11-20T12:09:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b8ba7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40679c0 0x7fd6b40a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b8ba7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40abad0 0x7fd6b40631a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b8ba7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b4063690 0x7fd6b414a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b8ba7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b40621e0 con 0x7fd6b4063690 2020-11-20T12:09:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b8ba7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4062480 con 0x7fd6b40abad0 2020-11-20T12:09:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b8ba7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4062720 con 0x7fd6b40679c0 2020-11-20T12:09:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b4063690 0x7fd6b414a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b4063690 0x7fd6b414a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39110/0 (socket says 172.21.15.44:39110) 2020-11-20T12:09:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b2d9d700 1 -- 172.21.15.44:0/1313873142 learned_addr learned my addr 172.21.15.44:0/1313873142 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b1d9b700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40abad0 0x7fd6b40631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b259c700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40679c0 0x7fd6b40a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.116+0000 7fd6b2d9d700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40679c0 msgr2=0x7fd6b40a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6b2d9d700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6b40679c0 0x7fd6b40a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6b2d9d700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40abad0 msgr2=0x7fd6b40631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6b2d9d700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40abad0 0x7fd6b40631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6b2d9d700 1 -- 172.21.15.44:0/1313873142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40629c0 con 0x7fd6b4063690 2020-11-20T12:09:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6b2d9d700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b4063690 0x7fd6b414a560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fd6ac00ca90 tx=0x7fd6ac007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac002240 con 0x7fd6b4063690 2020-11-20T12:09:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac0023a0 con 0x7fd6b4063690 2020-11-20T12:09:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b414aa50 con 0x7fd6b4063690 2020-11-20T12:09:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac001820 con 0x7fd6b4063690 2020-11-20T12:09:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.117+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b4133200 con 0x7fd6b4063690 2020-11-20T12:09:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6b8ba7700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40a91f0 0x7fd6b41334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b405c2a0 con 0x7fd6b40a91f0 2020-11-20T12:09:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6b259c700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40a91f0 0x7fd6b41334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd6ac019030 con 0x7fd6b4063690 2020-11-20T12:09:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6b259c700 1 -- 172.21.15.44:0/1313873142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6b4062b50 con 0x7fd6b40a91f0 2020-11-20T12:09:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6b259c700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40a91f0 0x7fd6b41334b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd6a800f750 tx=0x7fd6a8009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6a27fc700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d50 0x7fd688059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.118+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd6ac06f7f0 con 0x7fd6b4063690 2020-11-20T12:09:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6b1d9b700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d50 0x7fd688059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80070d0 con 0x7fd6b40a91f0 2020-11-20T12:09:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fd6a8007230 con 0x7fd6b40a91f0 2020-11-20T12:09:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40a91f0 msgr2=0x7fd6b41334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6a27fc700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40a91f0 0x7fd6b41334b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fd6a800f750 tx=0x7fd6a8009040).stop 2020-11-20T12:09:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40a91f0 msgr2=0x7fd6b41334b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6a27fc700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40a91f0 0x7fd6b41334b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.119+0000 7fd6b1d9b700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d50 0x7fd688059210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fd6a400b150 tx=0x7fd6a4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.126+0000 7fd6b8ba7700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40643c0 0x7fd6b40a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.126+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b40676e0 con 0x7fd6b40643c0 2020-11-20T12:09:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.127+0000 7fd6b259c700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40643c0 0x7fd6b40a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.127+0000 7fd6b259c700 1 -- 172.21.15.44:0/1313873142 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd6b40446d0 con 0x7fd6b40643c0 2020-11-20T12:09:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.127+0000 7fd6b259c700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40643c0 0x7fd6b40a7760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd6a8015000 tx=0x7fd6a8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.127+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a800da50 con 0x7fd6b40643c0 2020-11-20T12:09:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4265 (secure 0 0 0) 0x7fd6a80055d0 con 0x7fd6b40643c0 2020-11-20T12:09:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40643c0 msgr2=0x7fd6b40a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6a27fc700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40643c0 0x7fd6b40a7760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd6a8015000 tx=0x7fd6a8016000).stop 2020-11-20T12:09:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6a27fc700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40643c0 msgr2=0x7fd6b40a7760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6a27fc700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6b40643c0 0x7fd6b40a7760 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d50 msgr2=0x7fd688059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd688056d50 0x7fd688059210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fd6a400b150 tx=0x7fd6a4015000).stop 2020-11-20T12:09:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b4063690 msgr2=0x7fd6b414a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b4063690 0x7fd6b414a560 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fd6ac00ca90 tx=0x7fd6ac007dd0).stop 2020-11-20T12:09:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b2d9d700 1 -- 172.21.15.44:0/1313873142 reap_dead start 2020-11-20T12:09:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 shutdown_connections 2020-11-20T12:09:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 --2- 172.21.15.44:0/1313873142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b4063690 0x7fd6b414a560 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 >> 172.21.15.44:0/1313873142 conn(0x7fd6b4065290 msgr2=0x7fd6b40aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 shutdown_connections 2020-11-20T12:09:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.128+0000 7fd6b8ba7700 1 -- 172.21.15.44:0/1313873142 wait complete. 2020-11-20T12:09:10.142 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:09:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 62, 2020-11-20T12:09:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:09:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:09:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:10.173 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:10.174 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:09:10.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.297+0000 7fda73cff700 1 Processor -- start 2020-11-20T12:09:10.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.297+0000 7fda73cff700 1 -- start start 2020-11-20T12:09:10.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda73cff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fda6c0aa400 0x7fda6c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda73cff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c13ac60 0x7fda6c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda73cff700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0a9770 con 0x7fda6c0aa020 2020-11-20T12:09:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda73cff700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c0a56c0 con 0x7fda6c0aa400 2020-11-20T12:09:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda73cff700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0a6da0 con 0x7fda6c13ac60 2020-11-20T12:09:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda7129a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fda6c0aa400 0x7fda6c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda71a9b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fda6c0aa020 0x7fda6c0a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42866/0 (socket says 172.21.15.44:42866) 2020-11-20T12:09:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda71a9b700 1 -- 172.21.15.44:0/297790640 learned_addr learned my addr 172.21.15.44:0/297790640 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda7229c700 1 --2- 172.21.15.44:0/297790640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c13ac60 0x7fda6c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda7129a700 1 -- 172.21.15.44:0/297790640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c13ac60 msgr2=0x7fda6c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:09:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda7129a700 1 --2- 172.21.15.44:0/297790640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c13ac60 0x7fda6c0a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda7129a700 1 -- 172.21.15.44:0/297790640 >> v1:172.21.15.44:6789/0 conn(0x7fda6c0aa020 legacy=0x7fda6c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda70a99700 1 -- 172.21.15.44:0/297790640 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560412140 0 0) 0x7fda5c0010d0 con 0x7fda6c0aa020 2020-11-20T12:09:10.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.298+0000 7fda7129a700 1 -- 172.21.15.44:0/297790640 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0a7040 con 0x7fda6c0aa400 2020-11-20T12:09:10.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda7129a700 1 --2- 172.21.15.44:0/297790640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c0aa400 0x7fda6c0a9280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fda6800b9a0 tx=0x7fda68006df0).ready entity=mon.0 client_cookie=f717fac3b2449840 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda70a99700 1 -- 172.21.15.44:0/297790640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda680097b0 con 0x7fda6c0aa400 2020-11-20T12:09:10.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda70a99700 1 -- 172.21.15.44:0/297790640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda68009910 con 0x7fda6c0aa400 2020-11-20T12:09:10.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda70a99700 1 -- 172.21.15.44:0/297790640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68009c90 con 0x7fda6c0aa400 2020-11-20T12:09:10.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 -- 172.21.15.44:0/297790640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c0aa400 msgr2=0x7fda6c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 --2- 172.21.15.44:0/297790640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c0aa400 0x7fda6c0a9280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fda6800b9a0 tx=0x7fda68006df0).stop 2020-11-20T12:09:10.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 -- 172.21.15.44:0/297790640 shutdown_connections 2020-11-20T12:09:10.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 --2- 172.21.15.44:0/297790640 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c13ac60 0x7fda6c0a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 --2- 172.21.15.44:0/297790640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c0aa400 0x7fda6c0a9280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 -- 172.21.15.44:0/297790640 >> 172.21.15.44:0/297790640 conn(0x7fda6c062690 msgr2=0x7fda6c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 -- 172.21.15.44:0/297790640 shutdown_connections 2020-11-20T12:09:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.299+0000 7fda73cff700 1 -- 172.21.15.44:0/297790640 wait complete. 2020-11-20T12:09:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.300+0000 7fda73cff700 1 Processor -- start 2020-11-20T12:09:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.300+0000 7fda73cff700 1 -- start start 2020-11-20T12:09:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda73cff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda6c0aa020 0x7fda6c067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda73cff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c0aa400 0x7fda6c060170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda73cff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c13ac60 0x7fda6c0a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda73cff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c065220 con 0x7fda6c13ac60 2020-11-20T12:09:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda73cff700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0654c0 con 0x7fda6c0aa020 2020-11-20T12:09:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda73cff700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c065960 con 0x7fda6c0aa400 2020-11-20T12:09:10.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c13ac60 0x7fda6c0a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c13ac60 0x7fda6c0a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39128/0 (socket says 172.21.15.44:39128) 2020-11-20T12:09:10.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 -- 172.21.15.44:0/4139944596 learned_addr learned my addr 172.21.15.44:0/4139944596 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:10.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda71a9b700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda6c0aa020 0x7fda6c067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7129a700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c0aa400 0x7fda6c060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c0aa400 msgr2=0x7fda6c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c0aa400 0x7fda6c060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda6c0aa020 msgr2=0x7fda6c067750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda6c0aa020 0x7fda6c067750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 -- 172.21.15.44:0/4139944596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c067c40 con 0x7fda6c13ac60 2020-11-20T12:09:10.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda7229c700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c13ac60 0x7fda6c0a2a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fda6400ed20 tx=0x7fda640091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400eee0 con 0x7fda6c13ac60 2020-11-20T12:09:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.301+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda64009580 con 0x7fda6c13ac60 2020-11-20T12:09:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.302+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c060660 con 0x7fda6c13ac60 2020-11-20T12:09:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.302+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400cd30 con 0x7fda6c13ac60 2020-11-20T12:09:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.302+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c0a30d0 con 0x7fda6c13ac60 2020-11-20T12:09:10.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda73cff700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c134320 0x7fda6c134700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0650a0 con 0x7fda6c134320 2020-11-20T12:09:10.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fda64022070 con 0x7fda6c13ac60 2020-11-20T12:09:10.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda71a9b700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c134320 0x7fda6c134700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda59ffb700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda44056d20 0x7fda440591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda71a9b700 1 -- 172.21.15.44:0/4139944596 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda6c134bf0 con 0x7fda6c134320 2020-11-20T12:09:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fda640705c0 con 0x7fda6c13ac60 2020-11-20T12:09:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda7129a700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda44056d20 0x7fda440591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda71a9b700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c134320 0x7fda6c134700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fda5c001630 tx=0x7fda5c007860).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.303+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011650 con 0x7fda6c134320 2020-11-20T12:09:10.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.304+0000 7fda7129a700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda44056d20 0x7fda440591e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fda68006e30 tx=0x7fda68015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.304+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fda5c0117b0 con 0x7fda6c134320 2020-11-20T12:09:10.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.304+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c134320 msgr2=0x7fda6c134700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.304+0000 7fda59ffb700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c134320 0x7fda6c134700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fda5c001630 tx=0x7fda5c007860).stop 2020-11-20T12:09:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.304+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c134320 msgr2=0x7fda6c134700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.304+0000 7fda59ffb700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c134320 0x7fda6c134700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.311+0000 7fda73cff700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c14d690 0x7fda6c14daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.311+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c060850 con 0x7fda6c14d690 2020-11-20T12:09:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda71a9b700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c14d690 0x7fda6c14daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda71a9b700 1 -- 172.21.15.44:0/4139944596 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fda6c0a7220 con 0x7fda6c14d690 2020-11-20T12:09:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda71a9b700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c14d690 0x7fda6c14daa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fda5c007ad0 tx=0x7fda5c018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c014ce0 con 0x7fda6c14d690 2020-11-20T12:09:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4775 (secure 0 0 0) 0x7fda5c014e40 con 0x7fda6c14d690 2020-11-20T12:09:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c14d690 msgr2=0x7fda6c14daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda59ffb700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c14d690 0x7fda6c14daa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fda5c007ad0 tx=0x7fda5c018000).stop 2020-11-20T12:09:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda59ffb700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c14d690 msgr2=0x7fda6c14daa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.312+0000 7fda59ffb700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda6c14d690 0x7fda6c14daa0 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0x7fda5c007ad0 tx=0x7fda5c018000).stop 2020-11-20T12:09:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda44056d20 msgr2=0x7fda440591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda44056d20 0x7fda440591e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fda68006e30 tx=0x7fda68015000).stop 2020-11-20T12:09:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c13ac60 msgr2=0x7fda6c0a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 --2- 172.21.15.44:0/4139944596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6c13ac60 0x7fda6c0a2a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fda6400ed20 tx=0x7fda640091d0).stop 2020-11-20T12:09:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda7229c700 1 -- 172.21.15.44:0/4139944596 reap_dead start 2020-11-20T12:09:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda7229c700 1 -- 172.21.15.44:0/4139944596 reap_dead start 2020-11-20T12:09:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 shutdown_connections 2020-11-20T12:09:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 >> 172.21.15.44:0/4139944596 conn(0x7fda6c062690 msgr2=0x7fda6c13c540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 shutdown_connections 2020-11-20T12:09:10.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.313+0000 7fda73cff700 1 -- 172.21.15.44:0/4139944596 wait complete. 2020-11-20T12:09:10.332 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:10.332 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:10.333 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:10.333 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:09:10.333 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 62, 2020-11-20T12:09:10.333 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:10.334 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:09:10.334 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:10.334 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:10.334 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.335 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:09:10.335 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:10.335 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:10.335 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:10.336 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:10.336 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:10.336 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:10.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:10.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:10.337 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:10.337 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:10.337 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:10.337 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.337 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:10.338 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:10.338 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:10.338 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:10.338 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:10.339 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:10.339 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:10.339 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:10.339 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:10.340 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:10.340 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.340 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:10.340 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:09:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:09:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:09:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:09:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:09:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:10.368 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:10.368 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:10.369 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:09:10.369 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:09:10.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.492+0000 7f1db94b3700 1 Processor -- start 2020-11-20T12:09:10.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.492+0000 7f1db94b3700 1 -- start start 2020-11-20T12:09:10.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db94b3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1db40aa1e0 0x7f1db4065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db94b3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1db413c880 0x7f1db4066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db94b3700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db40a8bf0 con 0x7f1db40a7810 2020-11-20T12:09:10.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db94b3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db4066520 con 0x7f1db40aa1e0 2020-11-20T12:09:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db94b3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db40a86b0 con 0x7f1db413c880 2020-11-20T12:09:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db2ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1db40aa1e0 0x7f1db4065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db27fc700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1db40a7810 0x7f1db413c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42884/0 (socket says 172.21.15.44:42884) 2020-11-20T12:09:10.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db27fc700 1 -- 172.21.15.44:0/1791612381 learned_addr learned my addr 172.21.15.44:0/1791612381 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.493+0000 7f1db37fe700 1 --2- 172.21.15.44:0/1791612381 >> v2:172.21.15.79:3300/0 conn(0x7f1db413c880 0x7f1db4066030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db2ffd700 1 -- 172.21.15.44:0/1791612381 >> v2:172.21.15.79:3300/0 conn(0x7f1db413c880 msgr2=0x7f1db4066030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:09:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db2ffd700 1 --2- 172.21.15.44:0/1791612381 >> v2:172.21.15.79:3300/0 conn(0x7f1db413c880 0x7f1db4066030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db2ffd700 1 -- 172.21.15.44:0/1791612381 >> v1:172.21.15.44:6789/0 conn(0x7f1db40a7810 legacy=0x7f1db413c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db1ffb700 1 -- 172.21.15.44:0/1791612381 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536295809 0 0) 0x7f1db40a8bf0 con 0x7f1db40a7810 2020-11-20T12:09:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db2ffd700 1 -- 172.21.15.44:0/1791612381 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40a8950 con 0x7f1db40aa1e0 2020-11-20T12:09:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db37fe700 1 --2- 172.21.15.44:0/1791612381 >> v2:172.21.15.79:3300/0 conn(0x7f1db413c880 0x7f1db4066030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:09:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db2ffd700 1 --2- 172.21.15.44:0/1791612381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db40aa1e0 0x7f1db4065b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f1db40ace20 tx=0x7f1da8006df0).ready entity=mon.0 client_cookie=304ffe2a662feca4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db1ffb700 1 -- 172.21.15.44:0/1791612381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8009840 con 0x7f1db40aa1e0 2020-11-20T12:09:10.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db1ffb700 1 -- 172.21.15.44:0/1791612381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da80099a0 con 0x7f1db40aa1e0 2020-11-20T12:09:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db1ffb700 1 -- 172.21.15.44:0/1791612381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8009d20 con 0x7f1db40aa1e0 2020-11-20T12:09:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 -- 172.21.15.44:0/1791612381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db40aa1e0 msgr2=0x7f1db4065b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1791612381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db40aa1e0 0x7f1db4065b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f1db40ace20 tx=0x7f1da8006df0).stop 2020-11-20T12:09:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 -- 172.21.15.44:0/1791612381 shutdown_connections 2020-11-20T12:09:10.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1791612381 >> v2:172.21.15.79:3300/0 conn(0x7f1db413c880 0x7f1db4066030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1791612381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db40aa1e0 0x7f1db4065b40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 -- 172.21.15.44:0/1791612381 >> 172.21.15.44:0/1791612381 conn(0x7f1db4062690 msgr2=0x7f1db40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 -- 172.21.15.44:0/1791612381 shutdown_connections 2020-11-20T12:09:10.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.494+0000 7f1db94b3700 1 -- 172.21.15.44:0/1791612381 wait complete. 2020-11-20T12:09:10.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.495+0000 7f1db94b3700 1 Processor -- start 2020-11-20T12:09:10.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db94b3700 1 -- start start 2020-11-20T12:09:10.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db94b3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db40a7810 0x7f1db40a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db94b3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db40aa1e0 0x7f1db40a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db94b3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db413c880 0x7f1db41392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db94b3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4147dd0 con 0x7f1db413c880 2020-11-20T12:09:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db94b3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4148070 con 0x7f1db40aa1e0 2020-11-20T12:09:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db94b3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4148310 con 0x7f1db40a7810 2020-11-20T12:09:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db413c880 0x7f1db41392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db413c880 0x7f1db41392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39146/0 (socket says 172.21.15.44:39146) 2020-11-20T12:09:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db37fe700 1 -- 172.21.15.44:0/1371190447 learned_addr learned my addr 172.21.15.44:0/1371190447 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db27fc700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db40aa1e0 0x7f1db40a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.496+0000 7f1db2ffd700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db40a7810 0x7f1db40a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db37fe700 1 -- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db40a7810 msgr2=0x7f1db40a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db37fe700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db40a7810 0x7f1db40a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db37fe700 1 -- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db40aa1e0 msgr2=0x7f1db40a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db37fe700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db40aa1e0 0x7f1db40a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db37fe700 1 -- 172.21.15.44:0/1371190447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40a0b10 con 0x7f1db413c880 2020-11-20T12:09:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db37fe700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db413c880 0x7f1db41392e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1dac00db60 tx=0x7f1dac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1d9affd700 1 -- 172.21.15.44:0/1371190447 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00b900 con 0x7f1db413c880 2020-11-20T12:09:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1d9affd700 1 -- 172.21.15.44:0/1371190447 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac00ba60 con 0x7f1db413c880 2020-11-20T12:09:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db4147ac0 con 0x7f1db413c880 2020-11-20T12:09:10.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1d9affd700 1 -- 172.21.15.44:0/1371190447 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00be00 con 0x7f1db413c880 2020-11-20T12:09:10.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.497+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db40a1fb0 con 0x7f1db413c880 2020-11-20T12:09:10.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.498+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db41484d0 con 0x7f1db413c880 2020-11-20T12:09:10.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.498+0000 7f1d9affd700 1 -- 172.21.15.44:0/1371190447 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1dac012630 con 0x7f1db413c880 2020-11-20T12:09:10.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.498+0000 7f1d9affd700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d90056d20 0x7f1d900591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.498+0000 7f1d9affd700 1 -- 172.21.15.44:0/1371190447 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1dac070060 con 0x7f1db413c880 2020-11-20T12:09:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.499+0000 7f1db2ffd700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d90056d20 0x7f1d900591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.499+0000 7f1db2ffd700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d90056d20 0x7f1d900591e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1da800c010 tx=0x7f1da8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.502+0000 7f1d9affd700 1 -- 172.21.15.44:0/1371190447 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1dac045330 con 0x7f1db413c880 2020-11-20T12:09:10.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.747+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1db4061650 con 0x7f1db413c880 2020-11-20T12:09:10.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.748+0000 7f1d9affd700 1 -- 172.21.15.44:0/1371190447 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f1dac048950 con 0x7f1db413c880 2020-11-20T12:09:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d90056d20 msgr2=0x7f1d900591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d90056d20 0x7f1d900591e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1da800c010 tx=0x7f1da8015000).stop 2020-11-20T12:09:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db413c880 msgr2=0x7f1db41392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:10.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db413c880 0x7f1db41392e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1dac00db60 tx=0x7f1dac008040).stop 2020-11-20T12:09:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 shutdown_connections 2020-11-20T12:09:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1d90056d20 0x7f1d900591e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1db413c880 0x7f1db41392e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1db40a7810 0x7f1db40a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 --2- 172.21.15.44:0/1371190447 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1db40aa1e0 0x7f1db40a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:10.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 >> 172.21.15.44:0/1371190447 conn(0x7f1db4062690 msgr2=0x7f1db4066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:10.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 shutdown_connections 2020-11-20T12:09:10.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:10.750+0000 7f1db94b3700 1 -- 172.21.15.44:0/1371190447 wait complete. 2020-11-20T12:09:10.766 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:09:20.766 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:09:20.767 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:20.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.902+0000 7fb121f97700 1 Processor -- start 2020-11-20T12:09:20.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.902+0000 7fb121f97700 1 -- start start 2020-11-20T12:09:20.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb121f97700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:20.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb121f97700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0a9770 con 0x7fb11c0aa400 2020-11-20T12:09:20.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb121f97700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0a6da0 con 0x7fb11c0aa020 2020-11-20T12:09:20.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb121f97700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0a56c0 con 0x7fb11c0a3e10 2020-11-20T12:09:20.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb11affd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb11c0aa400 0x7fb11c0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42898/0 (socket says 172.21.15.44:42898) 2020-11-20T12:09:20.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb11affd700 1 -- 172.21.15.44:0/3117183668 learned_addr learned my addr 172.21.15.44:0/3117183668 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:20.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb11bfff700 1 --2- 172.21.15.44:0/3117183668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:20.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618724010 0 0) 0x7fb1100010d0 con 0x7fb11c0aa400 2020-11-20T12:09:20.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.903+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb108003240 con 0x7fb11c0aa400 2020-11-20T12:09:20.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905187995 0 0) 0x7fb11c0a6da0 con 0x7fb11c0aa020 2020-11-20T12:09:20.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1100010d0 con 0x7fb11c0aa020 2020-11-20T12:09:20.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11bfff700 1 -- 172.21.15.44:0/3117183668 >> v1:172.21.15.79:6789/0 conn(0x7fb11c0aa020 legacy=0x7fb11c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:20.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11bfff700 1 -- 172.21.15.44:0/3117183668 >> v1:172.21.15.44:6789/0 conn(0x7fb11c0aa400 legacy=0x7fb11c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:20.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1439490607 0 0) 0x7fb108003240 con 0x7fb11c0aa400 2020-11-20T12:09:20.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11bfff700 1 -- 172.21.15.44:0/3117183668 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0a7040 con 0x7fb11c0a3e10 2020-11-20T12:09:20.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11bfff700 1 --2- 172.21.15.44:0/3117183668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0a9280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb114009cc0 tx=0x7fb1140091d0).ready entity=mon.2 client_cookie=b430bd648703f292 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:20.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11400ba20 con 0x7fb11c0a3e10 2020-11-20T12:09:20.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11400bb80 con 0x7fb11c0a3e10 2020-11-20T12:09:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb11a7fc700 1 -- 172.21.15.44:0/3117183668 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114011400 con 0x7fb11c0a3e10 2020-11-20T12:09:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb121f97700 1 -- 172.21.15.44:0/3117183668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 msgr2=0x7fb11c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb121f97700 1 --2- 172.21.15.44:0/3117183668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0a9280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb114009cc0 tx=0x7fb1140091d0).stop 2020-11-20T12:09:20.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb121f97700 1 -- 172.21.15.44:0/3117183668 shutdown_connections 2020-11-20T12:09:20.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb121f97700 1 --2- 172.21.15.44:0/3117183668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0a9280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:20.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.904+0000 7fb121f97700 1 -- 172.21.15.44:0/3117183668 >> 172.21.15.44:0/3117183668 conn(0x7fb11c062690 msgr2=0x7fb11c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.905+0000 7fb121f97700 1 -- 172.21.15.44:0/3117183668 shutdown_connections 2020-11-20T12:09:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.905+0000 7fb121f97700 1 -- 172.21.15.44:0/3117183668 wait complete. 2020-11-20T12:09:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 Processor -- start 2020-11-20T12:09:20.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 -- start start 2020-11-20T12:09:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb11c0aa020 0x7fb11c146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:20.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c0aa400 0x7fb11c065180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c147260 con 0x7fb11c0aa400 2020-11-20T12:09:20.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c147500 con 0x7fb11c0aa020 2020-11-20T12:09:20.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb121f97700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c1477a0 con 0x7fb11c0a3e10 2020-11-20T12:09:20.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c0aa400 0x7fb11c065180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:20.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb11bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c0aa400 0x7fb11c065180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39160/0 (socket says 172.21.15.44:39160) 2020-11-20T12:09:20.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb11bfff700 1 -- 172.21.15.44:0/2720010426 learned_addr learned my addr 172.21.15.44:0/2720010426 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:20.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.906+0000 7fb11affd700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb11c0aa020 0x7fb11c146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:20.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11b7fe700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:20.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11bfff700 1 -- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 msgr2=0x7fb11c0664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:20.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11bfff700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:20.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11bfff700 1 -- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb11c0aa020 msgr2=0x7fb11c146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:20.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11bfff700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb11c0aa020 0x7fb11c146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:20.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11bfff700 1 -- 172.21.15.44:0/2720010426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c146f60 con 0x7fb11c0aa400 2020-11-20T12:09:20.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11affd700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb11c0aa020 0x7fb11c146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:20.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb11bfff700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c0aa400 0x7fb11c065180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb114009c10 tx=0x7fb114016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:20.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb0fb7fe700 1 -- 172.21.15.44:0/2720010426 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11401c5f0 con 0x7fb11c0aa400 2020-11-20T12:09:20.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb0fb7fe700 1 -- 172.21.15.44:0/2720010426 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11401c750 con 0x7fb11c0aa400 2020-11-20T12:09:20.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c09d470 con 0x7fb11c0aa400 2020-11-20T12:09:20.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb0fb7fe700 1 -- 172.21.15.44:0/2720010426 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11401cad0 con 0x7fb11c0aa400 2020-11-20T12:09:20.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.907+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0658b0 con 0x7fb11c0aa400 2020-11-20T12:09:20.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.908+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c147960 con 0x7fb11c0aa400 2020-11-20T12:09:20.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.909+0000 7fb0fb7fe700 1 -- 172.21.15.44:0/2720010426 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb114024030 con 0x7fb11c0aa400 2020-11-20T12:09:20.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.909+0000 7fb0fb7fe700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0f4056d20 0x7fb0f40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:20.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.909+0000 7fb0fb7fe700 1 -- 172.21.15.44:0/2720010426 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb1140724d0 con 0x7fb11c0aa400 2020-11-20T12:09:20.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.910+0000 7fb11b7fe700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0f4056d20 0x7fb0f40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:20.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.910+0000 7fb11b7fe700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0f4056d20 0x7fb0f40591e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb10c003740 tx=0x7fb10c0079e0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:20.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:20.912+0000 7fb0fb7fe700 1 -- 172.21.15.44:0/2720010426 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb114047880 con 0x7fb11c0aa400 2020-11-20T12:09:21.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.151+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb11c061650 con 0x7fb11c0aa400 2020-11-20T12:09:21.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.151+0000 7fb0fb7fe700 1 -- 172.21.15.44:0/2720010426 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fb11402c080 con 0x7fb11c0aa400 2020-11-20T12:09:21.156 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:21.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0f4056d20 msgr2=0x7fb0f40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0f4056d20 0x7fb0f40591e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb10c003740 tx=0x7fb10c0079e0).stop 2020-11-20T12:09:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c0aa400 msgr2=0x7fb11c065180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c0aa400 0x7fb11c065180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fb114009c10 tx=0x7fb114016000).stop 2020-11-20T12:09:21.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 shutdown_connections 2020-11-20T12:09:21.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb0f4056d20 0x7fb0f40591e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb11c0aa400 0x7fb11c065180 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb11c0a3e10 0x7fb11c0664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 --2- 172.21.15.44:0/2720010426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb11c0aa020 0x7fb11c146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 >> 172.21.15.44:0/2720010426 conn(0x7fb11c062690 msgr2=0x7fb11c13a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 shutdown_connections 2020-11-20T12:09:21.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.153+0000 7fb121f97700 1 -- 172.21.15.44:0/2720010426 wait complete. 2020-11-20T12:09:21.168 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:09:21.168 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:09:21.169 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:09:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.291+0000 7ff453c95700 1 Processor -- start 2020-11-20T12:09:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.291+0000 7ff453c95700 1 -- start start 2020-11-20T12:09:21.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff453c95700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff44c0a3e10 0x7ff44c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff453c95700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0a9770 con 0x7ff44c0aa020 2020-11-20T12:09:21.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff453c95700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0a6da0 con 0x7ff44c0aa400 2020-11-20T12:09:21.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff453c95700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0a56c0 con 0x7ff44c0a3e10 2020-11-20T12:09:21.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff451a31700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff44c0aa020 0x7ff44c0a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42912/0 (socket says 172.21.15.44:42912) 2020-11-20T12:09:21.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff451a31700 1 -- 172.21.15.44:0/2544370571 learned_addr learned my addr 172.21.15.44:0/2544370571 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:21.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff452232700 1 --2- 172.21.15.44:0/2544370571 >> v2:172.21.15.79:3300/0 conn(0x7ff44c0a3e10 0x7ff44c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff450a2f700 1 -- 172.21.15.44:0/2544370571 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751194376 0 0) 0x7ff4400010d0 con 0x7ff44c0aa020 2020-11-20T12:09:21.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff450a2f700 1 -- 172.21.15.44:0/2544370571 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff434003240 con 0x7ff44c0aa020 2020-11-20T12:09:21.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff452232700 1 -- 172.21.15.44:0/2544370571 >> v1:172.21.15.79:6789/0 conn(0x7ff44c0aa400 legacy=0x7ff44c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.292+0000 7ff452232700 1 -- 172.21.15.44:0/2544370571 >> v1:172.21.15.44:6789/0 conn(0x7ff44c0aa020 legacy=0x7ff44c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff452232700 1 -- 172.21.15.44:0/2544370571 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0a7040 con 0x7ff44c0a3e10 2020-11-20T12:09:21.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff452232700 1 --2- 172.21.15.44:0/2544370571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0a3e10 0x7ff44c0a9280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff444009c90 tx=0x7ff4440091d0).ready entity=mon.1 client_cookie=9180743f4aa079a6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff450a2f700 1 -- 172.21.15.44:0/2544370571 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400ba70 con 0x7ff44c0a3e10 2020-11-20T12:09:21.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff450a2f700 1 -- 172.21.15.44:0/2544370571 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400bbd0 con 0x7ff44c0a3e10 2020-11-20T12:09:21.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff450a2f700 1 -- 172.21.15.44:0/2544370571 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444011400 con 0x7ff44c0a3e10 2020-11-20T12:09:21.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff453c95700 1 -- 172.21.15.44:0/2544370571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0a3e10 msgr2=0x7ff44c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff453c95700 1 --2- 172.21.15.44:0/2544370571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0a3e10 0x7ff44c0a9280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff444009c90 tx=0x7ff4440091d0).stop 2020-11-20T12:09:21.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff453c95700 1 -- 172.21.15.44:0/2544370571 shutdown_connections 2020-11-20T12:09:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff453c95700 1 --2- 172.21.15.44:0/2544370571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0a3e10 0x7ff44c0a9280 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff453c95700 1 -- 172.21.15.44:0/2544370571 >> 172.21.15.44:0/2544370571 conn(0x7ff44c062690 msgr2=0x7ff44c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff453c95700 1 -- 172.21.15.44:0/2544370571 shutdown_connections 2020-11-20T12:09:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.293+0000 7ff453c95700 1 -- 172.21.15.44:0/2544370571 wait complete. 2020-11-20T12:09:21.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.294+0000 7ff453c95700 1 Processor -- start 2020-11-20T12:09:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.294+0000 7ff453c95700 1 -- start start 2020-11-20T12:09:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff453c95700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0a3e10 0x7ff44c066060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff453c95700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0aa020 0x7ff44c0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff453c95700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff44c0aa400 0x7ff44c14a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff453c95700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0653d0 con 0x7ff44c0a3e10 2020-11-20T12:09:21.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff453c95700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0680a0 con 0x7ff44c0aa020 2020-11-20T12:09:21.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff453c95700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0abbb0 con 0x7ff44c0aa400 2020-11-20T12:09:21.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0a3e10 0x7ff44c066060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0a3e10 0x7ff44c066060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39168/0 (socket says 172.21.15.44:39168) 2020-11-20T12:09:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 -- 172.21.15.44:0/4282792169 learned_addr learned my addr 172.21.15.44:0/4282792169 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451230700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0aa020 0x7ff44c0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff452232700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff44c0aa400 0x7ff44c14a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff44c0aa400 msgr2=0x7ff44c14a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff44c0aa400 0x7ff44c14a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0aa020 msgr2=0x7ff44c0ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff44c0aa020 0x7ff44c0ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 -- 172.21.15.44:0/4282792169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0abde0 con 0x7ff44c0a3e10 2020-11-20T12:09:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.295+0000 7ff451a31700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0a3e10 0x7ff44c066060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7ff44000c6f0 tx=0x7ff440007940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.296+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400072e0 con 0x7ff44c0a3e10 2020-11-20T12:09:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.296+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff440007440 con 0x7ff44c0a3e10 2020-11-20T12:09:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.296+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c0ac080 con 0x7ff44c0a3e10 2020-11-20T12:09:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.296+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440005820 con 0x7ff44c0a3e10 2020-11-20T12:09:21.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.296+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0ab1b0 con 0x7ff44c0a3e10 2020-11-20T12:09:21.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff453c95700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0ac210 0x7ff44c14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0ad890 con 0x7ff44c0ac210 2020-11-20T12:09:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff451230700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0ac210 0x7ff44c14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff440010070 con 0x7ff44c0a3e10 2020-11-20T12:09:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff451230700 1 -- 172.21.15.44:0/4282792169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff44c0a4780 con 0x7ff44c0ac210 2020-11-20T12:09:21.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff451230700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0ac210 0x7ff44c14a760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7ff44c065bd0 tx=0x7ff448007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff43dffb700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff424056d20 0x7ff4240591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff44006f6e0 con 0x7ff44c0a3e10 2020-11-20T12:09:21.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff452232700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff424056d20 0x7ff4240591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.297+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011650 con 0x7ff44c0ac210 2020-11-20T12:09:21.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.298+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7ff4480117b0 con 0x7ff44c0ac210 2020-11-20T12:09:21.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.298+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0ac210 msgr2=0x7ff44c14a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.298+0000 7ff43dffb700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0ac210 0x7ff44c14a760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7ff44c065bd0 tx=0x7ff448007b90).stop 2020-11-20T12:09:21.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.298+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0ac210 msgr2=0x7ff44c14a760 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:21.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.298+0000 7ff43dffb700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0ac210 0x7ff44c14a760 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0x7ff44c065bd0 tx=0x7ff448007b90).stop 2020-11-20T12:09:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.298+0000 7ff452232700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff424056d20 0x7ff4240591e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff444006db0 tx=0x7ff444016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.305+0000 7ff453c95700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c14ac50 0x7ff44c146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0a9770 con 0x7ff44c14ac50 2020-11-20T12:09:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff451230700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c14ac50 0x7ff44c146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff451230700 1 -- 172.21.15.44:0/4282792169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff44c061650 con 0x7ff44c14ac50 2020-11-20T12:09:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff451230700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c14ac50 0x7ff44c146ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff44800a000 tx=0x7ff448016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4480150d0 con 0x7ff44c14ac50 2020-11-20T12:09:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7ff448015230 con 0x7ff44c14ac50 2020-11-20T12:09:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c14ac50 msgr2=0x7ff44c146ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff43dffb700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c14ac50 0x7ff44c146ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff44800a000 tx=0x7ff448016000).stop 2020-11-20T12:09:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff43dffb700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c14ac50 msgr2=0x7ff44c146ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.306+0000 7ff43dffb700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c14ac50 0x7ff44c146ca0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0x7ff44800a000 tx=0x7ff448016000).stop 2020-11-20T12:09:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff424056d20 msgr2=0x7ff4240591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff424056d20 0x7ff4240591e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff444006db0 tx=0x7ff444016000).stop 2020-11-20T12:09:21.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0a3e10 msgr2=0x7ff44c066060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 --2- 172.21.15.44:0/4282792169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff44c0a3e10 0x7ff44c066060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7ff44000c6f0 tx=0x7ff440007940).stop 2020-11-20T12:09:21.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff452232700 1 -- 172.21.15.44:0/4282792169 reap_dead start 2020-11-20T12:09:21.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff452232700 1 -- 172.21.15.44:0/4282792169 reap_dead start 2020-11-20T12:09:21.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 shutdown_connections 2020-11-20T12:09:21.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 >> 172.21.15.44:0/4282792169 conn(0x7ff44c062690 msgr2=0x7ff44c139e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 shutdown_connections 2020-11-20T12:09:21.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.307+0000 7ff453c95700 1 -- 172.21.15.44:0/4282792169 wait complete. 2020-11-20T12:09:21.320 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:21.320 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:21.321 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:21.321 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:09:21.321 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 62, 2020-11-20T12:09:21.321 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:21.322 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:09:21.322 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:21.322 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:21.322 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.322 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 11, 2020-11-20T12:09:21.323 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:21.323 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:21.323 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:21.323 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.323 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.324 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.324 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.324 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.325 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.325 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.325 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:21.325 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:21.326 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.326 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.326 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.326 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.327 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.327 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.327 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.327 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.328 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:21.328 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:21.328 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:21.328 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:21.328 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:21.329 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:21.329 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:21.329 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:21.329 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:21.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:21.330 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:21.330 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:21.330 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:21.330 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:21.331 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:21.331 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.331 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.331 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.332 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.332 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.332 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.332 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.332 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.333 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:21.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.333 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:21.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.333 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:21.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:21.334 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.334 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.335 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:21.335 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.336 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:21.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.336 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.337 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:21.337 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:21.337 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.337 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.338 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.338 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:21.338 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:21.339 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.339 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.340 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:21.340 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.341 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:21.341 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.341 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.342 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:21.342 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:21.342 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.342 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.342 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.343 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:21.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:21.343 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.344 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.344 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:21.344 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.345 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:21.346 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.346 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.346 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.347 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:21.347 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:21.347 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.347 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.348 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.348 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:21.349 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:21.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.349 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.349 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.349 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.350 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.350 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.350 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:09:21.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.351 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.351 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.352 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.352 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.352 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:09:21.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.353 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.353 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.353 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.354 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.354 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:21.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.354 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.354 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.355 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:09:21.355 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.355 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.355 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:09:21.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.356 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.356 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.356 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.357 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.357 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:21.357 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:21.358 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:09:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.481+0000 7f429e46b700 1 Processor -- start 2020-11-20T12:09:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f429e46b700 1 -- start start 2020-11-20T12:09:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f429e46b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f42980a3e10 0x7f42980a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f429e46b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42980a9770 con 0x7f42980aa020 2020-11-20T12:09:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f429e46b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42980a6da0 con 0x7f42980aa400 2020-11-20T12:09:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f429e46b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42980a56c0 con 0x7f42980a3e10 2020-11-20T12:09:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f429ca08700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f42980a3e10 0x7f42980a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f4297fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f42980aa020 0x7f42980a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42928/0 (socket says 172.21.15.44:42928) 2020-11-20T12:09:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.482+0000 7f4297fff700 1 -- 172.21.15.44:0/1452772872 learned_addr learned my addr 172.21.15.44:0/1452772872 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429ca08700 1 -- 172.21.15.44:0/1452772872 >> v1:172.21.15.79:6789/0 conn(0x7f42980aa400 legacy=0x7f42980a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429ca08700 1 -- 172.21.15.44:0/1452772872 >> v1:172.21.15.44:6789/0 conn(0x7f42980aa020 legacy=0x7f42980a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f4296ffd700 1 -- 172.21.15.44:0/1452772872 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100252966 0 0) 0x7f42980a6da0 con 0x7f42980aa400 2020-11-20T12:09:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429ca08700 1 -- 172.21.15.44:0/1452772872 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42980a7040 con 0x7f42980a3e10 2020-11-20T12:09:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429ca08700 1 --2- 172.21.15.44:0/1452772872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980a3e10 0x7f42980a9280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f4290009c90 tx=0x7f42900091d0).ready entity=mon.0 client_cookie=cceb4ceca464a48c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f4296ffd700 1 -- 172.21.15.44:0/1452772872 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000bac0 con 0x7f42980a3e10 2020-11-20T12:09:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f4296ffd700 1 -- 172.21.15.44:0/1452772872 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429000bc20 con 0x7f42980a3e10 2020-11-20T12:09:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f4296ffd700 1 -- 172.21.15.44:0/1452772872 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290011400 con 0x7f42980a3e10 2020-11-20T12:09:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429e46b700 1 -- 172.21.15.44:0/1452772872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980a3e10 msgr2=0x7f42980a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429e46b700 1 --2- 172.21.15.44:0/1452772872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980a3e10 0x7f42980a9280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f4290009c90 tx=0x7f42900091d0).stop 2020-11-20T12:09:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429e46b700 1 -- 172.21.15.44:0/1452772872 shutdown_connections 2020-11-20T12:09:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429e46b700 1 --2- 172.21.15.44:0/1452772872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980a3e10 0x7f42980a9280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429e46b700 1 -- 172.21.15.44:0/1452772872 >> 172.21.15.44:0/1452772872 conn(0x7f4298062690 msgr2=0x7f42980a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429e46b700 1 -- 172.21.15.44:0/1452772872 shutdown_connections 2020-11-20T12:09:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.483+0000 7f429e46b700 1 -- 172.21.15.44:0/1452772872 wait complete. 2020-11-20T12:09:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.484+0000 7f429e46b700 1 Processor -- start 2020-11-20T12:09:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429e46b700 1 -- start start 2020-11-20T12:09:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429e46b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980a3e10 0x7f4298066390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429e46b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980aa020 0x7f4298146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429e46b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f42980aa400 0x7f42980aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429e46b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298065350 con 0x7f42980aa020 2020-11-20T12:09:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429e46b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42980655f0 con 0x7f42980a3e10 2020-11-20T12:09:21.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429e46b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4298065890 con 0x7f42980aa400 2020-11-20T12:09:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f42977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980aa020 0x7f4298146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f4297fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980a3e10 0x7f4298066390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f4297fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980a3e10 0x7f4298066390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59468/0 (socket says 172.21.15.44:59468) 2020-11-20T12:09:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f4297fff700 1 -- 172.21.15.44:0/2723349533 learned_addr learned my addr 172.21.15.44:0/2723349533 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f42977fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980aa020 0x7f4298146cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39188/0 (socket says 172.21.15.44:39188) 2020-11-20T12:09:21.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.485+0000 7f429ca08700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f42980aa400 0x7f42980aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f42977fe700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f42980aa400 msgr2=0x7f42980aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f42977fe700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f42980aa400 0x7f42980aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f42977fe700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980a3e10 msgr2=0x7f4298066390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f42977fe700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980a3e10 0x7f4298066390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f42977fe700 1 -- 172.21.15.44:0/2723349533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298067f40 con 0x7f42980aa020 2020-11-20T12:09:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f4297fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980a3e10 0x7f4298066390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f42977fe700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980aa020 0x7f4298146cb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f428c00c760 tx=0x7f428c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c007020 con 0x7f42980aa020 2020-11-20T12:09:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c007180 con 0x7f42980aa020 2020-11-20T12:09:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42981471a0 con 0x7f42980aa020 2020-11-20T12:09:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c0064f0 con 0x7f42980aa020 2020-11-20T12:09:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.486+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4298065f70 con 0x7f42980aa020 2020-11-20T12:09:21.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.487+0000 7f429e46b700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980ab380 0x7f429812f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.487+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42980681f0 con 0x7f42980ab380 2020-11-20T12:09:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.487+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f428c01a060 con 0x7f42980aa020 2020-11-20T12:09:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.487+0000 7f4297fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980ab380 0x7f429812f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.487+0000 7f4277fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4270056d20 0x7f42700591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.487+0000 7f4297fff700 1 -- 172.21.15.44:0/2723349533 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4298065a50 con 0x7f42980ab380 2020-11-20T12:09:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f428c06f530 con 0x7f42980aa020 2020-11-20T12:09:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f429ca08700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4270056d20 0x7f42700591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4297fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980ab380 0x7f429812f030 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f428800d150 tx=0x7f4288009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880070d0 con 0x7f42980ab380 2020-11-20T12:09:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f429ca08700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4270056d20 0x7f42700591e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f4290009e70 tx=0x7f4290016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4288007230 con 0x7f42980ab380 2020-11-20T12:09:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980ab380 msgr2=0x7f429812f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4277fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980ab380 0x7f429812f030 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f428800d150 tx=0x7f4288009040).stop 2020-11-20T12:09:21.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980ab380 msgr2=0x7f429812f030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:21.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.488+0000 7f4277fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42980ab380 0x7f429812f030 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.496+0000 7f429e46b700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429812f520 0x7f429814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.496+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298061650 con 0x7f429812f520 2020-11-20T12:09:21.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.496+0000 7f4297fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429812f520 0x7f429814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.496+0000 7f4297fff700 1 -- 172.21.15.44:0/2723349533 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f42980446d0 con 0x7f429812f520 2020-11-20T12:09:21.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f4297fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429812f520 0x7f429814a280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4288015000 tx=0x7f4288016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288014890 con 0x7f429812f520 2020-11-20T12:09:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f428800dd00 con 0x7f429812f520 2020-11-20T12:09:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429812f520 msgr2=0x7f429814a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f4277fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429812f520 0x7f429814a280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f4288015000 tx=0x7f4288016000).stop 2020-11-20T12:09:21.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f4277fff700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429812f520 msgr2=0x7f429814a280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:21.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f4277fff700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429812f520 0x7f429814a280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4270056d20 msgr2=0x7f42700591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429e46b700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4270056d20 0x7f42700591e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f4290009e70 tx=0x7f4290016000).stop 2020-11-20T12:09:21.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980aa020 msgr2=0x7f4298146cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429e46b700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980aa020 0x7f4298146cb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f428c00c760 tx=0x7f428c007b90).stop 2020-11-20T12:09:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429ca08700 1 -- 172.21.15.44:0/2723349533 reap_dead start 2020-11-20T12:09:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 shutdown_connections 2020-11-20T12:09:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429e46b700 1 --2- 172.21.15.44:0/2723349533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42980aa020 0x7f4298146cb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.497+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 >> 172.21.15.44:0/2723349533 conn(0x7f4298062690 msgr2=0x7f4298134660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.498+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 shutdown_connections 2020-11-20T12:09:21.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.498+0000 7f429e46b700 1 -- 172.21.15.44:0/2723349533 wait complete. 2020-11-20T12:09:21.515 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:21.515 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:21.516 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:21.516 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:09:21.516 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 62, 2020-11-20T12:09:21.516 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:21.517 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:09:21.517 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:21.517 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:21.517 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.518 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 11, 2020-11-20T12:09:21.518 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:21.518 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:21.519 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:21.519 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.519 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.519 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.520 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.520 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.520 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.520 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.521 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.521 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.521 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:21.521 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:21.521 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.522 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.522 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.522 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.522 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.523 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.523 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.523 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.523 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.524 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:21.524 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:21.524 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:21.525 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:21.525 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:21.525 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:21.525 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:21.526 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:21.526 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:21.526 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:21.526 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:21.526 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:21.527 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:21.527 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:21.527 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:21.527 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.528 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.528 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.528 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.529 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.529 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.529 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.529 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.530 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.530 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:21.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.530 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:21.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.531 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:21.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:21.531 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.531 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.532 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:21.532 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.533 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.533 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:21.533 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.534 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.534 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:21.534 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:21.535 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.535 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.535 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.536 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:21.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:21.536 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.537 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.538 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:21.538 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.539 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:21.539 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.539 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.539 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.540 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:21.540 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:21.540 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.540 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.541 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.541 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:21.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:21.542 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.542 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.543 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:21.543 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.544 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:21.544 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.544 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.545 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:21.545 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:21.545 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.546 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.546 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.546 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.547 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:21.547 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:21.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.547 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.548 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.548 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.548 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:21.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.549 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.549 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.549 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.550 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.550 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:21.550 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:21.552 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:09:21.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.675+0000 7f4aa5bb1700 1 Processor -- start 2020-11-20T12:09:21.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.675+0000 7f4aa5bb1700 1 -- start start 2020-11-20T12:09:21.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4aa5bb1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa00aa020 0x7f4aa00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4aa5bb1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4aa00aa400 0x7f4aa00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4aa5bb1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa00a9770 con 0x7f4aa013ac60 2020-11-20T12:09:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4aa5bb1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa00a6da0 con 0x7f4aa00aa400 2020-11-20T12:09:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4aa5bb1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00a7040 con 0x7f4aa00aa020 2020-11-20T12:09:21.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9ffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f4aa013ac60 0x7f4aa00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:42946/0 (socket says 172.21.15.44:42946) 2020-11-20T12:09:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9ffff700 1 -- 172.21.15.44:0/1785828321 learned_addr learned my addr 172.21.15.44:0/1785828321 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9f7fe700 1 --2- 172.21.15.44:0/1785828321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa00aa020 0x7f4aa00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9effd700 1 --2- 172.21.15.44:0/1785828321 >> v2:172.21.15.79:3300/0 conn(0x7f4aa00aa400 0x7f4aa00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9e7fc700 1 -- 172.21.15.44:0/1785828321 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335669831 0 0) 0x7f4a98001450 con 0x7f4aa013ac60 2020-11-20T12:09:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9e7fc700 1 -- 172.21.15.44:0/1785828321 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a8c003240 con 0x7f4aa013ac60 2020-11-20T12:09:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9f7fe700 1 -- 172.21.15.44:0/1785828321 >> v2:172.21.15.79:3300/0 conn(0x7f4aa00aa400 msgr2=0x7f4aa00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:09:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9f7fe700 1 --2- 172.21.15.44:0/1785828321 >> v2:172.21.15.79:3300/0 conn(0x7f4aa00aa400 0x7f4aa00a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9f7fe700 1 -- 172.21.15.44:0/1785828321 >> v1:172.21.15.44:6789/0 conn(0x7f4aa013ac60 legacy=0x7f4aa00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9f7fe700 1 -- 172.21.15.44:0/1785828321 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa00a6a90 con 0x7f4aa00aa020 2020-11-20T12:09:21.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9effd700 1 --2- 172.21.15.44:0/1785828321 >> v2:172.21.15.79:3300/0 conn(0x7f4aa00aa400 0x7f4aa00a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.676+0000 7f4a9f7fe700 1 --2- 172.21.15.44:0/1785828321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa00aa020 0x7f4aa00a9280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4a9000b9a0 tx=0x7f4a90006df0).ready entity=mon.2 client_cookie=f441d68bcbd57527 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4a9e7fc700 1 -- 172.21.15.44:0/1785828321 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90009680 con 0x7f4aa00aa020 2020-11-20T12:09:21.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4a9e7fc700 1 -- 172.21.15.44:0/1785828321 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a900097e0 con 0x7f4aa00aa020 2020-11-20T12:09:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4a9e7fc700 1 -- 172.21.15.44:0/1785828321 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90009b60 con 0x7f4aa00aa020 2020-11-20T12:09:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/1785828321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa00aa020 msgr2=0x7f4aa00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 --2- 172.21.15.44:0/1785828321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa00aa020 0x7f4aa00a9280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4a9000b9a0 tx=0x7f4a90006df0).stop 2020-11-20T12:09:21.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/1785828321 shutdown_connections 2020-11-20T12:09:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 --2- 172.21.15.44:0/1785828321 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa00aa020 0x7f4aa00a9280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 --2- 172.21.15.44:0/1785828321 >> v2:172.21.15.79:3300/0 conn(0x7f4aa00aa400 0x7f4aa00a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/1785828321 >> 172.21.15.44:0/1785828321 conn(0x7f4aa0062690 msgr2=0x7f4aa00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/1785828321 shutdown_connections 2020-11-20T12:09:21.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.677+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/1785828321 wait complete. 2020-11-20T12:09:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.678+0000 7f4aa5bb1700 1 Processor -- start 2020-11-20T12:09:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.678+0000 7f4aa5bb1700 1 -- start start 2020-11-20T12:09:21.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4aa5bb1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aa00aa020 0x7f4aa0067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4aa5bb1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4aa00aa400 0x7f4aa0134900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4aa5bb1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa013ac60 0x7f4aa012a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4aa5bb1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00ade50 con 0x7f4aa00aa400 2020-11-20T12:09:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4aa5bb1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00ac240 con 0x7f4aa00aa020 2020-11-20T12:09:21.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4aa5bb1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0066500 con 0x7f4aa013ac60 2020-11-20T12:09:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4aa00aa400 0x7f4aa0134900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aa00aa020 0x7f4aa0067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa013ac60 0x7f4aa012a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aa00aa020 0x7f4aa0067750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59484/0 (socket says 172.21.15.44:59484) 2020-11-20T12:09:21.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4aa00aa400 0x7f4aa0134900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39206/0 (socket says 172.21.15.44:39206) 2020-11-20T12:09:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 -- 172.21.15.44:0/3355575654 learned_addr learned my addr 172.21.15.44:0/3355575654 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa013ac60 msgr2=0x7f4aa012a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa013ac60 0x7f4aa012a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4aa00aa400 msgr2=0x7f4aa0134900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4aa00aa400 0x7f4aa0134900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 -- 172.21.15.44:0/3355575654 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa0066f70 con 0x7f4aa00aa020 2020-11-20T12:09:21.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9effd700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4aa00aa400 0x7f4aa0134900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:21.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9ffff700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa013ac60 0x7f4aa012a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:21.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.679+0000 7f4a9f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aa00aa020 0x7f4aa0067750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4a90006e30 tx=0x7f4a90015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.680+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90011bc0 con 0x7f4aa00aa020 2020-11-20T12:09:21.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.680+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a90011d20 con 0x7f4aa00aa020 2020-11-20T12:09:21.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.680+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9001e670 con 0x7f4aa00aa020 2020-11-20T12:09:21.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.680+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa00660d0 con 0x7f4aa00aa020 2020-11-20T12:09:21.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.680+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa0134df0 con 0x7f4aa00aa020 2020-11-20T12:09:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.680+0000 7f4aa5bb1700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa012ac10 0x7f4aa014b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.680+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00656d0 con 0x7f4aa012ac10 2020-11-20T12:09:21.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a9effd700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa012ac10 0x7f4aa014b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a9effd700 1 -- 172.21.15.44:0/3355575654 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4aa0142e90 con 0x7f4aa012ac10 2020-11-20T12:09:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4a90025030 con 0x7f4aa00aa020 2020-11-20T12:09:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a9effd700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa012ac10 0x7f4aa014b360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f4aa0062aa0 tx=0x7f4a94006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a78056d50 0x7f4a78059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4a900713a0 con 0x7f4aa00aa020 2020-11-20T12:09:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a9ffff700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a78056d50 0x7f4a78059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94013670 con 0x7f4aa012ac10 2020-11-20T12:09:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4a940137d0 con 0x7f4aa012ac10 2020-11-20T12:09:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa012ac10 msgr2=0x7f4aa014b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa012ac10 0x7f4aa014b360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f4aa0062aa0 tx=0x7f4a94006df0).stop 2020-11-20T12:09:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa012ac10 msgr2=0x7f4aa014b360 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.681+0000 7f4a7f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa012ac10 0x7f4aa014b360 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.682+0000 7f4a9ffff700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a78056d50 0x7f4a78059210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f4a980015b0 tx=0x7f4a98009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.689+0000 7f4aa5bb1700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa014b850 0x7f4aa014bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.689+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0061650 con 0x7f4aa014b850 2020-11-20T12:09:21.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.689+0000 7f4a9effd700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa014b850 0x7f4aa014bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a9effd700 1 -- 172.21.15.44:0/3355575654 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4aa00446d0 con 0x7f4aa014b850 2020-11-20T12:09:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a9effd700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa014b850 0x7f4aa014bc30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f4aa012a6a0 tx=0x7f4a94019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401ddb0 con 0x7f4aa014b850 2020-11-20T12:09:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f4a94027070 con 0x7f4aa014b850 2020-11-20T12:09:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa014b850 msgr2=0x7f4aa014bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a7f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa014b850 0x7f4aa014bc30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f4aa012a6a0 tx=0x7f4a94019000).stop 2020-11-20T12:09:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a7f7fe700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa014b850 msgr2=0x7f4aa014bc30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.690+0000 7f4a7f7fe700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4aa014b850 0x7f4aa014bc30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a78056d50 msgr2=0x7f4a78059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a78056d50 0x7f4a78059210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f4a980015b0 tx=0x7f4a98009040).stop 2020-11-20T12:09:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aa00aa020 msgr2=0x7f4aa0067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 --2- 172.21.15.44:0/3355575654 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aa00aa020 0x7f4aa0067750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4a90006e30 tx=0x7f4a90015000).stop 2020-11-20T12:09:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4a9ffff700 1 -- 172.21.15.44:0/3355575654 reap_dead start 2020-11-20T12:09:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4a9ffff700 1 -- 172.21.15.44:0/3355575654 reap_dead start 2020-11-20T12:09:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 shutdown_connections 2020-11-20T12:09:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 >> 172.21.15.44:0/3355575654 conn(0x7f4aa0062690 msgr2=0x7f4aa0138ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 shutdown_connections 2020-11-20T12:09:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.691+0000 7f4aa5bb1700 1 -- 172.21.15.44:0/3355575654 wait complete. 2020-11-20T12:09:21.711 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:21.711 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:21.712 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:21.712 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:09:21.712 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 62, 2020-11-20T12:09:21.712 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:21.712 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:09:21.713 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:21.713 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:21.713 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.713 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:09:21.713 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:21.714 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:21.714 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:21.714 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.714 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.715 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.715 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.715 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.715 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.716 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.716 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.716 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:21.716 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:21.716 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.717 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.717 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.717 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.718 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.718 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.718 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.719 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.719 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:21.719 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:21.720 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:21.720 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:21.720 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:21.720 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:21.720 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:21.721 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:21.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:21.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:21.721 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:21.722 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:21.722 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:21.722 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:21.722 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:21.723 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:21.723 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:21.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:21.723 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:21.724 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.725 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.728 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.730 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:21.731 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.733 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.734 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:21.736 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:21.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:21.740 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:21.740 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:21.741 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:21.741 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:21.741 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.741 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.742 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:21.742 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:21.742 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:21.743 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:21.743 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:21.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.743 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.744 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:21.744 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:21.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.744 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.744 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.745 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.745 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.745 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:09:21.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.746 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.746 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.746 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.746 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.746 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:09:21.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.747 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.747 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.748 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:09:21.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.748 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.748 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:21.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.749 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.749 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.749 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:09:21.750 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.750 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:21.750 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:09:21.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:21.751 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:21.751 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:21.751 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:21.751 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:21.751 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:21.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:21.752 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:21.752 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:21.753 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:09:21.754 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:09:21.754 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:09:21.754 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:09:21.755 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:09:21.755 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:09:21.776 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:09:21.776 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:09:21.777 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.964+0000 7f1e9ab1a700 1 Processor -- start 2020-11-20T12:09:21.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.964+0000 7f1e9ab1a700 1 -- start start 2020-11-20T12:09:21.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e9ab1a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9413c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e9ab1a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1e94066250 0x7f1e9413c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e9ab1a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e940a8bf0 con 0x7f1e940aa1e0 2020-11-20T12:09:21.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e9ab1a700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e940a86b0 con 0x7f1e94066250 2020-11-20T12:09:21.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e9ab1a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940a8950 con 0x7f1e940a7810 2020-11-20T12:09:21.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e988b6700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f1e940aa1e0 legacy=0x7f1e94065b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:21.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e990b7700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f1e94066250 msgr2=0x7f1e9413c660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e990b7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1e94066250 0x7f1e9413c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9413c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9413c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50672/0 (socket says 172.21.15.44:50672) 2020-11-20T12:09:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 -- 172.21.15.44:0/1312280027 learned_addr learned my addr 172.21.15.44:0/1312280027 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:21.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 -- 172.21.15.44:0/1312280027 >> v2:172.21.15.44:3300/0 conn(0x7f1e94066250 msgr2=0x7f1e9413c660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 --2- 172.21.15.44:0/1312280027 >> v2:172.21.15.44:3300/0 conn(0x7f1e94066250 0x7f1e9413c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 -- 172.21.15.44:0/1312280027 >> v1:172.21.15.44:6789/0 conn(0x7f1e940aa1e0 legacy=0x7f1e94065b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 -- 172.21.15.44:0/1312280027 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940aa5c0 con 0x7f1e940a7810 2020-11-20T12:09:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.965+0000 7f1e93fff700 1 --2- 172.21.15.44:0/1312280027 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9413c170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1e88006fd0 tx=0x7f1e88006eb0).ready entity=mon.2 client_cookie=61e0f9a26ac9871 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e937fe700 1 -- 172.21.15.44:0/1312280027 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88009890 con 0x7f1e940a7810 2020-11-20T12:09:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e937fe700 1 -- 172.21.15.44:0/1312280027 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e880099f0 con 0x7f1e940a7810 2020-11-20T12:09:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e937fe700 1 -- 172.21.15.44:0/1312280027 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88009d70 con 0x7f1e940a7810 2020-11-20T12:09:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/1312280027 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 msgr2=0x7f1e9413c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/1312280027 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9413c170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1e88006fd0 tx=0x7f1e88006eb0).stop 2020-11-20T12:09:21.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/1312280027 shutdown_connections 2020-11-20T12:09:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/1312280027 >> v2:172.21.15.44:3300/0 conn(0x7f1e94066250 0x7f1e9413c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/1312280027 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9413c170 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/1312280027 >> 172.21.15.44:0/1312280027 conn(0x7f1e94062690 msgr2=0x7f1e940a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/1312280027 shutdown_connections 2020-11-20T12:09:21.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.966+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/1312280027 wait complete. 2020-11-20T12:09:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.967+0000 7f1e9ab1a700 1 Processor -- start 2020-11-20T12:09:21.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.967+0000 7f1e9ab1a700 1 -- start start 2020-11-20T12:09:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e9ab1a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e94066250 0x7f1e94118350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e9ab1a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e9ab1a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e940aa1e0 0x7f1e9414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e9ab1a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94115510 con 0x7f1e940aa1e0 2020-11-20T12:09:21.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e9ab1a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941157b0 con 0x7f1e94066250 2020-11-20T12:09:21.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e9ab1a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94115a50 con 0x7f1e940a7810 2020-11-20T12:09:21.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e990b7700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e940aa1e0 msgr2=0x7f1e9414dd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:21.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e990b7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e940aa1e0 0x7f1e9414dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:21.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e988b6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e94066250 0x7f1e94118350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9414d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50678/0 (socket says 172.21.15.44:50678) 2020-11-20T12:09:21.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 -- 172.21.15.44:0/215244989 learned_addr learned my addr 172.21.15.44:0/215244989 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 -- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e94066250 msgr2=0x7f1e94118350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e94066250 0x7f1e94118350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 -- 172.21.15.44:0/215244989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e940aa1e0 msgr2=0x7f1e9414dd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e940aa1e0 0x7f1e9414dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 -- 172.21.15.44:0/215244989 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940a5f70 con 0x7f1e940a7810 2020-11-20T12:09:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e988b6700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e94066250 0x7f1e94118350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.968+0000 7f1e93fff700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9414d8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1e8800b1b0 tx=0x7f1e88015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.969+0000 7f1e90ff9700 1 -- 172.21.15.44:0/215244989 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8801e460 con 0x7f1e940a7810 2020-11-20T12:09:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.969+0000 7f1e90ff9700 1 -- 172.21.15.44:0/215244989 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8801e5c0 con 0x7f1e940a7810 2020-11-20T12:09:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.969+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e94116260 con 0x7f1e940a7810 2020-11-20T12:09:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.969+0000 7f1e90ff9700 1 -- 172.21.15.44:0/215244989 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8801e980 con 0x7f1e940a7810 2020-11-20T12:09:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.969+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e941152a0 con 0x7f1e940a7810 2020-11-20T12:09:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.970+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e940a6100 con 0x7f1e940a7810 2020-11-20T12:09:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.970+0000 7f1e90ff9700 1 -- 172.21.15.44:0/215244989 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1e8801a040 con 0x7f1e940a7810 2020-11-20T12:09:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.970+0000 7f1e90ff9700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e6c056cd0 0x7f1e6c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.970+0000 7f1e90ff9700 1 -- 172.21.15.44:0/215244989 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1e880716f0 con 0x7f1e940a7810 2020-11-20T12:09:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.970+0000 7f1e988b6700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e6c056cd0 0x7f1e6c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.971+0000 7f1e988b6700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e6c056cd0 0x7f1e6c059190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f1e8c003a20 tx=0x7f1e8c0091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:21.974+0000 7f1e90ff9700 1 -- 172.21.15.44:0/215244989 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1e88046a70 con 0x7f1e940a7810 2020-11-20T12:09:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.214+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1e941189c0 con 0x7f1e940a7810 2020-11-20T12:09:22.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.215+0000 7f1e90ff9700 1 -- 172.21.15.44:0/215244989 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f1e941189c0 con 0x7f1e940a7810 2020-11-20T12:09:22.219 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:22.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e6c056cd0 msgr2=0x7f1e6c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:22.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e6c056cd0 0x7f1e6c059190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f1e8c003a20 tx=0x7f1e8c0091d0).stop 2020-11-20T12:09:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 msgr2=0x7f1e9414d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9414d8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1e8800b1b0 tx=0x7f1e88015000).stop 2020-11-20T12:09:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 shutdown_connections 2020-11-20T12:09:22.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1e6c056cd0 0x7f1e6c059190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:22.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e940aa1e0 0x7f1e9414dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:22.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e94066250 0x7f1e94118350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:22.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 --2- 172.21.15.44:0/215244989 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1e940a7810 0x7f1e9414d8a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:22.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 >> 172.21.15.44:0/215244989 conn(0x7f1e94062690 msgr2=0x7f1e94065a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:22.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 shutdown_connections 2020-11-20T12:09:22.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:22.216+0000 7f1e9ab1a700 1 -- 172.21.15.44:0/215244989 wait complete. 2020-11-20T12:09:25.233 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:25.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.806+0000 7ffa0757b700 1 Processor -- start 2020-11-20T12:09:25.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.806+0000 7ffa0757b700 1 -- start start 2020-11-20T12:09:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.806+0000 7ffa0757b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa400 0x7ffa000a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.806+0000 7ffa0757b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000a9770 con 0x7ffa0013ac60 2020-11-20T12:09:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.806+0000 7ffa0757b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000a6da0 con 0x7ffa000aa020 2020-11-20T12:09:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.806+0000 7ffa0757b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000a56c0 con 0x7ffa000aa400 2020-11-20T12:09:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.806+0000 7ffa05b18700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7ffa0013ac60 legacy=0x7ffa000a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ffa04b16700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa400 0x7ffa000a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ffa05317700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ffa000aa020 0x7ffa000a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:45870/0 (socket says 172.21.15.44:45870) 2020-11-20T12:09:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ffa05317700 1 -- 172.21.15.44:0/4292882475 learned_addr learned my addr 172.21.15.44:0/4292882475 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ff9f3fff700 1 -- 172.21.15.44:0/4292882475 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476029601 0 0) 0x7ffa000a6da0 con 0x7ffa000aa020 2020-11-20T12:09:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ff9f3fff700 1 -- 172.21.15.44:0/4292882475 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e8003240 con 0x7ffa000aa020 2020-11-20T12:09:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ffa04b16700 1 -- 172.21.15.44:0/4292882475 >> v1:172.21.15.79:6789/0 conn(0x7ffa000aa020 legacy=0x7ffa000a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:25.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ffa04b16700 1 -- 172.21.15.44:0/4292882475 >> v1:172.21.15.44:6789/0 conn(0x7ffa0013ac60 legacy=0x7ffa000a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ffa04b16700 1 -- 172.21.15.44:0/4292882475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000a7040 con 0x7ffa000aa400 2020-11-20T12:09:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.807+0000 7ffa04b16700 1 --2- 172.21.15.44:0/4292882475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa400 0x7ffa000a9280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7ff9fc009cc0 tx=0x7ff9fc0091d0).ready entity=mon.2 client_cookie=5968de61021d4d59 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ff9f3fff700 1 -- 172.21.15.44:0/4292882475 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00ba90 con 0x7ffa000aa400 2020-11-20T12:09:25.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ff9f3fff700 1 -- 172.21.15.44:0/4292882475 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc00bbf0 con 0x7ffa000aa400 2020-11-20T12:09:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ff9f3fff700 1 -- 172.21.15.44:0/4292882475 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc011400 con 0x7ffa000aa400 2020-11-20T12:09:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ffa0757b700 1 -- 172.21.15.44:0/4292882475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa400 msgr2=0x7ffa000a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ffa0757b700 1 --2- 172.21.15.44:0/4292882475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa400 0x7ffa000a9280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7ff9fc009cc0 tx=0x7ff9fc0091d0).stop 2020-11-20T12:09:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ffa0757b700 1 -- 172.21.15.44:0/4292882475 shutdown_connections 2020-11-20T12:09:25.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ffa0757b700 1 --2- 172.21.15.44:0/4292882475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa400 0x7ffa000a9280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ffa0757b700 1 -- 172.21.15.44:0/4292882475 >> 172.21.15.44:0/4292882475 conn(0x7ffa00062690 msgr2=0x7ffa000a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ffa0757b700 1 -- 172.21.15.44:0/4292882475 shutdown_connections 2020-11-20T12:09:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.808+0000 7ffa0757b700 1 -- 172.21.15.44:0/4292882475 wait complete. 2020-11-20T12:09:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.809+0000 7ffa0757b700 1 Processor -- start 2020-11-20T12:09:25.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.809+0000 7ffa0757b700 1 -- start start 2020-11-20T12:09:25.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa020 0x7ffa000abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:25.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa000aa400 0x7ffa00139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:25.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0013ac60 0x7ffa000acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:25.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000ac510 con 0x7ffa0013ac60 2020-11-20T12:09:25.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000ac7b0 con 0x7ffa000aa400 2020-11-20T12:09:25.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000aca50 con 0x7ffa000aa020 2020-11-20T12:09:25.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa05b18700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0013ac60 msgr2=0x7ffa000acf10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:25.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa05b18700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0013ac60 0x7ffa000acf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:25.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa05317700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa020 0x7ffa000abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:25.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa000aa400 0x7ffa00139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:25.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa000aa400 0x7ffa00139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59534/0 (socket says 172.21.15.44:59534) 2020-11-20T12:09:25.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 -- 172.21.15.44:0/3437096557 learned_addr learned my addr 172.21.15.44:0/3437096557 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:25.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa05317700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa020 0x7ffa000abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50688/0 (socket says 172.21.15.44:50688) 2020-11-20T12:09:25.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 -- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa020 msgr2=0x7ffa000abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:25.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa020 0x7ffa000abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:25.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 -- 172.21.15.44:0/3437096557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0013ac60 msgr2=0x7ffa000acf10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:25.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0013ac60 0x7ffa000acf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:25.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 -- 172.21.15.44:0/3437096557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0009e940 con 0x7ffa000aa400 2020-11-20T12:09:25.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa05317700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa020 0x7ffa000abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:25.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa04b16700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa000aa400 0x7ffa00139cf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff9fc006f00 tx=0x7ff9fc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:25.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ff9f17fa700 1 -- 172.21.15.44:0/3437096557 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc011e60 con 0x7ffa000aa400 2020-11-20T12:09:25.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ff9f17fa700 1 -- 172.21.15.44:0/3437096557 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc01c5f0 con 0x7ffa000aa400 2020-11-20T12:09:25.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0009eb90 con 0x7ffa000aa400 2020-11-20T12:09:25.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ff9f17fa700 1 -- 172.21.15.44:0/3437096557 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01c970 con 0x7ffa000aa400 2020-11-20T12:09:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.811+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa000ad640 con 0x7ffa000aa400 2020-11-20T12:09:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.812+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa000acc10 con 0x7ffa000aa400 2020-11-20T12:09:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.812+0000 7ff9f17fa700 1 -- 172.21.15.44:0/3437096557 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff9fc024030 con 0x7ffa000aa400 2020-11-20T12:09:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.812+0000 7ff9f17fa700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff9d8056d00 0x7ff9d80591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.812+0000 7ff9f17fa700 1 -- 172.21.15.44:0/3437096557 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff9fc071a20 con 0x7ffa000aa400 2020-11-20T12:09:25.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.813+0000 7ffa05317700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff9d8056d00 0x7ff9d80591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:25.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.813+0000 7ffa05317700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff9d8056d00 0x7ff9d80591c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff9f40055a0 tx=0x7ff9f4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:25.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:25.818+0000 7ff9f17fa700 1 -- 172.21.15.44:0/3437096557 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff9fc046da0 con 0x7ffa000aa400 2020-11-20T12:09:26.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.060+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffa000a9770 con 0x7ffa000aa400 2020-11-20T12:09:26.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.061+0000 7ff9f17fa700 1 -- 172.21.15.44:0/3437096557 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7ffa000a9770 con 0x7ffa000aa400 2020-11-20T12:09:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff9d8056d00 msgr2=0x7ff9d80591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff9d8056d00 0x7ff9d80591c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff9f40055a0 tx=0x7ff9f4009040).stop 2020-11-20T12:09:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa000aa400 msgr2=0x7ffa00139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa000aa400 0x7ffa00139cf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff9fc006f00 tx=0x7ff9fc016000).stop 2020-11-20T12:09:26.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 shutdown_connections 2020-11-20T12:09:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff9d8056d00 0x7ff9d80591c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa0013ac60 0x7ffa000acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa000aa020 0x7ffa000abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:26.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 --2- 172.21.15.44:0/3437096557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa000aa400 0x7ffa00139cf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 >> 172.21.15.44:0/3437096557 conn(0x7ffa00062690 msgr2=0x7ffa00134570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 shutdown_connections 2020-11-20T12:09:26.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:26.063+0000 7ffa0757b700 1 -- 172.21.15.44:0/3437096557 wait complete. 2020-11-20T12:09:26.070 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:29.078 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:29.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.213+0000 7f34d388a700 1 Processor -- start 2020-11-20T12:09:29.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.213+0000 7f34d388a700 1 -- start start 2020-11-20T12:09:29.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d388a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0a3e10 0x7f34cc0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:29.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d388a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc0a9770 con 0x7f34cc0aa020 2020-11-20T12:09:29.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d388a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc0a6da0 con 0x7f34cc0aa400 2020-11-20T12:09:29.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d388a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0a56c0 con 0x7f34cc0a3e10 2020-11-20T12:09:29.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d1626700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f34cc0aa020 legacy=0x7f34cc0a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:29.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d0e25700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f34cc0aa400 0x7f34cc0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:45882/0 (socket says 172.21.15.44:45882) 2020-11-20T12:09:29.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d0e25700 1 -- 172.21.15.44:0/4158229182 learned_addr learned my addr 172.21.15.44:0/4158229182 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:29.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.214+0000 7f34d1e27700 1 --2- 172.21.15.44:0/4158229182 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0a3e10 0x7f34cc0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:29.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d1e27700 1 -- 172.21.15.44:0/4158229182 >> v1:172.21.15.79:6789/0 conn(0x7f34cc0aa400 legacy=0x7f34cc0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:29.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d1e27700 1 -- 172.21.15.44:0/4158229182 >> v1:172.21.15.44:6789/0 conn(0x7f34cc0aa020 legacy=0x7f34cc0a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:29.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d1e27700 1 -- 172.21.15.44:0/4158229182 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0a7040 con 0x7f34cc0a3e10 2020-11-20T12:09:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d1e27700 1 --2- 172.21.15.44:0/4158229182 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0a3e10 0x7f34cc0a9280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f34c400b9d0 tx=0x7f34c4006df0).ready entity=mon.2 client_cookie=3b71cfaa6c2b070f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34bbfff700 1 -- 172.21.15.44:0/4158229182 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4009830 con 0x7f34cc0a3e10 2020-11-20T12:09:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34bbfff700 1 -- 172.21.15.44:0/4158229182 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4009990 con 0x7f34cc0a3e10 2020-11-20T12:09:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34bbfff700 1 -- 172.21.15.44:0/4158229182 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4009d10 con 0x7f34cc0a3e10 2020-11-20T12:09:29.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d388a700 1 -- 172.21.15.44:0/4158229182 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0a3e10 msgr2=0x7f34cc0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d388a700 1 --2- 172.21.15.44:0/4158229182 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0a3e10 0x7f34cc0a9280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f34c400b9d0 tx=0x7f34c4006df0).stop 2020-11-20T12:09:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d388a700 1 -- 172.21.15.44:0/4158229182 shutdown_connections 2020-11-20T12:09:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d388a700 1 --2- 172.21.15.44:0/4158229182 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0a3e10 0x7f34cc0a9280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:29.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.215+0000 7f34d388a700 1 -- 172.21.15.44:0/4158229182 >> 172.21.15.44:0/4158229182 conn(0x7f34cc062690 msgr2=0x7f34cc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:29.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.216+0000 7f34d388a700 1 -- 172.21.15.44:0/4158229182 shutdown_connections 2020-11-20T12:09:29.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.216+0000 7f34d388a700 1 -- 172.21.15.44:0/4158229182 wait complete. 2020-11-20T12:09:29.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.216+0000 7f34d388a700 1 Processor -- start 2020-11-20T12:09:29.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.217+0000 7f34d388a700 1 -- start start 2020-11-20T12:09:29.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d388a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34cc0a3e10 0x7f34cc067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:29.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d388a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0aa020 0x7f34cc14a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:29.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d388a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34cc0aa400 0x7f34cc0abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:29.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d388a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0ab4e0 con 0x7f34cc0a3e10 2020-11-20T12:09:29.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d388a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0ab780 con 0x7f34cc0aa400 2020-11-20T12:09:29.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d388a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0aba20 con 0x7f34cc0aa020 2020-11-20T12:09:29.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d1626700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34cc0a3e10 msgr2=0x7f34cc067e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:29.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d1626700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34cc0a3e10 0x7f34cc067e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:29.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0aa020 0x7f34cc14a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:29.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d1e27700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34cc0aa400 0x7f34cc0abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0aa020 0x7f34cc14a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50704/0 (socket says 172.21.15.44:50704) 2020-11-20T12:09:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d1e27700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34cc0aa400 0x7f34cc0abec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59550/0 (socket says 172.21.15.44:59550) 2020-11-20T12:09:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 -- 172.21.15.44:0/3203658270 learned_addr learned my addr 172.21.15.44:0/3203658270 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 -- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34cc0aa400 msgr2=0x7f34cc0abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:29.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34cc0aa400 0x7f34cc0abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 -- 172.21.15.44:0/3203658270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34cc0a3e10 msgr2=0x7f34cc067e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34cc0a3e10 0x7f34cc067e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 -- 172.21.15.44:0/3203658270 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0ab160 con 0x7f34cc0aa020 2020-11-20T12:09:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.218+0000 7f34d0e25700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0aa020 0x7f34cc14a2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f34c800ca90 tx=0x7f34c8001460).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:29.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.219+0000 7f34b97fa700 1 -- 172.21.15.44:0/3203658270 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8005b80 con 0x7f34cc0aa020 2020-11-20T12:09:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.219+0000 7f34b97fa700 1 -- 172.21.15.44:0/3203658270 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c8005ce0 con 0x7f34cc0aa020 2020-11-20T12:09:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.219+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc14ad90 con 0x7f34cc0aa020 2020-11-20T12:09:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.219+0000 7f34b97fa700 1 -- 172.21.15.44:0/3203658270 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8007320 con 0x7f34cc0aa020 2020-11-20T12:09:29.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.219+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc0ac570 con 0x7f34cc0aa020 2020-11-20T12:09:29.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.220+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc0ab350 con 0x7f34cc0aa020 2020-11-20T12:09:29.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.220+0000 7f34b97fa700 1 -- 172.21.15.44:0/3203658270 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f34c8010030 con 0x7f34cc0aa020 2020-11-20T12:09:29.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.220+0000 7f34b97fa700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f34a4056cd0 0x7f34a4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:29.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.220+0000 7f34b97fa700 1 -- 172.21.15.44:0/3203658270 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f34c80701d0 con 0x7f34cc0aa020 2020-11-20T12:09:29.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.221+0000 7f34d1626700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f34a4056cd0 0x7f34a4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:29.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.221+0000 7f34d1626700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f34a4056cd0 0x7f34a4059190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f34bc0036a0 tx=0x7f34bc006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:29.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.225+0000 7f34b97fa700 1 -- 172.21.15.44:0/3203658270 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f34c8045390 con 0x7f34cc0aa020 2020-11-20T12:09:29.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.466+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f34cc0abbb0 con 0x7f34cc0aa020 2020-11-20T12:09:29.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.467+0000 7f34b97fa700 1 -- 172.21.15.44:0/3203658270 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f34c80489b0 con 0x7f34cc0aa020 2020-11-20T12:09:29.471 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:29.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f34a4056cd0 msgr2=0x7f34a4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f34a4056cd0 0x7f34a4059190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f34bc0036a0 tx=0x7f34bc006ae0).stop 2020-11-20T12:09:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0aa020 msgr2=0x7f34cc14a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0aa020 0x7f34cc14a2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f34c800ca90 tx=0x7f34c8001460).stop 2020-11-20T12:09:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 shutdown_connections 2020-11-20T12:09:29.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f34a4056cd0 0x7f34a4059190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:29.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34cc0aa400 0x7f34cc0abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:29.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34cc0a3e10 0x7f34cc067e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:29.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 --2- 172.21.15.44:0/3203658270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34cc0aa020 0x7f34cc14a2d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:29.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 >> 172.21.15.44:0/3203658270 conn(0x7f34cc062690 msgr2=0x7f34cc0a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:29.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 shutdown_connections 2020-11-20T12:09:29.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:29.468+0000 7f34d388a700 1 -- 172.21.15.44:0/3203658270 wait complete. 2020-11-20T12:09:32.491 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:32.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.626+0000 7fadd3cee700 1 Processor -- start 2020-11-20T12:09:32.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.626+0000 7fadd3cee700 1 -- start start 2020-11-20T12:09:32.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd3cee700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:32.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd3cee700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0a6260 con 0x7fadcc13b0a0 2020-11-20T12:09:32.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd3cee700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0a6490 con 0x7fadcc0a4c30 2020-11-20T12:09:32.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd3cee700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0a52e0 con 0x7fadcc060330 2020-11-20T12:09:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd228b700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fadcc13b0a0 legacy=0x7fadcc141810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd1a8a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:32.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd1a8a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59554/0 (socket says 172.21.15.44:59554) 2020-11-20T12:09:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 learned_addr learned my addr 172.21.15.44:0/1763589885 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:32.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:32.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.627+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:32.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.828+0000 7fadd228b700 1 -- 172.21.15.44:0/1763589885 >> v1:172.21.15.44:6789/0 conn(0x7fadcc13b0a0 legacy=0x7fadcc141810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:32.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.828+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.828+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.828+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.828+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:32.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:32.828+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:09:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:33.229+0000 7fadd228b700 1 -- 172.21.15.44:0/1763589885 >> v1:172.21.15.44:6789/0 conn(0x7fadcc13b0a0 legacy=0x7fadcc141810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:33.229+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:33.229+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:33.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:33.229+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:33.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:33.229+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:33.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:33.229+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:09:34.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:34.030+0000 7fadd228b700 1 -- 172.21.15.44:0/1763589885 >> v1:172.21.15.44:6789/0 conn(0x7fadcc13b0a0 legacy=0x7fadcc141810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:34.030+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:34.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:34.030+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:34.030+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:34.030+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:34.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:34.030+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:09:35.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 msgr2=0x7fadcc0a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:35.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.79:3300/0 conn(0x7fadcc060330 0x7fadcc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:35.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 -- 172.21.15.44:0/1763589885 >> v1:172.21.15.79:6789/0 conn(0x7fadcc0a4c30 legacy=0x7fadcc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:35.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 -- 172.21.15.44:0/1763589885 >> v1:172.21.15.44:6789/0 conn(0x7fadcc13b0a0 legacy=0x7fadcc141810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:35.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 0x7fadac005810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:35.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:35.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 -- 172.21.15.44:0/1763589885 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0a52e0 con 0x7fadac0033b0 2020-11-20T12:09:35.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadbb7fe700 1 -- 172.21.15.44:0/1763589885 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac00ab60 con 0x7fadac005db0 2020-11-20T12:09:35.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 msgr2=0x7fadac005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:35.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.627+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 0x7fadac005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:35.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.628+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:35.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.628+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadac008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:09:35.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.628+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadac008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:35.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.628+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:35.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.628+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:35.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.828+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 msgr2=0x7fadac005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:35.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.828+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 0x7fadac005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:09:35.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.829+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:35.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.829+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadac008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:35.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.829+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadac008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:35.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.829+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:35.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:35.829+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:09:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:36.229+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 msgr2=0x7fadac005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:36.229+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 0x7fadac005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:09:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:36.230+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:36.230+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadac008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:09:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:36.230+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadac008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:09:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:36.230+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:36.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:36.230+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:09:37.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.030+0000 7fadd1a8a700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 msgr2=0x7fadac005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:37.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.030+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 0x7fadac005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:09:37.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.031+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.032+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 msgr2=0x7fadac005810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.032+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> v2:172.21.15.44:3300/0 conn(0x7fadac0033b0 0x7fadac005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.032+0000 7fadd1289700 1 -- 172.21.15.44:0/1763589885 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0a6730 con 0x7fadac005db0 2020-11-20T12:09:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.032+0000 7fadd1289700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fadc80079a0 tx=0x7fadc8009040).ready entity=mon.2 client_cookie=2442751cfd2459e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.032+0000 7fadd0a88700 1 -- 172.21.15.44:0/1763589885 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80146e0 con 0x7fadac005db0 2020-11-20T12:09:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.032+0000 7fadd0a88700 1 -- 172.21.15.44:0/1763589885 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc8014840 con 0x7fadac005db0 2020-11-20T12:09:37.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.032+0000 7fadd0a88700 1 -- 172.21.15.44:0/1763589885 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8014bc0 con 0x7fadac005db0 2020-11-20T12:09:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.033+0000 7fadd3cee700 1 -- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadac008270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.033+0000 7fadd3cee700 1 --2- 172.21.15.44:0/1763589885 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadac008270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fadc80079a0 tx=0x7fadc8009040).stop 2020-11-20T12:09:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.033+0000 7fadd228b700 1 -- 172.21.15.44:0/1763589885 reap_dead start 2020-11-20T12:09:37.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.033+0000 7fadd3cee700 1 -- 172.21.15.44:0/1763589885 shutdown_connections 2020-11-20T12:09:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.033+0000 7fadd3cee700 1 -- 172.21.15.44:0/1763589885 >> 172.21.15.44:0/1763589885 conn(0x7fadcc062690 msgr2=0x7fadcc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.033+0000 7fadd3cee700 1 -- 172.21.15.44:0/1763589885 shutdown_connections 2020-11-20T12:09:37.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.033+0000 7fadd3cee700 1 -- 172.21.15.44:0/1763589885 wait complete. 2020-11-20T12:09:37.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.034+0000 7fadd3cee700 1 Processor -- start 2020-11-20T12:09:37.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.034+0000 7fadd3cee700 1 -- start start 2020-11-20T12:09:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd3cee700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadcc0ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd3cee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadcc14a620 0x7fadcc0acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd3cee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc0ad1d0 0x7fadcc137120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd3cee700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0ac890 con 0x7fadcc0ad1d0 2020-11-20T12:09:37.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd3cee700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0ad650 con 0x7fadcc14a620 2020-11-20T12:09:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd3cee700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0adc60 con 0x7fadac005db0 2020-11-20T12:09:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd228b700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc0ad1d0 msgr2=0x7fadcc137120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd228b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc0ad1d0 0x7fadcc137120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:37.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd1a8a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadcc0ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd1289700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadcc14a620 0x7fadcc0acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd1a8a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadcc0ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50746/0 (socket says 172.21.15.44:50746) 2020-11-20T12:09:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.035+0000 7fadd1a8a700 1 -- 172.21.15.44:0/4168795684 learned_addr learned my addr 172.21.15.44:0/4168795684 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd1a8a700 1 -- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadcc14a620 msgr2=0x7fadcc0acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadcc14a620 0x7fadcc0acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd1a8a700 1 -- 172.21.15.44:0/4168795684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc0ad1d0 msgr2=0x7fadcc137120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:37.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc0ad1d0 0x7fadcc137120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:37.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd1a8a700 1 -- 172.21.15.44:0/4168795684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc14aba0 con 0x7fadac005db0 2020-11-20T12:09:37.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd1289700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadcc14a620 0x7fadcc0acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:37.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd1a8a700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadcc0ac080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fadbc012c50 tx=0x7fadbc008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadb9ffb700 1 -- 172.21.15.44:0/4168795684 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00fb10 con 0x7fadac005db0 2020-11-20T12:09:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadb9ffb700 1 -- 172.21.15.44:0/4168795684 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00fc70 con 0x7fadac005db0 2020-11-20T12:09:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc14ae40 con 0x7fadac005db0 2020-11-20T12:09:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.036+0000 7fadb9ffb700 1 -- 172.21.15.44:0/4168795684 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc015450 con 0x7fadac005db0 2020-11-20T12:09:37.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.037+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc137610 con 0x7fadac005db0 2020-11-20T12:09:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.038+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc134260 con 0x7fadac005db0 2020-11-20T12:09:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.038+0000 7fadb9ffb700 1 -- 172.21.15.44:0/4168795684 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fadbc025070 con 0x7fadac005db0 2020-11-20T12:09:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.039+0000 7fadb9ffb700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fada4056cd0 0x7fada4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.039+0000 7fadb9ffb700 1 -- 172.21.15.44:0/4168795684 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fadbc0708d0 con 0x7fadac005db0 2020-11-20T12:09:37.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.039+0000 7fadd1289700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fada4056cd0 0x7fada4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.039+0000 7fadd1289700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fada4056cd0 0x7fada4059190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fadcc0ab950 tx=0x7fadc8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.043+0000 7fadb9ffb700 1 -- 172.21.15.44:0/4168795684 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fadbc045b40 con 0x7fadac005db0 2020-11-20T12:09:37.060 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:09:37.066 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:09:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.295+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fadcc0aca00 con 0x7fadac005db0 2020-11-20T12:09:37.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.296+0000 7fadb9ffb700 1 -- 172.21.15.44:0/4168795684 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fadbc023070 con 0x7fadac005db0 2020-11-20T12:09:37.300 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":64,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fada4056cd0 msgr2=0x7fada4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fada4056cd0 0x7fada4059190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fadcc0ab950 tx=0x7fadc8009040).stop 2020-11-20T12:09:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 msgr2=0x7fadcc0ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadcc0ac080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fadbc012c50 tx=0x7fadbc008040).stop 2020-11-20T12:09:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 shutdown_connections 2020-11-20T12:09:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fada4056cd0 0x7fada4059190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadcc0ad1d0 0x7fadcc137120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fadac005db0 0x7fadcc0ac080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 --2- 172.21.15.44:0/4168795684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadcc14a620 0x7fadcc0acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 >> 172.21.15.44:0/4168795684 conn(0x7fadcc062690 msgr2=0x7fadcc133360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 shutdown_connections 2020-11-20T12:09:37.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.297+0000 7fadd3cee700 1 -- 172.21.15.44:0/4168795684 wait complete. 2020-11-20T12:09:37.315 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:09:37.315 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:09:37.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.449+0000 7fef6eba4700 1 Processor -- start 2020-11-20T12:09:37.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.449+0000 7fef6eba4700 1 -- start start 2020-11-20T12:09:37.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.449+0000 7fef6eba4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef680a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.449+0000 7fef6eba4700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680a9770 con 0x7fef6813ac60 2020-11-20T12:09:37.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.449+0000 7fef6eba4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680a6da0 con 0x7fef680aa020 2020-11-20T12:09:37.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.449+0000 7fef6eba4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef680a56c0 con 0x7fef680aa400 2020-11-20T12:09:37.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.449+0000 7fef6d141700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fef6813ac60 legacy=0x7fef680a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:37.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef6c940700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fef680aa020 0x7fef680a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:45938/0 (socket says 172.21.15.44:45938) 2020-11-20T12:09:37.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef6c940700 1 -- 172.21.15.44:0/1289959019 learned_addr learned my addr 172.21.15.44:0/1289959019 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:37.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef67fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef677fe700 1 -- 172.21.15.44:0/1289959019 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464541373 0 0) 0x7fef680a6da0 con 0x7fef680aa020 2020-11-20T12:09:37.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef677fe700 1 -- 172.21.15.44:0/1289959019 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef54003240 con 0x7fef680aa020 2020-11-20T12:09:37.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef67fff700 1 -- 172.21.15.44:0/1289959019 >> v1:172.21.15.79:6789/0 conn(0x7fef680aa020 legacy=0x7fef680a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef67fff700 1 -- 172.21.15.44:0/1289959019 >> v1:172.21.15.44:6789/0 conn(0x7fef6813ac60 legacy=0x7fef680a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:37.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef67fff700 1 -- 172.21.15.44:0/1289959019 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680a7040 con 0x7fef680aa400 2020-11-20T12:09:37.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef67fff700 1 --2- 172.21.15.44:0/1289959019 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef680a9280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fef5c00b9d0 tx=0x7fef5c006df0).ready entity=mon.2 client_cookie=baf3db4253b335ae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef677fe700 1 -- 172.21.15.44:0/1289959019 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c009770 con 0x7fef680aa400 2020-11-20T12:09:37.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.450+0000 7fef677fe700 1 -- 172.21.15.44:0/1289959019 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c0098d0 con 0x7fef680aa400 2020-11-20T12:09:37.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef677fe700 1 -- 172.21.15.44:0/1289959019 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c009c50 con 0x7fef680aa400 2020-11-20T12:09:37.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef6eba4700 1 -- 172.21.15.44:0/1289959019 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 msgr2=0x7fef680a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef6eba4700 1 --2- 172.21.15.44:0/1289959019 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef680a9280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fef5c00b9d0 tx=0x7fef5c006df0).stop 2020-11-20T12:09:37.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef6eba4700 1 -- 172.21.15.44:0/1289959019 shutdown_connections 2020-11-20T12:09:37.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef6eba4700 1 --2- 172.21.15.44:0/1289959019 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef680a9280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef6eba4700 1 -- 172.21.15.44:0/1289959019 >> 172.21.15.44:0/1289959019 conn(0x7fef68062690 msgr2=0x7fef680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:37.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef6eba4700 1 -- 172.21.15.44:0/1289959019 shutdown_connections 2020-11-20T12:09:37.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.451+0000 7fef6eba4700 1 -- 172.21.15.44:0/1289959019 wait complete. 2020-11-20T12:09:37.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.452+0000 7fef6eba4700 1 Processor -- start 2020-11-20T12:09:37.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.452+0000 7fef6eba4700 1 -- start start 2020-11-20T12:09:37.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6eba4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef680aa020 0x7fef680acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6eba4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef6809de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6eba4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6813ac60 0x7fef6809e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6eba4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6809d9c0 con 0x7fef680aa020 2020-11-20T12:09:37.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6eba4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68060da0 con 0x7fef6813ac60 2020-11-20T12:09:37.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6eba4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef68061040 con 0x7fef680aa400 2020-11-20T12:09:37.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6c940700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef680aa020 msgr2=0x7fef680acf70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:37.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6c940700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef680aa020 0x7fef680acf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:37.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef6809de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef6809de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50760/0 (socket says 172.21.15.44:50760) 2020-11-20T12:09:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 -- 172.21.15.44:0/3998630526 learned_addr learned my addr 172.21.15.44:0/3998630526 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef6d141700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6813ac60 0x7fef6809e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6813ac60 msgr2=0x7fef6809e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6813ac60 0x7fef6809e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef680aa020 msgr2=0x7fef680acf70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef680aa020 0x7fef680acf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 -- 172.21.15.44:0/3998630526 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680ad980 con 0x7fef680aa400 2020-11-20T12:09:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.453+0000 7fef67fff700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef6809de20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fef5c002fc0 tx=0x7fef5c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.454+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c011dd0 con 0x7fef680aa400 2020-11-20T12:09:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.454+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c01e460 con 0x7fef680aa400 2020-11-20T12:09:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.454+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef68060890 con 0x7fef680aa400 2020-11-20T12:09:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.454+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c01e7e0 con 0x7fef680aa400 2020-11-20T12:09:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.454+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6809ffe0 con 0x7fef680aa400 2020-11-20T12:09:37.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.454+0000 7fef6eba4700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6814baf0 0x7fef6814bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.454+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6809db80 con 0x7fef6814baf0 2020-11-20T12:09:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef6c940700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6814baf0 0x7fef6814bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fef5c025030 con 0x7fef680aa400 2020-11-20T12:09:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef64ff9700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef40056cd0 0x7fef40059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef6c940700 1 -- 172.21.15.44:0/3998630526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef68061650 con 0x7fef6814baf0 2020-11-20T12:09:37.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fef5c0716b0 con 0x7fef680aa400 2020-11-20T12:09:37.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef6c940700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6814baf0 0x7fef6814bed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fef58004590 tx=0x7fef58007b10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef6d141700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef40056cd0 0x7fef40059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.455+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef580072e0 con 0x7fef6814baf0 2020-11-20T12:09:37.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.456+0000 7fef6d141700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef40056cd0 0x7fef40059190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fef60003a20 tx=0x7fef600090f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.456+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fef58007440 con 0x7fef6814baf0 2020-11-20T12:09:37.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.456+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6814baf0 msgr2=0x7fef6814bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.456+0000 7fef64ff9700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6814baf0 0x7fef6814bed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fef58004590 tx=0x7fef58007b10).stop 2020-11-20T12:09:37.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.456+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6814baf0 msgr2=0x7fef6814bed0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:37.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.456+0000 7fef64ff9700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef6814baf0 0x7fef6814bed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.464+0000 7fef6eba4700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef680672b0 0x7fef680676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.464+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680a9770 con 0x7fef680672b0 2020-11-20T12:09:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.464+0000 7fef6c940700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef680672b0 0x7fef680676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.464+0000 7fef6c940700 1 -- 172.21.15.44:0/3998630526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef680a6a90 con 0x7fef680672b0 2020-11-20T12:09:37.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.464+0000 7fef6c940700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef680672b0 0x7fef680676c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fef58012000 tx=0x7fef58015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58014b80 con 0x7fef680672b0 2020-11-20T12:09:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4426 (secure 0 0 0) 0x7fef580070d0 con 0x7fef680672b0 2020-11-20T12:09:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef680672b0 msgr2=0x7fef680676c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef64ff9700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef680672b0 0x7fef680676c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fef58012000 tx=0x7fef58015000).stop 2020-11-20T12:09:37.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef64ff9700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef680672b0 msgr2=0x7fef680676c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef64ff9700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef680672b0 0x7fef680676c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef40056cd0 msgr2=0x7fef40059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef40056cd0 0x7fef40059190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fef60003a20 tx=0x7fef600090f0).stop 2020-11-20T12:09:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 msgr2=0x7fef6809de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef6809de20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fef5c002fc0 tx=0x7fef5c015000).stop 2020-11-20T12:09:37.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6d141700 1 -- 172.21.15.44:0/3998630526 reap_dead start 2020-11-20T12:09:37.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 shutdown_connections 2020-11-20T12:09:37.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 --2- 172.21.15.44:0/3998630526 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef680aa400 0x7fef6809de20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 >> 172.21.15.44:0/3998630526 conn(0x7fef68062690 msgr2=0x7fef680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:37.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 shutdown_connections 2020-11-20T12:09:37.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.465+0000 7fef6eba4700 1 -- 172.21.15.44:0/3998630526 wait complete. 2020-11-20T12:09:37.478 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:09:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 64, 2020-11-20T12:09:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:09:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:09:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:37.515 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:37.516 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:09:37.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.643+0000 7fbd15dd0700 1 Processor -- start 2020-11-20T12:09:37.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.643+0000 7fbd15dd0700 1 -- start start 2020-11-20T12:09:37.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd15dd0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd100aa400 0x7fbd100a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd15dd0700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd100a9770 con 0x7fbd100aa020 2020-11-20T12:09:37.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd15dd0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd100a6da0 con 0x7fbd1013ac60 2020-11-20T12:09:37.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd15dd0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd100a56c0 con 0x7fbd100aa400 2020-11-20T12:09:37.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0f7fe700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fbd100aa020 legacy=0x7fbd100a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:09:37.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd100aa400 0x7fbd100a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0ffff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbd1013ac60 0x7fbd100a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:45960/0 (socket says 172.21.15.44:45960) 2020-11-20T12:09:37.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0ffff700 1 -- 172.21.15.44:0/4243332821 learned_addr learned my addr 172.21.15.44:0/4243332821 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:37.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0effd700 1 -- 172.21.15.44:0/4243332821 >> v1:172.21.15.79:6789/0 conn(0x7fbd1013ac60 legacy=0x7fbd100a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0effd700 1 -- 172.21.15.44:0/4243332821 >> v1:172.21.15.44:6789/0 conn(0x7fbd100aa020 legacy=0x7fbd100a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:09:37.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0e7fc700 1 -- 172.21.15.44:0/4243332821 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559647244 0 0) 0x7fbd100a6da0 con 0x7fbd1013ac60 2020-11-20T12:09:37.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.644+0000 7fbd0effd700 1 -- 172.21.15.44:0/4243332821 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100a7040 con 0x7fbd100aa400 2020-11-20T12:09:37.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd0effd700 1 --2- 172.21.15.44:0/4243332821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 0x7fbd100a9280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fbd0400b9a0 tx=0x7fbd04006df0).ready entity=mon.1 client_cookie=f4082a493803a233 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd0e7fc700 1 -- 172.21.15.44:0/4243332821 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04009770 con 0x7fbd100aa400 2020-11-20T12:09:37.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd0e7fc700 1 -- 172.21.15.44:0/4243332821 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd040098d0 con 0x7fbd100aa400 2020-11-20T12:09:37.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd0e7fc700 1 -- 172.21.15.44:0/4243332821 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04009c50 con 0x7fbd100aa400 2020-11-20T12:09:37.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd15dd0700 1 -- 172.21.15.44:0/4243332821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 msgr2=0x7fbd100a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd15dd0700 1 --2- 172.21.15.44:0/4243332821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 0x7fbd100a9280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fbd0400b9a0 tx=0x7fbd04006df0).stop 2020-11-20T12:09:37.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd15dd0700 1 -- 172.21.15.44:0/4243332821 shutdown_connections 2020-11-20T12:09:37.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd15dd0700 1 --2- 172.21.15.44:0/4243332821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 0x7fbd100a9280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd15dd0700 1 -- 172.21.15.44:0/4243332821 >> 172.21.15.44:0/4243332821 conn(0x7fbd10062690 msgr2=0x7fbd100a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:37.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd15dd0700 1 -- 172.21.15.44:0/4243332821 shutdown_connections 2020-11-20T12:09:37.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.645+0000 7fbd15dd0700 1 -- 172.21.15.44:0/4243332821 wait complete. 2020-11-20T12:09:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.646+0000 7fbd15dd0700 1 Processor -- start 2020-11-20T12:09:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.646+0000 7fbd15dd0700 1 -- start start 2020-11-20T12:09:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd15dd0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd100aa020 0x7fbd10065e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd15dd0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 0x7fbd100ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd15dd0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd1013ac60 0x7fbd10146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd15dd0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1014a5d0 con 0x7fbd100aa020 2020-11-20T12:09:37.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd15dd0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1014a870 con 0x7fbd100aa400 2020-11-20T12:09:37.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd15dd0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1014ab10 con 0x7fbd1013ac60 2020-11-20T12:09:37.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd0f7fe700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd100aa020 msgr2=0x7fbd10065e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:09:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd0f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd100aa020 0x7fbd10065e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd0ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd1013ac60 0x7fbd10146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd0effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 0x7fbd100ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd0ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd1013ac60 0x7fbd10146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:50780/0 (socket says 172.21.15.44:50780) 2020-11-20T12:09:37.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.647+0000 7fbd0ffff700 1 -- 172.21.15.44:0/2272863247 learned_addr learned my addr 172.21.15.44:0/2272863247 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd0ffff700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 msgr2=0x7fbd100ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd0ffff700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd100aa400 0x7fbd100ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd0ffff700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd100aa020 msgr2=0x7fbd10065e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:37.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd0ffff700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd100aa020 0x7fbd10065e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:09:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd0ffff700 1 -- 172.21.15.44:0/2272863247 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100ad640 con 0x7fbd1013ac60 2020-11-20T12:09:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd0ffff700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd1013ac60 0x7fbd10146a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fbd0800cae0 tx=0x7fbd08001320).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08005b80 con 0x7fbd1013ac60 2020-11-20T12:09:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd08005ce0 con 0x7fbd1013ac60 2020-11-20T12:09:37.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1014a050 con 0x7fbd1013ac60 2020-11-20T12:09:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08007360 con 0x7fbd1013ac60 2020-11-20T12:09:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.648+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1014a3a0 con 0x7fbd1013ac60 2020-11-20T12:09:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.649+0000 7fbd15dd0700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd100676b0 0x7fbd10067a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.649+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1014acd0 con 0x7fbd100676b0 2020-11-20T12:09:37.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.649+0000 7fbd0f7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd100676b0 0x7fbd10067a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.649+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbd08010030 con 0x7fbd1013ac60 2020-11-20T12:09:37.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.649+0000 7fbd0f7fe700 1 -- 172.21.15.44:0/2272863247 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd10061650 con 0x7fbd100676b0 2020-11-20T12:09:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.649+0000 7fbd0f7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd100676b0 0x7fbd10067a90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fbd00009640 tx=0x7fbd00006e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.649+0000 7fbcef7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbce8056cd0 0x7fbce8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbd0806ff70 con 0x7fbd1013ac60 2020-11-20T12:09:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000050d0 con 0x7fbd100676b0 2020-11-20T12:09:37.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbd0effd700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbce8056cd0 0x7fbce8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fbd00005230 con 0x7fbd100676b0 2020-11-20T12:09:37.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd100676b0 msgr2=0x7fbd10067a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbcef7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd100676b0 0x7fbd10067a90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fbd00009640 tx=0x7fbd00006e60).stop 2020-11-20T12:09:37.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd100676b0 msgr2=0x7fbd10067a90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:37.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbcef7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd100676b0 0x7fbd10067a90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.650+0000 7fbd0effd700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbce8056cd0 0x7fbce8059190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fbd0400b150 tx=0x7fbd04015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.658+0000 7fbd15dd0700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd10067f80 0x7fbd100ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:37.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.658+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100446d0 con 0x7fbd10067f80 2020-11-20T12:09:37.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.658+0000 7fbd0f7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd10067f80 0x7fbd100ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:37.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.658+0000 7fbd0f7fe700 1 -- 172.21.15.44:0/2272863247 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbd100613b0 con 0x7fbd10067f80 2020-11-20T12:09:37.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbd0f7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd10067f80 0x7fbd100ab990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fbd00009640 tx=0x7fbd00013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:37.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000b4b0 con 0x7fbd10067f80 2020-11-20T12:09:37.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4764 (secure 0 0 0) 0x7fbd000036d0 con 0x7fbd10067f80 2020-11-20T12:09:37.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd10067f80 msgr2=0x7fbd100ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbcef7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd10067f80 0x7fbd100ab990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fbd00009640 tx=0x7fbd00013000).stop 2020-11-20T12:09:37.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbcef7fe700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd10067f80 msgr2=0x7fbd100ab990 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:09:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbcef7fe700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd10067f80 0x7fbd100ab990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbce8056cd0 msgr2=0x7fbce8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbd15dd0700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbce8056cd0 0x7fbce8059190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fbd0400b150 tx=0x7fbd04015000).stop 2020-11-20T12:09:37.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd1013ac60 msgr2=0x7fbd10146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:37.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbd15dd0700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd1013ac60 0x7fbd10146a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fbd0800cae0 tx=0x7fbd08001320).stop 2020-11-20T12:09:37.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.659+0000 7fbd0ffff700 1 -- 172.21.15.44:0/2272863247 reap_dead start 2020-11-20T12:09:37.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.660+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 shutdown_connections 2020-11-20T12:09:37.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.660+0000 7fbd15dd0700 1 --2- 172.21.15.44:0/2272863247 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd1013ac60 0x7fbd10146a80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:37.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.660+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 >> 172.21.15.44:0/2272863247 conn(0x7fbd10062690 msgr2=0x7fbd10139ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.660+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 shutdown_connections 2020-11-20T12:09:37.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:37.660+0000 7fbd15dd0700 1 -- 172.21.15.44:0/2272863247 wait complete. 2020-11-20T12:09:37.675 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:09:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:09:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 64, 2020-11-20T12:09:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:09:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:09:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:09:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:09:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:09:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:09:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:37.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:37.679 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:37.679 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:37.679 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:37.679 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:37.680 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:37.680 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.680 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:09:37.680 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:09:37.680 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:37.681 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:37.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:37.681 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:37.682 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:37.682 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:37.682 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:37.682 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:37.683 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.683 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:09:37.683 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:09:37.683 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:09:37.684 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:09:37.684 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:09:37.684 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:09:37.684 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:37.685 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:09:37.685 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:09:37.685 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:09:37.685 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:09:37.685 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:09:37.686 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:09:37.686 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:09:37.686 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:09:37.686 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:09:37.686 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:09:37.687 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:09:37.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:09:37.687 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:09:37.687 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:09:37.687 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:09:37.688 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:09:37.688 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.688 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:09:37.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.688 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:09:37.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.689 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:09:37.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:09:37.689 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:37.689 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:37.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:37.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:09:37.690 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:37.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:09:37.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.692 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:09:37.692 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:09:37.692 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:37.693 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:37.693 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:37.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.693 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:09:37.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:09:37.694 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:37.694 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:37.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:37.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:09:37.695 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:37.696 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:09:37.696 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.696 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.697 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:09:37.697 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:09:37.697 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:37.697 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:37.697 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:37.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.698 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:09:37.698 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:09:37.698 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:09:37.699 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:09:37.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:09:37.699 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:09:37.699 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:09:37.700 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:09:37.700 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:09:37.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.701 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.701 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:09:37.701 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:09:37.702 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:09:37.702 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:09:37.702 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:09:37.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.702 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.703 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:09:37.703 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:09:37.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.703 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.704 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.704 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:09:37.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.704 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.704 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:09:37.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.705 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.705 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.705 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:09:37.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.706 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.706 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:09:37.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.706 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.706 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:09:37.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.707 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.707 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:09:37.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.708 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.708 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.708 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:09:37.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.708 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:09:37.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:09:37.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:09:37.709 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:09:37.709 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:09:37.710 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:09:37.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:09:37.710 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:09:37.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:09:37.710 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:09:37.711 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:09:37.712 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:09:57.713 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:09:57.713 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:09:57.713 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:09:57.714 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:09:57.719 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:09:57.721 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:09:57.721 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:09:57.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.859+0000 7fc11f667700 1 Processor -- start 2020-11-20T12:09:57.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.859+0000 7fc11f667700 1 -- start start 2020-11-20T12:09:57.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.859+0000 7fc11f667700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc1180aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:57.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.859+0000 7fc11f667700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc118137bb0 0x7fc118138060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.859+0000 7fc11f667700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc118067470 con 0x7fc1180a1a40 2020-11-20T12:09:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.859+0000 7fc11f667700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1180a9b30 con 0x7fc118137bb0 2020-11-20T12:09:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.859+0000 7fc11f667700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180ab240 con 0x7fc118066de0 2020-11-20T12:09:57.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.860+0000 7fc11d403700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc1180a1a40 0x7fc1180666d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:45980/0 (socket says 172.21.15.44:45980) 2020-11-20T12:09:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.860+0000 7fc11d403700 1 -- 172.21.15.44:0/2007668782 learned_addr learned my addr 172.21.15.44:0/2007668782 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:09:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.860+0000 7fc11cc02700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc1180aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.860+0000 7fc11dc04700 1 --2- 172.21.15.44:0/2007668782 >> v2:172.21.15.79:3300/0 conn(0x7fc118137bb0 0x7fc118138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.860+0000 7fc107fff700 1 -- 172.21.15.44:0/2007668782 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708192129 0 0) 0x7fc118067470 con 0x7fc1180a1a40 2020-11-20T12:09:57.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.860+0000 7fc107fff700 1 -- 172.21.15.44:0/2007668782 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc100003240 con 0x7fc1180a1a40 2020-11-20T12:09:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.860+0000 7fc11dc04700 1 -- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 msgr2=0x7fc1180aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:09:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11dc04700 1 --2- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc1180aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11dc04700 1 -- 172.21.15.44:0/2007668782 >> v1:172.21.15.79:6789/0 conn(0x7fc1180a1a40 legacy=0x7fc1180666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11dc04700 1 -- 172.21.15.44:0/2007668782 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc118067710 con 0x7fc118137bb0 2020-11-20T12:09:57.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11cc02700 1 --2- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc1180aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:09:57.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11dc04700 1 --2- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc118137bb0 0x7fc118138060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc110009c90 tx=0x7fc1100091d0).ready entity=mon.1 client_cookie=d9fd22f80c1473c7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:57.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc107fff700 1 -- 172.21.15.44:0/2007668782 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000b9c0 con 0x7fc118137bb0 2020-11-20T12:09:57.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc107fff700 1 -- 172.21.15.44:0/2007668782 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000bb20 con 0x7fc118137bb0 2020-11-20T12:09:57.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc107fff700 1 -- 172.21.15.44:0/2007668782 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000bea0 con 0x7fc118137bb0 2020-11-20T12:09:57.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11f667700 1 -- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc118137bb0 msgr2=0x7fc118138060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:57.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11f667700 1 --2- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc118137bb0 0x7fc118138060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc110009c90 tx=0x7fc1100091d0).stop 2020-11-20T12:09:57.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11f667700 1 -- 172.21.15.44:0/2007668782 shutdown_connections 2020-11-20T12:09:57.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11f667700 1 --2- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc118137bb0 0x7fc118138060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:57.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11f667700 1 --2- 172.21.15.44:0/2007668782 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc1180aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:57.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.861+0000 7fc11f667700 1 -- 172.21.15.44:0/2007668782 >> 172.21.15.44:0/2007668782 conn(0x7fc1180af030 msgr2=0x7fc1180af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:57.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.862+0000 7fc11f667700 1 -- 172.21.15.44:0/2007668782 shutdown_connections 2020-11-20T12:09:57.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.862+0000 7fc11f667700 1 -- 172.21.15.44:0/2007668782 wait complete. 2020-11-20T12:09:57.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 Processor -- start 2020-11-20T12:09:57.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 -- start start 2020-11-20T12:09:57.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc11813ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:57.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1180a1a40 0x7fc11814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 0x7fc11814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118063a70 con 0x7fc118137bb0 2020-11-20T12:09:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11814e240 con 0x7fc1180a1a40 2020-11-20T12:09:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11f667700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc118063530 con 0x7fc118066de0 2020-11-20T12:09:57.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.863+0000 7fc11dc04700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 0x7fc11814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11dc04700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 0x7fc11814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39364/0 (socket says 172.21.15.44:39364) 2020-11-20T12:09:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11dc04700 1 -- 172.21.15.44:0/1359971437 learned_addr learned my addr 172.21.15.44:0/1359971437 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:09:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11cc02700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1180a1a40 0x7fc11814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11dc04700 1 -- 172.21.15.44:0/1359971437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 msgr2=0x7fc11814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:09:57.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11dc04700 1 -- 172.21.15.44:0/1359971437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 msgr2=0x7fc11814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:09:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11d403700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc11813ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11dc04700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 0x7fc11814dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:09:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11dc04700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 0x7fc11814dd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:09:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11cc02700 1 -- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 msgr2=0x7fc11813ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:57.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11cc02700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc11813ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11cc02700 1 -- 172.21.15.44:0/1359971437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 msgr2=0x7fc11814dd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:09:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11cc02700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 0x7fc11814dd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11cc02700 1 -- 172.21.15.44:0/1359971437 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180637d0 con 0x7fc1180a1a40 2020-11-20T12:09:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11cc02700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1180a1a40 0x7fc11814d860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc114006fd0 tx=0x7fc114006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:57.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc1057fa700 1 -- 172.21.15.44:0/1359971437 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400caa0 con 0x7fc1180a1a40 2020-11-20T12:09:57.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc1057fa700 1 -- 172.21.15.44:0/1359971437 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400cc00 con 0x7fc1180a1a40 2020-11-20T12:09:57.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1180a9380 con 0x7fc1180a1a40 2020-11-20T12:09:57.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.864+0000 7fc1057fa700 1 -- 172.21.15.44:0/1359971437 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140123f0 con 0x7fc1180a1a40 2020-11-20T12:09:57.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.865+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1180632c0 con 0x7fc1180a1a40 2020-11-20T12:09:57.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.865+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11805c2a0 con 0x7fc1180a1a40 2020-11-20T12:09:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.866+0000 7fc1057fa700 1 -- 172.21.15.44:0/1359971437 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc1140127a0 con 0x7fc1180a1a40 2020-11-20T12:09:57.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.866+0000 7fc1057fa700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc0f0056d20 0x7fc0f00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:09:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.866+0000 7fc1057fa700 1 -- 172.21.15.44:0/1359971437 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc1140706d0 con 0x7fc1180a1a40 2020-11-20T12:09:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.866+0000 7fc11d403700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc0f0056d20 0x7fc0f00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:09:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.867+0000 7fc11d403700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc0f0056d20 0x7fc0f00591e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc108007fd0 tx=0x7fc108009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:09:57.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:57.870+0000 7fc1057fa700 1 -- 172.21.15.44:0/1359971437 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc114045940 con 0x7fc1180a1a40 2020-11-20T12:09:58.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.108+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc1180446d0 con 0x7fc1180a1a40 2020-11-20T12:09:58.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.109+0000 7fc1057fa700 1 -- 172.21.15.44:0/1359971437 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fc114048f60 con 0x7fc1180a1a40 2020-11-20T12:09:58.114 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":64,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:09:58.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc0f0056d20 msgr2=0x7fc0f00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:58.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc0f0056d20 0x7fc0f00591e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fc108007fd0 tx=0x7fc108009040).stop 2020-11-20T12:09:58.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1180a1a40 msgr2=0x7fc11814d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:09:58.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1180a1a40 0x7fc11814d860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc114006fd0 tx=0x7fc114006df0).stop 2020-11-20T12:09:58.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 shutdown_connections 2020-11-20T12:09:58.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc0f0056d20 0x7fc0f00591e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc118137bb0 0x7fc11814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc118066de0 0x7fc11813ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:58.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 --2- 172.21.15.44:0/1359971437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1180a1a40 0x7fc11814d860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:09:58.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 >> 172.21.15.44:0/1359971437 conn(0x7fc1180af030 msgr2=0x7fc1180ac280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:09:58.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 shutdown_connections 2020-11-20T12:09:58.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:09:58.110+0000 7fc11f667700 1 -- 172.21.15.44:0/1359971437 wait complete. 2020-11-20T12:10:01.127 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:10:01.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.266+0000 7f0f02694700 1 Processor -- start 2020-11-20T12:10:01.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.266+0000 7f0f02694700 1 -- start start 2020-11-20T12:10:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0f02694700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0f02694700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0f02694700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0efc0a6260 con 0x7f0efc13ae80 2020-11-20T12:10:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0f02694700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc0a52e0 con 0x7f0efc0a4c30 2020-11-20T12:10:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0f02694700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc0aa730 con 0x7f0efc060330 2020-11-20T12:10:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0ef3fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0ef3fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39386/0 (socket says 172.21.15.44:39386) 2020-11-20T12:10:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 learned_addr learned my addr 172.21.15.44:0/1086776052 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:01.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:01.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:01.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.267+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:01.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:01.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:01.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:01.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:01.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:01.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:01.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:01.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.468+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:01.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:01.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:01.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:01.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:01.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:01.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:01.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:01.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:01.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:01.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:01.869+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:02.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:02.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:02.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:02.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:02.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:02.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:02.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:02.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0efbfff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:02.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:02.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:02.670+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 msgr2=0x7f0efc0a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:04.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.79:3300/0 conn(0x7f0efc060330 0x7f0efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:04.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0efc13a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:04.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0efc13a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:04.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 >> v1:172.21.15.79:6789/0 conn(0x7f0efc13ae80 legacy=0x7f0efc0aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:04.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:04.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0efc0a6490 con 0x7f0ee00033d0 2020-11-20T12:10:04.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0a52e0 con 0x7f0ee0006fe0 2020-11-20T12:10:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.267+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.268+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.268+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.268+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.268+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:04.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:04.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.469+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:04.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.469+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:04.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:04.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:04.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.870+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.870+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:04.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.870+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.870+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:04.870+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:05.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:05.671+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:05.671+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:05.671+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:05.671+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:05.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:05.671+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:07.151 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:10:07.163 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:10:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 msgr2=0x7f0ee00094a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0efa7fc700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ee0006fe0 0x7f0ee00094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 >> v1:172.21.15.44:6789/0 conn(0x7f0ee00033d0 legacy=0x7f0ee0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:07.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0f00c31700 1 -- 172.21.15.44:0/1086776052 reap_dead start 2020-11-20T12:10:07.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0efa7fc700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee000aca0 con 0x7f0ee0009cc0 2020-11-20T12:10:07.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee0003820 con 0x7f0efc13ae80 2020-11-20T12:10:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0efa7fc700 1 -- 172.21.15.44:0/1086776052 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc0a52e0 con 0x7f0efc0a4c30 2020-11-20T12:10:07.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:07.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:07.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.268+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:07.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:07.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.469+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:07.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.469+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:07.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:07.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:07.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.870+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:07.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.870+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:07.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.870+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:07.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.870+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:07.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:07.870+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:08.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:08.671+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:08.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:08.671+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:08.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:08.671+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:08.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:08.671+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:08.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:08.671+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:09.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.467+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574022053 0 0) 0x7f0ee0003820 con 0x7f0efc13ae80 2020-11-20T12:10:09.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.467+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0edc003240 con 0x7f0efc13ae80 2020-11-20T12:10:09.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1824671446 0 0) 0x7f0edc003240 con 0x7f0efc13ae80 2020-11-20T12:10:09.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 msgr2=0x7f0ee000ffa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:09.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 --2- 172.21.15.44:0/1086776052 >> v2:172.21.15.44:3300/0 conn(0x7f0efc0a4c30 0x7f0ee000ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 >> v1:172.21.15.44:6789/0 conn(0x7f0ee0009cc0 legacy=0x7f0ee000a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee00049c0 con 0x7f0efc13ae80 2020-11-20T12:10:09.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f0ef4001810 con 0x7f0efc13ae80 2020-11-20T12:10:09.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 >> v1:172.21.15.79:6789/0 conn(0x7f0efc13ae80 legacy=0x7f0ee00124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:09.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0edc003730 0x7f0edc003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0edc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0edc004e50 0x7f0edc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0a52e0 con 0x7f0edc004e50 2020-11-20T12:10:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc005330 con 0x7f0edc0044e0 2020-11-20T12:10:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc00bc20 con 0x7f0edc003730 2020-11-20T12:10:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0f00c31700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0edc004e50 0x7f0edc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0edc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0efbfff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0edc003730 0x7f0edc003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0edc003730 msgr2=0x7f0edc003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0edc003730 0x7f0edc003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0edc004e50 msgr2=0x7f0edc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0edc004e50 0x7f0edc00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0ef3fff700 1 -- 172.21.15.44:0/1086776052 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc003240 con 0x7f0edc0044e0 2020-11-20T12:10:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.469+0000 7f0f00c31700 1 -- 172.21.15.44:0/1086776052 reap_dead start 2020-11-20T12:10:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.470+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0edc004960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ee4006cc0 tx=0x7f0ee4008040).ready entity=mon.1 client_cookie=581a730b00854326 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.470+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4017df0 con 0x7f0edc0044e0 2020-11-20T12:10:09.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.471+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee401f030 con 0x7f0edc0044e0 2020-11-20T12:10:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.471+0000 7f0efb7fe700 1 -- 172.21.15.44:0/1086776052 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4025630 con 0x7f0edc0044e0 2020-11-20T12:10:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.472+0000 7f0f02694700 1 -- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0edc004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.472+0000 7f0f02694700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0edc004960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0ee4006cc0 tx=0x7f0ee4008040).stop 2020-11-20T12:10:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.472+0000 7f0f02694700 1 -- 172.21.15.44:0/1086776052 shutdown_connections 2020-11-20T12:10:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.472+0000 7f0f02694700 1 --2- 172.21.15.44:0/1086776052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0edc004960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.472+0000 7f0f02694700 1 -- 172.21.15.44:0/1086776052 >> 172.21.15.44:0/1086776052 conn(0x7f0efc062690 msgr2=0x7f0efc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.472+0000 7f0f02694700 1 -- 172.21.15.44:0/1086776052 shutdown_connections 2020-11-20T12:10:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.472+0000 7f0f02694700 1 -- 172.21.15.44:0/1086776052 wait complete. 2020-11-20T12:10:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.473+0000 7f0f02694700 1 Processor -- start 2020-11-20T12:10:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0f02694700 1 -- start start 2020-11-20T12:10:09.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0f02694700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:09.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0f02694700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:09.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0f02694700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:09.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0f02694700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0ab290 con 0x7f0efc12fe60 2020-11-20T12:10:09.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0f02694700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc14ade0 con 0x7f0edc0044e0 2020-11-20T12:10:09.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0f02694700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc130690 con 0x7f0efc14a470 2020-11-20T12:10:09.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.474+0000 7f0efbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39430/0 (socket says 172.21.15.44:39430) 2020-11-20T12:10:09.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 learned_addr learned my addr 172.21.15.44:0/316280478 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.475+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:09.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:09.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:09.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:09.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:09.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:09.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:09.688 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:09.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:09.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:09.676+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:10.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:10.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:10.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:10.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:10.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:10.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:10.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.077+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:10.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:10.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:10.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:10.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:10.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:10.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:10.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:10.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:10.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:10.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:10.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:10.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:10.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:10.878+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0efc14a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0efc14a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 msgr2=0x7f0efc134690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0edc0044e0 0x7f0efc134690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0efc1312d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0efc1312d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc14ade0 con 0x7f0efc12fe60 2020-11-20T12:10:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc130690 con 0x7f0ed8003400 2020-11-20T12:10:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800a830 con 0x7f0efc14a470 2020-11-20T12:10:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:12.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:12.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:12.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:12.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:12.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:12.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:12.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:12.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.475+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:12.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:12.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:12.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:12.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:12.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:12.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:12.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:12.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:12.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:12.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:12.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:12.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:12.676+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:13.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:13.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:13.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:13.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.077+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:13.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:13.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:13.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:13.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:13.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:13.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:13.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:13.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:13.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:13.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:13.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:13.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:13.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:13.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:13.879+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:15.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:15.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:15.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:15.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:15.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 msgr2=0x7f0ed800a2f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0efc12fe60 0x7f0ed800a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:15.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:15.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc14ade0 con 0x7f0ed800ac70 2020-11-20T12:10:15.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc130690 con 0x7f0ed8003400 2020-11-20T12:10:15.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800a830 con 0x7f0efc14a470 2020-11-20T12:10:15.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:15.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:15.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:15.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:15.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:15.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:15.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:15.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:15.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:15.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:15.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:15.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:15.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:15.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:15.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.476+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:15.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:15.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:15.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:15.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:15.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:15.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:15.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:15.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:15.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:15.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:15.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:15.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:15.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:15.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:15.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:15.677+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:16.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:16.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:16.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:16.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.078+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:16.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:16.880+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:18.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:18.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed8005850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed8005850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 msgr2=0x7f0ed800d060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800ac70 0x7f0ed800d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:18.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc14ade0 con 0x7f0ed800b050 2020-11-20T12:10:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc130690 con 0x7f0ed8003400 2020-11-20T12:10:18.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef2ffd700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800a830 con 0x7f0efc14a470 2020-11-20T12:10:18.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:18.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:18.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:18.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:18.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:18.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:18.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:18.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.476+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:18.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:18.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:18.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:18.677+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:19.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:19.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:19.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:19.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:19.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:19.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:19.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:19.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:10:19.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed80057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:19.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:19.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:10:19.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:19.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0f00c31700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:19.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.078+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:10:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0efbfff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:10:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 msgr2=0x7f0ed8007d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 msgr2=0x7f0ed800b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:19.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0ef3fff700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc130b90 con 0x7f0ed8003400 2020-11-20T12:10:19.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.879+0000 7f0f00c31700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:19.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.880+0000 7f0ef3fff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0efc065b90 tx=0x7f0ee4029000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:19.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.880+0000 7f0ef8ff9700 1 -- 172.21.15.44:0/316280478 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4025630 con 0x7f0ed8003400 2020-11-20T12:10:19.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.880+0000 7f0ef8ff9700 1 -- 172.21.15.44:0/316280478 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee403a040 con 0x7f0ed8003400 2020-11-20T12:10:19.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.880+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc130e30 con 0x7f0ed8003400 2020-11-20T12:10:19.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.880+0000 7f0ef8ff9700 1 -- 172.21.15.44:0/316280478 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4040660 con 0x7f0ed8003400 2020-11-20T12:10:19.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.880+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc1302e0 con 0x7f0ed8003400 2020-11-20T12:10:19.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.881+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc130fc0 con 0x7f0ed8003400 2020-11-20T12:10:19.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.881+0000 7f0ef8ff9700 1 -- 172.21.15.44:0/316280478 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0ee402e040 con 0x7f0ed8003400 2020-11-20T12:10:19.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.881+0000 7f0ef8ff9700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ecc056d20 0x7f0ecc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:19.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.882+0000 7f0ef8ff9700 1 -- 172.21.15.44:0/316280478 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0ee4075dc0 con 0x7f0ed8003400 2020-11-20T12:10:19.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.882+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ecc056d20 0x7f0ecc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:19.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.882+0000 7f0efbfff700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ecc056d20 0x7f0ecc0591e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f0eec000bc0 tx=0x7f0eec008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:19.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:19.886+0000 7f0ef8ff9700 1 -- 172.21.15.44:0/316280478 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0ee404d2d0 con 0x7f0ed8003400 2020-11-20T12:10:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.142+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0efc061650 con 0x7f0ed8003400 2020-11-20T12:10:20.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.143+0000 7f0ef8ff9700 1 -- 172.21.15.44:0/316280478 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7f0ee40027f0 con 0x7f0ed8003400 2020-11-20T12:10:20.148 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":80,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:10:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ecc056d20 msgr2=0x7f0ecc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ecc056d20 0x7f0ecc0591e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f0eec000bc0 tx=0x7f0eec008040).stop 2020-11-20T12:10:20.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 msgr2=0x7f0ed80057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:20.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0efc065b90 tx=0x7f0ee4029000).stop 2020-11-20T12:10:20.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 shutdown_connections 2020-11-20T12:10:20.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0ecc056d20 0x7f0ecc0591e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:20.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0efc14a470 0x7f0ed8007d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:20.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ed8003400 0x7f0ed80057f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:20.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 --2- 172.21.15.44:0/316280478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed800b050 0x7f0ed800b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:20.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 >> 172.21.15.44:0/316280478 conn(0x7f0efc062690 msgr2=0x7f0efc0a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:20.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 shutdown_connections 2020-11-20T12:10:20.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:20.145+0000 7f0f02694700 1 -- 172.21.15.44:0/316280478 wait complete. 2020-11-20T12:10:23.161 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:10:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.297+0000 7fd6d15e8700 1 Processor -- start 2020-11-20T12:10:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.297+0000 7fd6d15e8700 1 -- start start 2020-11-20T12:10:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.298+0000 7fd6d15e8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.298+0000 7fd6d15e8700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0a9770 con 0x7fd6cc13ac60 2020-11-20T12:10:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.298+0000 7fd6d15e8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0a6da0 con 0x7fd6cc0aa020 2020-11-20T12:10:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.298+0000 7fd6d15e8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0a56c0 con 0x7fd6cc0aa400 2020-11-20T12:10:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.298+0000 7fd6cb7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd6cc13ac60 0x7fd6cc0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43280/0 (socket says 172.21.15.44:43280) 2020-11-20T12:10:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.298+0000 7fd6cb7fe700 1 -- 172.21.15.44:0/2476114950 learned_addr learned my addr 172.21.15.44:0/2476114950 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.298+0000 7fd6ca7fc700 1 --2- 172.21.15.44:0/2476114950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6c9ffb700 1 -- 172.21.15.44:0/2476114950 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022045201 0 0) 0x7fd6c4001450 con 0x7fd6cc13ac60 2020-11-20T12:10:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6c9ffb700 1 -- 172.21.15.44:0/2476114950 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ac003240 con 0x7fd6cc13ac60 2020-11-20T12:10:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6c9ffb700 1 -- 172.21.15.44:0/2476114950 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792312361 0 0) 0x7fd6cc0a6da0 con 0x7fd6cc0aa020 2020-11-20T12:10:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6c9ffb700 1 -- 172.21.15.44:0/2476114950 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c4001450 con 0x7fd6cc0aa020 2020-11-20T12:10:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6ca7fc700 1 -- 172.21.15.44:0/2476114950 >> v1:172.21.15.79:6789/0 conn(0x7fd6cc0aa020 legacy=0x7fd6cc0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6ca7fc700 1 -- 172.21.15.44:0/2476114950 >> v1:172.21.15.44:6789/0 conn(0x7fd6cc13ac60 legacy=0x7fd6cc0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6ca7fc700 1 -- 172.21.15.44:0/2476114950 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0a7040 con 0x7fd6cc0aa400 2020-11-20T12:10:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6ca7fc700 1 --2- 172.21.15.44:0/2476114950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc0a9280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd6bc00b9d0 tx=0x7fd6bc006df0).ready entity=mon.2 client_cookie=bc1266acc7df031d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6c9ffb700 1 -- 172.21.15.44:0/2476114950 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc009680 con 0x7fd6cc0aa400 2020-11-20T12:10:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6c9ffb700 1 -- 172.21.15.44:0/2476114950 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0097e0 con 0x7fd6cc0aa400 2020-11-20T12:10:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.299+0000 7fd6c9ffb700 1 -- 172.21.15.44:0/2476114950 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc009b60 con 0x7fd6cc0aa400 2020-11-20T12:10:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.300+0000 7fd6d15e8700 1 -- 172.21.15.44:0/2476114950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 msgr2=0x7fd6cc0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.300+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/2476114950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc0a9280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd6bc00b9d0 tx=0x7fd6bc006df0).stop 2020-11-20T12:10:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.300+0000 7fd6d15e8700 1 -- 172.21.15.44:0/2476114950 shutdown_connections 2020-11-20T12:10:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.300+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/2476114950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc0a9280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.300+0000 7fd6d15e8700 1 -- 172.21.15.44:0/2476114950 >> 172.21.15.44:0/2476114950 conn(0x7fd6cc062690 msgr2=0x7fd6cc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.300+0000 7fd6d15e8700 1 -- 172.21.15.44:0/2476114950 shutdown_connections 2020-11-20T12:10:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.300+0000 7fd6d15e8700 1 -- 172.21.15.44:0/2476114950 wait complete. 2020-11-20T12:10:23.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 Processor -- start 2020-11-20T12:10:23.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 -- start start 2020-11-20T12:10:23.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6cc0aa020 0x7fd6cc134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc060330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc13ac60 0x7fd6cc067470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc060f40 con 0x7fd6cc13ac60 2020-11-20T12:10:23.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc134c60 con 0x7fd6cc0aa020 2020-11-20T12:10:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.301+0000 7fd6d15e8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc134f00 con 0x7fd6cc0aa400 2020-11-20T12:10:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc13ac60 0x7fd6cc067470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc13ac60 0x7fd6cc067470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39538/0 (socket says 172.21.15.44:39538) 2020-11-20T12:10:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 -- 172.21.15.44:0/65788956 learned_addr learned my addr 172.21.15.44:0/65788956 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6ca7fc700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc060330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6caffd700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6cc0aa020 0x7fd6cc134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 -- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 msgr2=0x7fd6cc060330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc060330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 -- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6cc0aa020 msgr2=0x7fd6cc134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6cc0aa020 0x7fd6cc134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 -- 172.21.15.44:0/65788956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc133e80 con 0x7fd6cc13ac60 2020-11-20T12:10:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6cb7fe700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc13ac60 0x7fd6cc067470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd6c40016b0 tx=0x7fd6c4007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6c2ffd700 1 -- 172.21.15.44:0/65788956 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4005b80 con 0x7fd6cc13ac60 2020-11-20T12:10:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6c2ffd700 1 -- 172.21.15.44:0/65788956 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4005ce0 con 0x7fd6cc13ac60 2020-11-20T12:10:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc1348c0 con 0x7fd6cc13ac60 2020-11-20T12:10:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6c2ffd700 1 -- 172.21.15.44:0/65788956 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4007320 con 0x7fd6cc13ac60 2020-11-20T12:10:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.302+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc1327c0 con 0x7fd6cc13ac60 2020-11-20T12:10:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.304+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc1340d0 con 0x7fd6cc13ac60 2020-11-20T12:10:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.304+0000 7fd6c2ffd700 1 -- 172.21.15.44:0/65788956 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd6c4010030 con 0x7fd6cc13ac60 2020-11-20T12:10:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.304+0000 7fd6c2ffd700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6a8056d20 0x7fd6a80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.304+0000 7fd6c2ffd700 1 -- 172.21.15.44:0/65788956 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd6c406ff70 con 0x7fd6cc13ac60 2020-11-20T12:10:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.304+0000 7fd6caffd700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6a8056d20 0x7fd6a80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.305+0000 7fd6caffd700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6a8056d20 0x7fd6a80591e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fd6b4000bc0 tx=0x7fd6b4007a00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.308+0000 7fd6c2ffd700 1 -- 172.21.15.44:0/65788956 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd6cc1340d0 con 0x7fd6cc13ac60 2020-11-20T12:10:23.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.548+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd6cc0a9770 con 0x7fd6cc13ac60 2020-11-20T12:10:23.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.548+0000 7fd6c2ffd700 1 -- 172.21.15.44:0/65788956 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7fd6cc0a9770 con 0x7fd6cc13ac60 2020-11-20T12:10:23.552 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:10:23.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6a8056d20 msgr2=0x7fd6a80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6a8056d20 0x7fd6a80591e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fd6b4000bc0 tx=0x7fd6b4007a00).stop 2020-11-20T12:10:23.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc13ac60 msgr2=0x7fd6cc067470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc13ac60 0x7fd6cc067470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd6c40016b0 tx=0x7fd6c4007e20).stop 2020-11-20T12:10:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 shutdown_connections 2020-11-20T12:10:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6a8056d20 0x7fd6a80591e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6cc13ac60 0x7fd6cc067470 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6cc0aa020 0x7fd6cc134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 --2- 172.21.15.44:0/65788956 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6cc0aa400 0x7fd6cc060330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 >> 172.21.15.44:0/65788956 conn(0x7fd6cc062690 msgr2=0x7fd6cc135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:23.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 shutdown_connections 2020-11-20T12:10:23.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.550+0000 7fd6d15e8700 1 -- 172.21.15.44:0/65788956 wait complete. 2020-11-20T12:10:23.565 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:10:23.565 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:10:23.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.698+0000 7f486a1ce700 1 Processor -- start 2020-11-20T12:10:23.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.698+0000 7f486a1ce700 1 -- start start 2020-11-20T12:10:23.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f486a1ce700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48640a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f486a1ce700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f48640a4c30 0x7f486413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f486a1ce700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48640a6260 con 0x7f486413ae80 2020-11-20T12:10:23.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f486a1ce700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48640a52e0 con 0x7f48640a4c30 2020-11-20T12:10:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f486a1ce700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48640aa730 con 0x7f4864060330 2020-11-20T12:10:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f4862ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f48640a4c30 0x7f486413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f4862ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f48640a4c30 0x7f486413a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39544/0 (socket says 172.21.15.44:39544) 2020-11-20T12:10:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f4862ffd700 1 -- 172.21.15.44:0/1709118564 learned_addr learned my addr 172.21.15.44:0/1709118564 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:23.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.699+0000 7f48637fe700 1 --2- 172.21.15.44:0/1709118564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48640a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f4862ffd700 1 -- 172.21.15.44:0/1709118564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 msgr2=0x7f48640a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f4862ffd700 1 --2- 172.21.15.44:0/1709118564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48640a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f4862ffd700 1 -- 172.21.15.44:0/1709118564 >> v1:172.21.15.79:6789/0 conn(0x7f486413ae80 legacy=0x7f48640aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f4862ffd700 1 -- 172.21.15.44:0/1709118564 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48640a6490 con 0x7f48640a4c30 2020-11-20T12:10:23.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f48637fe700 1 --2- 172.21.15.44:0/1709118564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48640a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:10:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f4862ffd700 1 --2- 172.21.15.44:0/1709118564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f486413a990 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f485800ba20 tx=0x7f4858006eb0).ready entity=mon.0 client_cookie=bd036b37e2dd7542 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f48627fc700 1 -- 172.21.15.44:0/1709118564 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858009890 con 0x7f48640a4c30 2020-11-20T12:10:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f48627fc700 1 -- 172.21.15.44:0/1709118564 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48580099f0 con 0x7f48640a4c30 2020-11-20T12:10:23.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f48627fc700 1 -- 172.21.15.44:0/1709118564 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858009d70 con 0x7f48640a4c30 2020-11-20T12:10:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f486a1ce700 1 -- 172.21.15.44:0/1709118564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 msgr2=0x7f486413a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f486a1ce700 1 --2- 172.21.15.44:0/1709118564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f486413a990 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f485800ba20 tx=0x7f4858006eb0).stop 2020-11-20T12:10:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f486a1ce700 1 -- 172.21.15.44:0/1709118564 shutdown_connections 2020-11-20T12:10:23.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f486a1ce700 1 --2- 172.21.15.44:0/1709118564 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48640a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f486a1ce700 1 --2- 172.21.15.44:0/1709118564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f486413a990 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f486a1ce700 1 -- 172.21.15.44:0/1709118564 >> 172.21.15.44:0/1709118564 conn(0x7f4864062690 msgr2=0x7f48640a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.700+0000 7f486a1ce700 1 -- 172.21.15.44:0/1709118564 shutdown_connections 2020-11-20T12:10:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.701+0000 7f486a1ce700 1 -- 172.21.15.44:0/1709118564 wait complete. 2020-11-20T12:10:23.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 Processor -- start 2020-11-20T12:10:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 -- start start 2020-11-20T12:10:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48641376b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f4864133420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f486413ae80 0x7f48640abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4864133a50 con 0x7f48640a4c30 2020-11-20T12:10:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4864133cf0 con 0x7f486413ae80 2020-11-20T12:10:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f486a1ce700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4864146e50 con 0x7f4864060330 2020-11-20T12:10:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f4864133420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f4864133420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39552/0 (socket says 172.21.15.44:39552) 2020-11-20T12:10:23.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f4862ffd700 1 -- 172.21.15.44:0/3526722228 learned_addr learned my addr 172.21.15.44:0/3526722228 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f48637fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48641376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.702+0000 7f4863fff700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f486413ae80 0x7f48640abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f4862ffd700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 msgr2=0x7f48641376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f4862ffd700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4864060330 0x7f48641376b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f4862ffd700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f486413ae80 msgr2=0x7f48640abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f4862ffd700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f486413ae80 0x7f48640abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f4862ffd700 1 -- 172.21.15.44:0/3526722228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48641472f0 con 0x7f48640a4c30 2020-11-20T12:10:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f4862ffd700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f4864133420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f4858010000 tx=0x7f4858015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858011c80 con 0x7f48640a4c30 2020-11-20T12:10:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4858011de0 con 0x7f48640a4c30 2020-11-20T12:10:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4864147540 con 0x7f48640a4c30 2020-11-20T12:10:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485801d6a0 con 0x7f48640a4c30 2020-11-20T12:10:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.703+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4864147a30 con 0x7f48640a4c30 2020-11-20T12:10:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f486a1ce700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486412e160 0x7f486412e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4864133910 con 0x7f486412e160 2020-11-20T12:10:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f48637fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486412e160 0x7f486412e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f485800f070 con 0x7f48640a4c30 2020-11-20T12:10:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f48637fe700 1 -- 172.21.15.44:0/3526722228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4864147c50 con 0x7f486412e160 2020-11-20T12:10:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f48637fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486412e160 0x7f486412e540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f4854010b00 tx=0x7f4854008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f48437fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f483c056d20 0x7f483c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.704+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4858071ee0 con 0x7f48640a4c30 2020-11-20T12:10:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485400ea90 con 0x7f486412e160 2020-11-20T12:10:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f4863fff700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f483c056d20 0x7f483c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f485400ebf0 con 0x7f486412e160 2020-11-20T12:10:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486412e160 msgr2=0x7f486412e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f48437fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486412e160 0x7f486412e540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f4854010b00 tx=0x7f4854008040).stop 2020-11-20T12:10:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486412e160 msgr2=0x7f486412e540 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f48437fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486412e160 0x7f486412e540 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.705+0000 7f4863fff700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f483c056d20 0x7f483c0591e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f485c0012a0 tx=0x7f485c007f10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.712+0000 7f486a1ce700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486414a290 0x7f486414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.712+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4864061650 con 0x7f486414a290 2020-11-20T12:10:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.712+0000 7f48637fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486414a290 0x7f486414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48637fe700 1 -- 172.21.15.44:0/3526722228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f48640446d0 con 0x7f486414a290 2020-11-20T12:10:23.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48637fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486414a290 0x7f486414a6a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4854010b00 tx=0x7f4854008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854006240 con 0x7f486414a290 2020-11-20T12:10:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7f485400e880 con 0x7f486414a290 2020-11-20T12:10:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486414a290 msgr2=0x7f486414a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48437fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486414a290 0x7f486414a6a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4854010b00 tx=0x7f4854008040).stop 2020-11-20T12:10:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48437fe700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486414a290 msgr2=0x7f486414a6a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f48437fe700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f486414a290 0x7f486414a6a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f483c056d20 msgr2=0x7f483c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f486a1ce700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f483c056d20 0x7f483c0591e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f485c0012a0 tx=0x7f485c007f10).stop 2020-11-20T12:10:23.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 msgr2=0x7f4864133420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f486a1ce700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f4864133420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f4858010000 tx=0x7f4858015000).stop 2020-11-20T12:10:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.713+0000 7f4863fff700 1 -- 172.21.15.44:0/3526722228 reap_dead start 2020-11-20T12:10:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.714+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 shutdown_connections 2020-11-20T12:10:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.714+0000 7f486a1ce700 1 --2- 172.21.15.44:0/3526722228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48640a4c30 0x7f4864133420 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.714+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 >> 172.21.15.44:0/3526722228 conn(0x7f4864062690 msgr2=0x7f4864131100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.714+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 shutdown_connections 2020-11-20T12:10:23.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.714+0000 7f486a1ce700 1 -- 172.21.15.44:0/3526722228 wait complete. 2020-11-20T12:10:23.730 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:10:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 82, 2020-11-20T12:10:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:10:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:10:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:23.762 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:23.763 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:10:23.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.887+0000 7fa8d3b03700 1 Processor -- start 2020-11-20T12:10:23.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.887+0000 7fa8d3b03700 1 -- start start 2020-11-20T12:10:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.887+0000 7fa8d3b03700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa020 0x7fa8cc0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.887+0000 7fa8d3b03700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa8cc13ac60 0x7fa8cc0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.887+0000 7fa8d3b03700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0a9770 con 0x7fa8cc0aa400 2020-11-20T12:10:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.887+0000 7fa8d3b03700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0a56c0 con 0x7fa8cc13ac60 2020-11-20T12:10:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.887+0000 7fa8d3b03700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0a6da0 con 0x7fa8cc0aa020 2020-11-20T12:10:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa8cc13ac60 0x7fa8cc0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa8cc13ac60 0x7fa8cc0a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39560/0 (socket says 172.21.15.44:39560) 2020-11-20T12:10:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 -- 172.21.15.44:0/2760235022 learned_addr learned my addr 172.21.15.44:0/2760235022 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:23.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d189f700 1 --2- 172.21.15.44:0/2760235022 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa020 0x7fa8cc0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 -- 172.21.15.44:0/2760235022 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa020 msgr2=0x7fa8cc0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:23.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 --2- 172.21.15.44:0/2760235022 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa020 0x7fa8cc0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 -- 172.21.15.44:0/2760235022 >> v1:172.21.15.79:6789/0 conn(0x7fa8cc0aa400 legacy=0x7fa8cc0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 -- 172.21.15.44:0/2760235022 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0a7040 con 0x7fa8cc13ac60 2020-11-20T12:10:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d20a0700 1 --2- 172.21.15.44:0/2760235022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0a3700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fa8c8006fd0 tx=0x7fa8c8006eb0).ready entity=mon.0 client_cookie=30fd4c1ec72c9383 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d089d700 1 -- 172.21.15.44:0/2760235022 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8009890 con 0x7fa8cc13ac60 2020-11-20T12:10:23.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d089d700 1 -- 172.21.15.44:0/2760235022 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c80099f0 con 0x7fa8cc13ac60 2020-11-20T12:10:23.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.888+0000 7fa8d089d700 1 -- 172.21.15.44:0/2760235022 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8009d70 con 0x7fa8cc13ac60 2020-11-20T12:10:23.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 -- 172.21.15.44:0/2760235022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 msgr2=0x7fa8cc0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 --2- 172.21.15.44:0/2760235022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0a3700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fa8c8006fd0 tx=0x7fa8c8006eb0).stop 2020-11-20T12:10:23.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 -- 172.21.15.44:0/2760235022 shutdown_connections 2020-11-20T12:10:23.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 --2- 172.21.15.44:0/2760235022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0a3700 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 --2- 172.21.15.44:0/2760235022 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa020 0x7fa8cc0a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 -- 172.21.15.44:0/2760235022 >> 172.21.15.44:0/2760235022 conn(0x7fa8cc062690 msgr2=0x7fa8cc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:23.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 -- 172.21.15.44:0/2760235022 shutdown_connections 2020-11-20T12:10:23.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.889+0000 7fa8d3b03700 1 -- 172.21.15.44:0/2760235022 wait complete. 2020-11-20T12:10:23.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 Processor -- start 2020-11-20T12:10:23.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 -- start start 2020-11-20T12:10:23.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0aa020 0x7fa8cc0ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa400 0x7fa8cc13a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0ac850 con 0x7fa8cc13ac60 2020-11-20T12:10:23.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0acae0 con 0x7fa8cc0aa020 2020-11-20T12:10:23.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d3b03700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc066ab0 con 0x7fa8cc0aa400 2020-11-20T12:10:23.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d20a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d189f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0aa020 0x7fa8cc0ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d20a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0671f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39570/0 (socket says 172.21.15.44:39570) 2020-11-20T12:10:23.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.890+0000 7fa8d20a0700 1 -- 172.21.15.44:0/3328358008 learned_addr learned my addr 172.21.15.44:0/3328358008 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:23.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d109e700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa400 0x7fa8cc13a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d20a0700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa400 msgr2=0x7fa8cc13a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d20a0700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8cc0aa400 0x7fa8cc13a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d20a0700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0aa020 msgr2=0x7fa8cc0ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d20a0700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0aa020 0x7fa8cc0ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:23.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d20a0700 1 -- 172.21.15.44:0/3328358008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc066d50 con 0x7fa8cc13ac60 2020-11-20T12:10:23.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d20a0700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0671f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fa8cc061280 tx=0x7fa8c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800bec0 con 0x7fa8cc13ac60 2020-11-20T12:10:23.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c801d5d0 con 0x7fa8cc13ac60 2020-11-20T12:10:23.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc13a5a0 con 0x7fa8cc13ac60 2020-11-20T12:10:23.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c801d970 con 0x7fa8cc13ac60 2020-11-20T12:10:23.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.891+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc067c80 con 0x7fa8cc13ac60 2020-11-20T12:10:23.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.892+0000 7fa8d3b03700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc12ae30 0x7fa8cc12b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.892+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc13a730 con 0x7fa8cc12ae30 2020-11-20T12:10:23.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.892+0000 7fa8d189f700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc12ae30 0x7fa8cc12b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.892+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa8c8025030 con 0x7fa8cc13ac60 2020-11-20T12:10:23.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8d189f700 1 -- 172.21.15.44:0/3328358008 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8cc0a9480 con 0x7fa8cc12ae30 2020-11-20T12:10:23.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8d189f700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc12ae30 0x7fa8cc12b210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fa8c4011b90 tx=0x7fa8c4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa8a4056d20 0x7fa8a40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa8c8071890 con 0x7fa8cc13ac60 2020-11-20T12:10:23.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400fad0 con 0x7fa8cc12ae30 2020-11-20T12:10:23.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8d109e700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa8a4056d20 0x7fa8a40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fa8c400fc30 con 0x7fa8cc12ae30 2020-11-20T12:10:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc12ae30 msgr2=0x7fa8cc12b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc12ae30 0x7fa8cc12b210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fa8c4011b90 tx=0x7fa8c4008040).stop 2020-11-20T12:10:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc12ae30 msgr2=0x7fa8cc12b210 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8b9ffb700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc12ae30 0x7fa8cc12b210 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0x7fa8c4011b90 tx=0x7fa8c4008040).stop 2020-11-20T12:10:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.893+0000 7fa8d109e700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa8a4056d20 0x7fa8a40591e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa8cc0a9a60 tx=0x7fa8bc007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.901+0000 7fa8d3b03700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0a7740 0x7fa8cc0a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.901+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0ac990 con 0x7fa8cc0a7740 2020-11-20T12:10:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.901+0000 7fa8d189f700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0a7740 0x7fa8cc0a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8d189f700 1 -- 172.21.15.44:0/3328358008 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa8cc0a9770 con 0x7fa8cc0a7740 2020-11-20T12:10:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8d189f700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0a7740 0x7fa8cc0a7b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa8c4018000 tx=0x7fa8c401b000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4028070 con 0x7fa8cc0a7740 2020-11-20T12:10:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4435 (secure 0 0 0) 0x7fa8c400f8c0 con 0x7fa8cc0a7740 2020-11-20T12:10:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0a7740 msgr2=0x7fa8cc0a7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8b9ffb700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0a7740 0x7fa8cc0a7b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa8c4018000 tx=0x7fa8c401b000).stop 2020-11-20T12:10:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8b9ffb700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0a7740 msgr2=0x7fa8cc0a7b50 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.902+0000 7fa8b9ffb700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8cc0a7740 0x7fa8cc0a7b50 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0x7fa8c4018000 tx=0x7fa8c401b000).stop 2020-11-20T12:10:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa8a4056d20 msgr2=0x7fa8a40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa8a4056d20 0x7fa8a40591e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa8cc0a9a60 tx=0x7fa8bc007b90).stop 2020-11-20T12:10:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 msgr2=0x7fa8cc0671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 --2- 172.21.15.44:0/3328358008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8cc13ac60 0x7fa8cc0671f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fa8cc061280 tx=0x7fa8c8015000).stop 2020-11-20T12:10:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d20a0700 1 -- 172.21.15.44:0/3328358008 reap_dead start 2020-11-20T12:10:23.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d20a0700 1 -- 172.21.15.44:0/3328358008 reap_dead start 2020-11-20T12:10:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 shutdown_connections 2020-11-20T12:10:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 >> 172.21.15.44:0/3328358008 conn(0x7fa8cc062690 msgr2=0x7fa8cc136950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 shutdown_connections 2020-11-20T12:10:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:23.903+0000 7fa8d3b03700 1 -- 172.21.15.44:0/3328358008 wait complete. 2020-11-20T12:10:23.917 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:23.917 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:23.917 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:23.917 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:10:23.917 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 82, 2020-11-20T12:10:23.918 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:23.918 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:10:23.918 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:23.918 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:23.918 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.919 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:10:23.919 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:23.919 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:23.919 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:23.919 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:23.920 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:23.920 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:23.920 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:23.920 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:23.920 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:23.921 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:23.921 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:23.921 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.921 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:23.921 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:23.922 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:23.922 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:23.922 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:23.922 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:23.922 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:23.923 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:23.923 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:23.923 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:23.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.924 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:23.924 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:23.924 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:23.924 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:23.924 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:23.925 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:23.925 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:23.925 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:23.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:23.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:23.926 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:23.926 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:23.926 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:23.926 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:23.926 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:23.927 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:23.927 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:23.927 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:23.927 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:23.927 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:23.928 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:23.928 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:23.928 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:23.928 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.928 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:23.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.929 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:23.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.929 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:23.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:23.930 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:23.930 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:23.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:23.931 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:23.931 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.931 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:23.932 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:23.932 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.932 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.933 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:23.933 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:23.933 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:23.933 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:23.933 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:23.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.934 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:23.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:23.934 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:23.935 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:23.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:23.935 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:23.935 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.936 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:23.936 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:23.936 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.937 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.937 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:23.938 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:23.938 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:23.938 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:23.938 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:23.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.939 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:23.939 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:23.939 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:23.939 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:23.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.940 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:23.940 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:23.940 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:23.941 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:23.941 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:23.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.942 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.942 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:23.942 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:23.942 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:10:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:23.948 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:23.950 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:10:24.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.072+0000 7f196fc83700 1 Processor -- start 2020-11-20T12:10:24.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.072+0000 7f196fc83700 1 -- start start 2020-11-20T12:10:24.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.073+0000 7f196fc83700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19680aa400 0x7f19680a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.073+0000 7f196fc83700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680a9770 con 0x7f196813ac60 2020-11-20T12:10:24.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.073+0000 7f196fc83700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680a6da0 con 0x7f19680aa020 2020-11-20T12:10:24.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.073+0000 7f196fc83700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19680a56c0 con 0x7f19680aa400 2020-11-20T12:10:24.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.073+0000 7f196e220700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f196813ac60 0x7f19680a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43328/0 (socket says 172.21.15.44:43328) 2020-11-20T12:10:24.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.073+0000 7f196e220700 1 -- 172.21.15.44:0/2928240128 learned_addr learned my addr 172.21.15.44:0/2928240128 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:24.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.073+0000 7f196d21e700 1 --2- 172.21.15.44:0/2928240128 >> v2:172.21.15.79:3300/0 conn(0x7f19680aa400 0x7f19680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196ca1d700 1 -- 172.21.15.44:0/2928240128 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143085547 0 0) 0x7f19600010d0 con 0x7f196813ac60 2020-11-20T12:10:24.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196ca1d700 1 -- 172.21.15.44:0/2928240128 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1950003240 con 0x7f196813ac60 2020-11-20T12:10:24.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196ca1d700 1 -- 172.21.15.44:0/2928240128 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937879423 0 0) 0x7f19680a6da0 con 0x7f19680aa020 2020-11-20T12:10:24.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196ca1d700 1 -- 172.21.15.44:0/2928240128 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19600010d0 con 0x7f19680aa020 2020-11-20T12:10:24.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196d21e700 1 -- 172.21.15.44:0/2928240128 >> v1:172.21.15.79:6789/0 conn(0x7f19680aa020 legacy=0x7f19680a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196d21e700 1 -- 172.21.15.44:0/2928240128 >> v1:172.21.15.44:6789/0 conn(0x7f196813ac60 legacy=0x7f19680a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196d21e700 1 -- 172.21.15.44:0/2928240128 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680a7040 con 0x7f19680aa400 2020-11-20T12:10:24.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196d21e700 1 --2- 172.21.15.44:0/2928240128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19680aa400 0x7f19680a9280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1964009c90 tx=0x7f19640091d0).ready entity=mon.1 client_cookie=e9402b02c292779d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196ca1d700 1 -- 172.21.15.44:0/2928240128 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400b9c0 con 0x7f19680aa400 2020-11-20T12:10:24.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196ca1d700 1 -- 172.21.15.44:0/2928240128 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196400bb20 con 0x7f19680aa400 2020-11-20T12:10:24.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.074+0000 7f196ca1d700 1 -- 172.21.15.44:0/2928240128 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400bea0 con 0x7f19680aa400 2020-11-20T12:10:24.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.075+0000 7f196fc83700 1 -- 172.21.15.44:0/2928240128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19680aa400 msgr2=0x7f19680a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.075+0000 7f196fc83700 1 --2- 172.21.15.44:0/2928240128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19680aa400 0x7f19680a9280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1964009c90 tx=0x7f19640091d0).stop 2020-11-20T12:10:24.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.075+0000 7f196fc83700 1 -- 172.21.15.44:0/2928240128 shutdown_connections 2020-11-20T12:10:24.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.075+0000 7f196fc83700 1 --2- 172.21.15.44:0/2928240128 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19680aa400 0x7f19680a9280 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.075+0000 7f196fc83700 1 -- 172.21.15.44:0/2928240128 >> 172.21.15.44:0/2928240128 conn(0x7f1968062690 msgr2=0x7f19680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:24.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.075+0000 7f196fc83700 1 -- 172.21.15.44:0/2928240128 shutdown_connections 2020-11-20T12:10:24.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.075+0000 7f196fc83700 1 -- 172.21.15.44:0/2928240128 wait complete. 2020-11-20T12:10:24.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 Processor -- start 2020-11-20T12:10:24.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 -- start start 2020-11-20T12:10:24.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19680aa020 0x7f1968067510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19680aa400 0x7f19680acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f196813ac60 0x7f1968147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680ab6c0 con 0x7f19680aa400 2020-11-20T12:10:24.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680aaf80 con 0x7f196813ac60 2020-11-20T12:10:24.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196fc83700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19680ab220 con 0x7f19680aa020 2020-11-20T12:10:24.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196d21e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19680aa400 0x7f19680acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196d21e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19680aa400 0x7f19680acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39584/0 (socket says 172.21.15.44:39584) 2020-11-20T12:10:24.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196d21e700 1 -- 172.21.15.44:0/3920901673 learned_addr learned my addr 172.21.15.44:0/3920901673 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:24.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196da1f700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19680aa020 0x7f1968067510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.076+0000 7f196e220700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f196813ac60 0x7f1968147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196d21e700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19680aa020 msgr2=0x7f1968067510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196d21e700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19680aa020 0x7f1968067510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196d21e700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f196813ac60 msgr2=0x7f1968147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196d21e700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f196813ac60 0x7f1968147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196d21e700 1 -- 172.21.15.44:0/3920901673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680acfb0 con 0x7f19680aa400 2020-11-20T12:10:24.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196d21e700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19680aa400 0x7f19680acac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1964009370 tx=0x7f1964016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196401d480 con 0x7f19680aa400 2020-11-20T12:10:24.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196401d5e0 con 0x7f19680aa400 2020-11-20T12:10:24.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196401d960 con 0x7f19680aa400 2020-11-20T12:10:24.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19680ad600 con 0x7f19680aa400 2020-11-20T12:10:24.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.077+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1968148220 con 0x7f19680aa400 2020-11-20T12:10:24.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.078+0000 7f196fc83700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19681310c0 0x7f19681314a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.078+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1968061650 con 0x7f19681310c0 2020-11-20T12:10:24.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.078+0000 7f196da1f700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19681310c0 0x7f19681314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.078+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1964017070 con 0x7f19680aa400 2020-11-20T12:10:24.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f196da1f700 1 -- 172.21.15.44:0/3920901673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19680446d0 con 0x7f19681310c0 2020-11-20T12:10:24.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1940056d50 0x7f1940059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f19640723e0 con 0x7f19680aa400 2020-11-20T12:10:24.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f196da1f700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19681310c0 0x7f19681314a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f1958002920 tx=0x7f1958007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f196e220700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1940056d50 0x7f1940059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195800f930 con 0x7f19681310c0 2020-11-20T12:10:24.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f195800fa90 con 0x7f19681310c0 2020-11-20T12:10:24.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f196e220700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1940056d50 0x7f1940059210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f1960001330 tx=0x7f1960009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19681310c0 msgr2=0x7f19681314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19681310c0 0x7f19681314a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f1958002920 tx=0x7f1958007a20).stop 2020-11-20T12:10:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19681310c0 msgr2=0x7f19681314a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:24.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.079+0000 7f1955ffb700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f19681310c0 0x7f19681314a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.087+0000 7f196fc83700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1968060170 0x7f1968060580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.087+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1968069560 con 0x7f1968060170 2020-11-20T12:10:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.087+0000 7f196da1f700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1968060170 0x7f1968060580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.087+0000 7f196da1f700 1 -- 172.21.15.44:0/3920901673 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1968061350 con 0x7f1968060170 2020-11-20T12:10:24.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196da1f700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1968060170 0x7f1968060580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f1958002920 tx=0x7f1958017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958014e40 con 0x7f1968060170 2020-11-20T12:10:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4775 (secure 0 0 0) 0x7f195800f720 con 0x7f1968060170 2020-11-20T12:10:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1968060170 msgr2=0x7f1968060580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f1955ffb700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1968060170 0x7f1968060580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f1958002920 tx=0x7f1958017000).stop 2020-11-20T12:10:24.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f1955ffb700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1968060170 msgr2=0x7f1968060580 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f1955ffb700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1968060170 0x7f1968060580 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0x7f1958002920 tx=0x7f1958017000).stop 2020-11-20T12:10:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1940056d50 msgr2=0x7f1940059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196fc83700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1940056d50 0x7f1940059210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f1960001330 tx=0x7f1960009040).stop 2020-11-20T12:10:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19680aa400 msgr2=0x7f19680acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196fc83700 1 --2- 172.21.15.44:0/3920901673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19680aa400 0x7f19680acac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1964009370 tx=0x7f1964016000).stop 2020-11-20T12:10:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196e220700 1 -- 172.21.15.44:0/3920901673 reap_dead start 2020-11-20T12:10:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196e220700 1 -- 172.21.15.44:0/3920901673 reap_dead start 2020-11-20T12:10:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 shutdown_connections 2020-11-20T12:10:24.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 >> 172.21.15.44:0/3920901673 conn(0x7f1968062690 msgr2=0x7f1968136c40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:24.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.088+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 shutdown_connections 2020-11-20T12:10:24.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.089+0000 7f196fc83700 1 -- 172.21.15.44:0/3920901673 wait complete. 2020-11-20T12:10:24.109 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:24.110 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:10:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 82, 2020-11-20T12:10:24.111 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:10:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:24.112 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:10:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:24.113 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:24.114 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:24.115 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:24.116 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:24.116 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:24.116 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:24.116 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:24.116 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:24.117 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:24.117 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:24.117 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:24.117 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:24.117 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:24.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.118 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:24.118 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:24.118 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:24.119 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:24.119 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:24.119 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:24.119 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:24.119 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:24.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:24.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:24.120 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:24.120 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:24.120 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:24.121 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:24.121 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:24.121 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:24.122 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:24.122 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:24.122 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:24.122 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:24.122 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:24.123 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:24.123 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:24.123 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:24.123 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:24.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.124 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:24.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.124 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:24.124 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:24.124 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:24.125 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:24.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:24.125 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:24.126 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:24.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:24.127 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:24.127 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:24.127 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.127 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:24.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.128 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:24.128 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:24.128 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:24.128 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:24.128 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:24.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.129 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:24.129 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:24.129 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:24.130 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:24.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:24.130 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:24.130 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:24.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:24.131 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:24.132 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:24.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.132 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:24.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.132 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:24.133 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:24.133 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:24.133 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:24.133 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:24.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.134 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:24.134 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:24.134 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:24.134 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:24.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:24.135 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:24.135 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:24.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:10:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:10:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:10:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:10:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:10:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:24.147 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:24.147 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:10:24.148 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:10:24.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.283+0000 7fa7c13f7700 1 Processor -- start 2020-11-20T12:10:24.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.283+0000 7fa7c13f7700 1 -- start start 2020-11-20T12:10:24.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7c13f7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa7bc13a990 0x7fa7bc13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7c13f7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0a6260 con 0x7fa7bc13f800 2020-11-20T12:10:24.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7c13f7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0a6490 con 0x7fa7bc060330 2020-11-20T12:10:24.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7c13f7700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc0a52e0 con 0x7fa7bc13a990 2020-11-20T12:10:24.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7ba7fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa7bc13a990 0x7fa7bc13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7bb7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa7bc13f800 0x7fa7bc141bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43346/0 (socket says 172.21.15.44:43346) 2020-11-20T12:10:24.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7bb7fe700 1 -- 172.21.15.44:0/1168887710 learned_addr learned my addr 172.21.15.44:0/1168887710 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:24.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.284+0000 7fa7ba7fc700 1 -- 172.21.15.44:0/1168887710 >> v1:172.21.15.79:6789/0 conn(0x7fa7bc060330 legacy=0x7fa7bc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:24.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7ba7fc700 1 -- 172.21.15.44:0/1168887710 >> v1:172.21.15.44:6789/0 conn(0x7fa7bc13f800 legacy=0x7fa7bc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7b9ffb700 1 -- 172.21.15.44:0/1168887710 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965125082 0 0) 0x7fa7bc0a6260 con 0x7fa7bc13f800 2020-11-20T12:10:24.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7ba7fc700 1 -- 172.21.15.44:0/1168887710 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0a6730 con 0x7fa7bc13a990 2020-11-20T12:10:24.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7ba7fc700 1 --2- 172.21.15.44:0/1168887710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc13ae20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa7ac00b9a0 tx=0x7fa7ac006df0).ready entity=mon.0 client_cookie=8752c203773635dd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7b9ffb700 1 -- 172.21.15.44:0/1168887710 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0097b0 con 0x7fa7bc13a990 2020-11-20T12:10:24.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7b9ffb700 1 -- 172.21.15.44:0/1168887710 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac009910 con 0x7fa7bc13a990 2020-11-20T12:10:24.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7b9ffb700 1 -- 172.21.15.44:0/1168887710 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac009c90 con 0x7fa7bc13a990 2020-11-20T12:10:24.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7c13f7700 1 -- 172.21.15.44:0/1168887710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 msgr2=0x7fa7bc13ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/1168887710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc13ae20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa7ac00b9a0 tx=0x7fa7ac006df0).stop 2020-11-20T12:10:24.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7c13f7700 1 -- 172.21.15.44:0/1168887710 shutdown_connections 2020-11-20T12:10:24.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/1168887710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc13ae20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7c13f7700 1 -- 172.21.15.44:0/1168887710 >> 172.21.15.44:0/1168887710 conn(0x7fa7bc062690 msgr2=0x7fa7bc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:24.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7c13f7700 1 -- 172.21.15.44:0/1168887710 shutdown_connections 2020-11-20T12:10:24.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.285+0000 7fa7c13f7700 1 -- 172.21.15.44:0/1168887710 wait complete. 2020-11-20T12:10:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.286+0000 7fa7c13f7700 1 Processor -- start 2020-11-20T12:10:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.286+0000 7fa7c13f7700 1 -- start start 2020-11-20T12:10:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7c13f7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7bc060330 0x7fa7bc134e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7c13f7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc14a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7c13f7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa7bc13f800 0x7fa7bc0a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7c13f7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc1389c0 con 0x7fa7bc13a990 2020-11-20T12:10:24.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7c13f7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc138720 con 0x7fa7bc060330 2020-11-20T12:10:24.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7c13f7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc066140 con 0x7fa7bc13f800 2020-11-20T12:10:24.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc14a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39602/0 (socket says 172.21.15.44:39602) 2020-11-20T12:10:24.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 -- 172.21.15.44:0/3177323350 learned_addr learned my addr 172.21.15.44:0/3177323350 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:24.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7baffd700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7bc060330 0x7fa7bc134e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7bb7fe700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa7bc13f800 0x7fa7bc0a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 -- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa7bc13f800 msgr2=0x7fa7bc0a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa7bc13f800 0x7fa7bc0a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 -- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7bc060330 msgr2=0x7fa7bc134e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7bc060330 0x7fa7bc134e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.287+0000 7fa7ba7fc700 1 -- 172.21.15.44:0/3177323350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc14a540 con 0x7fa7bc13a990 2020-11-20T12:10:24.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.288+0000 7fa7ba7fc700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc14a050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa7ac006e30 tx=0x7fa7ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.288+0000 7fa7b2ffd700 1 -- 172.21.15.44:0/3177323350 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac01e460 con 0x7fa7bc13a990 2020-11-20T12:10:24.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.288+0000 7fa7b2ffd700 1 -- 172.21.15.44:0/3177323350 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac01e5c0 con 0x7fa7bc13a990 2020-11-20T12:10:24.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.288+0000 7fa7b2ffd700 1 -- 172.21.15.44:0/3177323350 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac01e960 con 0x7fa7bc13a990 2020-11-20T12:10:24.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.288+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc130d70 con 0x7fa7bc13a990 2020-11-20T12:10:24.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.288+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc067c80 con 0x7fa7bc13a990 2020-11-20T12:10:24.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.289+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc066300 con 0x7fa7bc13a990 2020-11-20T12:10:24.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.289+0000 7fa7b2ffd700 1 -- 172.21.15.44:0/3177323350 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa7ac025030 con 0x7fa7bc13a990 2020-11-20T12:10:24.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.289+0000 7fa7b2ffd700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa798056d20 0x7fa7980591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:24.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.289+0000 7fa7b2ffd700 1 -- 172.21.15.44:0/3177323350 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa7ac071970 con 0x7fa7bc13a990 2020-11-20T12:10:24.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.289+0000 7fa7baffd700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa798056d20 0x7fa7980591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:24.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.290+0000 7fa7baffd700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa798056d20 0x7fa7980591e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fa7a4003750 tx=0x7fa7a4007b70).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:24.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.294+0000 7fa7b2ffd700 1 -- 172.21.15.44:0/3177323350 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa7ac046cd0 con 0x7fa7bc13a990 2020-11-20T12:10:24.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.535+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fa7bc061650 con 0x7fa7bc13a990 2020-11-20T12:10:24.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.535+0000 7fa7b2ffd700 1 -- 172.21.15.44:0/3177323350 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fa7ac023070 con 0x7fa7bc13a990 2020-11-20T12:10:24.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa798056d20 msgr2=0x7fa7980591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa798056d20 0x7fa7980591e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fa7a4003750 tx=0x7fa7a4007b70).stop 2020-11-20T12:10:24.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 msgr2=0x7fa7bc14a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:24.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc14a050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa7ac006e30 tx=0x7fa7ac015000).stop 2020-11-20T12:10:24.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 shutdown_connections 2020-11-20T12:10:24.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa798056d20 0x7fa7980591e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa7bc13f800 0x7fa7bc0a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa7bc060330 0x7fa7bc134e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 --2- 172.21.15.44:0/3177323350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7bc13a990 0x7fa7bc14a050 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:24.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 >> 172.21.15.44:0/3177323350 conn(0x7fa7bc062690 msgr2=0x7fa7bc1314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:24.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 shutdown_connections 2020-11-20T12:10:24.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:24.537+0000 7fa7c13f7700 1 -- 172.21.15.44:0/3177323350 wait complete. 2020-11-20T12:10:24.559 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:10:34.560 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:10:34.561 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:10:34.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.696+0000 7fb6a763a700 1 Processor -- start 2020-11-20T12:10:34.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.696+0000 7fb6a763a700 1 -- start start 2020-11-20T12:10:34.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.696+0000 7fb6a763a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 0x7fb6a00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.696+0000 7fb6a763a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a00a9770 con 0x7fb6a00aa400 2020-11-20T12:10:34.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.696+0000 7fb6a763a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a00a6da0 con 0x7fb6a00aa020 2020-11-20T12:10:34.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.696+0000 7fb6a763a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00a56c0 con 0x7fb6a00a3e10 2020-11-20T12:10:34.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb6a4bd5700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb6a00aa400 0x7fb6a00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43360/0 (socket says 172.21.15.44:43360) 2020-11-20T12:10:34.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb6a4bd5700 1 -- 172.21.15.44:0/2116125367 learned_addr learned my addr 172.21.15.44:0/2116125367 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:34.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb6a5bd7700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 0x7fb6a00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755587873 0 0) 0x7fb6940010d0 con 0x7fb6a00aa400 2020-11-20T12:10:34.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb68c003240 con 0x7fb6a00aa400 2020-11-20T12:10:34.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3854564642 0 0) 0x7fb68c003240 con 0x7fb6a00aa400 2020-11-20T12:10:34.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 msgr2=0x7fb6a00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:34.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 0x7fb6a00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 >> v1:172.21.15.79:6789/0 conn(0x7fb6a00aa020 legacy=0x7fb6a00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00a7040 con 0x7fb6a00aa400 2020-11-20T12:10:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fb694001670 con 0x7fb6a00aa400 2020-11-20T12:10:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 >> v1:172.21.15.44:6789/0 conn(0x7fb6a00aa400 legacy=0x7fb6a00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:34.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb68c003760 0x7fb68c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb68c0040a0 0x7fb68c004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 0x7fb68c004be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00a56c0 con 0x7fb68c003760 2020-11-20T12:10:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c005120 con 0x7fb68c0040a0 2020-11-20T12:10:34.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c0052e0 con 0x7fb6a00a3e10 2020-11-20T12:10:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.697+0000 7fb6a53d6700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb68c003760 0x7fb68c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a4bd5700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb68c0040a0 0x7fb68c004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a5bd7700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 0x7fb68c004be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a53d6700 1 -- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 msgr2=0x7fb68c004be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:34.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a53d6700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a00a3e10 0x7fb68c004be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a53d6700 1 -- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb68c0040a0 msgr2=0x7fb68c004580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:34.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a53d6700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb68c0040a0 0x7fb68c004580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a53d6700 1 -- 172.21.15.44:0/2116125367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c003240 con 0x7fb68c003760 2020-11-20T12:10:34.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a4bd5700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb68c0040a0 0x7fb68c004580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:34.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb6a53d6700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb68c003760 0x7fb68c003b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb698001f20 tx=0x7fb698007ef0).ready entity=mon.0 client_cookie=87ec2d70a5880afc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:34.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698001610 con 0x7fb68c003760 2020-11-20T12:10:34.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb698015070 con 0x7fb68c003760 2020-11-20T12:10:34.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.698+0000 7fb69ffff700 1 -- 172.21.15.44:0/2116125367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69800cb10 con 0x7fb68c003760 2020-11-20T12:10:34.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.699+0000 7fb6a763a700 1 -- 172.21.15.44:0/2116125367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb68c003760 msgr2=0x7fb68c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:34.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.699+0000 7fb6a763a700 1 --2- 172.21.15.44:0/2116125367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb68c003760 0x7fb68c003b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb698001f20 tx=0x7fb698007ef0).stop 2020-11-20T12:10:34.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.699+0000 7fb6a5bd7700 1 -- 172.21.15.44:0/2116125367 reap_dead start 2020-11-20T12:10:34.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.699+0000 7fb6a763a700 1 -- 172.21.15.44:0/2116125367 shutdown_connections 2020-11-20T12:10:34.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.699+0000 7fb6a763a700 1 -- 172.21.15.44:0/2116125367 >> 172.21.15.44:0/2116125367 conn(0x7fb6a0062690 msgr2=0x7fb6a00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.699+0000 7fb6a763a700 1 -- 172.21.15.44:0/2116125367 shutdown_connections 2020-11-20T12:10:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.699+0000 7fb6a763a700 1 -- 172.21.15.44:0/2116125367 wait complete. 2020-11-20T12:10:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 Processor -- start 2020-11-20T12:10:34.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 -- start start 2020-11-20T12:10:34.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a00ad2e0 0x7fb6a0067c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a0146610 0x7fb6a00ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6a00ab700 0x7fb6a01492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00ac670 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a0145ce0 con 0x7fb6a00ab700 2020-11-20T12:10:34.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.700+0000 7fb6a763a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a0145ed0 con 0x7fb6a0146610 2020-11-20T12:10:34.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a00ad2e0 0x7fb6a0067c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a00ad2e0 0x7fb6a0067c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39620/0 (socket says 172.21.15.44:39620) 2020-11-20T12:10:34.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 -- 172.21.15.44:0/3061150735 learned_addr learned my addr 172.21.15.44:0/3061150735 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a4bd5700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a0146610 0x7fb6a00ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a5bd7700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6a00ab700 0x7fb6a01492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 -- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a0146610 msgr2=0x7fb6a00ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a0146610 0x7fb6a00ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 -- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6a00ab700 msgr2=0x7fb6a01492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6a00ab700 0x7fb6a01492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 -- 172.21.15.44:0/3061150735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0146170 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a5bd7700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6a00ab700 0x7fb6a01492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:10:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a53d6700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a00ad2e0 0x7fb6a0067c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb698014000 tx=0x7fb698018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:34.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb69d7fa700 1 -- 172.21.15.44:0/3061150735 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69801f070 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb69d7fa700 1 -- 172.21.15.44:0/3061150735 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb698003030 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a0146a90 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb69d7fa700 1 -- 172.21.15.44:0/3061150735 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698007250 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.701+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a0066d60 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.702+0000 7fb69d7fa700 1 -- 172.21.15.44:0/3061150735 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb69802b070 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.702+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a0068150 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.703+0000 7fb69d7fa700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb678056d20 0x7fb6780591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:34.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.703+0000 7fb69d7fa700 1 -- 172.21.15.44:0/3061150735 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb698074c60 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.703+0000 7fb6a4bd5700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb678056d20 0x7fb6780591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:34.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.703+0000 7fb6a4bd5700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb678056d20 0x7fb6780591e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb694007b80 tx=0x7fb694009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:34.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.707+0000 7fb69d7fa700 1 -- 172.21.15.44:0/3061150735 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb698049e20 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.947+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb6a0061650 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.947+0000 7fb69d7fa700 1 -- 172.21.15.44:0/3061150735 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fb69804d440 con 0x7fb6a00ad2e0 2020-11-20T12:10:34.951 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:10:34.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.948+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb678056d20 msgr2=0x7fb6780591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.948+0000 7fb6a763a700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb678056d20 0x7fb6780591e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb694007b80 tx=0x7fb694009040).stop 2020-11-20T12:10:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a00ad2e0 msgr2=0x7fb6a0067c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a00ad2e0 0x7fb6a0067c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb698014000 tx=0x7fb698018000).stop 2020-11-20T12:10:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 shutdown_connections 2020-11-20T12:10:34.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb678056d20 0x7fb6780591e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6a00ab700 0x7fb6a01492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6a00ad2e0 0x7fb6a0067c20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 --2- 172.21.15.44:0/3061150735 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6a0146610 0x7fb6a00ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:34.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 >> 172.21.15.44:0/3061150735 conn(0x7fb6a0062690 msgr2=0x7fb6a0136ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 shutdown_connections 2020-11-20T12:10:34.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:34.949+0000 7fb6a763a700 1 -- 172.21.15.44:0/3061150735 wait complete. 2020-11-20T12:10:34.963 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:10:34.964 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:10:34.964 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:10:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.099+0000 7fa673839700 1 Processor -- start 2020-11-20T12:10:35.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa673839700 1 -- start start 2020-11-20T12:10:35.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa673839700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa66c0aa020 0x7fa66c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa673839700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa400 0x7fa66c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa673839700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66c0a9770 con 0x7fa66c13ac60 2020-11-20T12:10:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa673839700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c0a6da0 con 0x7fa66c0aa020 2020-11-20T12:10:35.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa673839700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0a7040 con 0x7fa66c0aa400 2020-11-20T12:10:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa6715d5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa66c0aa020 0x7fa66c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.100+0000 7fa670dd4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa400 0x7fa66c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa670dd4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa400 0x7fa66c0a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:51070/0 (socket says 172.21.15.44:51070) 2020-11-20T12:10:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa671dd6700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa66c13ac60 0x7fa66c0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:46252/0 (socket says 172.21.15.44:46252) 2020-11-20T12:10:35.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa671dd6700 1 -- 172.21.15.44:0/1407544377 learned_addr learned my addr 172.21.15.44:0/1407544377 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa6715d5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa66c0aa020 0x7fa66c0a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59910/0 (socket says 172.21.15.44:59910) 2020-11-20T12:10:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa6715d5700 1 -- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa400 msgr2=0x7fa66c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa6715d5700 1 --2- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa400 0x7fa66c0a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa6715d5700 1 -- 172.21.15.44:0/1407544377 >> v1:172.21.15.79:6789/0 conn(0x7fa66c13ac60 legacy=0x7fa66c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa6715d5700 1 -- 172.21.15.44:0/1407544377 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c0a6a90 con 0x7fa66c0aa020 2020-11-20T12:10:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa670dd4700 1 --2- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa400 0x7fa66c0a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa6715d5700 1 --2- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c0aa020 0x7fa66c0a9280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa66000b9a0 tx=0x7fa660006df0).ready entity=mon.1 client_cookie=f3ad836e3a6d5588 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa65ffff700 1 -- 172.21.15.44:0/1407544377 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600097b0 con 0x7fa66c0aa020 2020-11-20T12:10:35.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa65ffff700 1 -- 172.21.15.44:0/1407544377 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa660009910 con 0x7fa66c0aa020 2020-11-20T12:10:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.101+0000 7fa65ffff700 1 -- 172.21.15.44:0/1407544377 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660009c90 con 0x7fa66c0aa020 2020-11-20T12:10:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 -- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c0aa020 msgr2=0x7fa66c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 --2- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c0aa020 0x7fa66c0a9280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa66000b9a0 tx=0x7fa660006df0).stop 2020-11-20T12:10:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 -- 172.21.15.44:0/1407544377 shutdown_connections 2020-11-20T12:10:35.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 --2- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c0aa020 0x7fa66c0a9280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 --2- 172.21.15.44:0/1407544377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa400 0x7fa66c0a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 -- 172.21.15.44:0/1407544377 >> 172.21.15.44:0/1407544377 conn(0x7fa66c062690 msgr2=0x7fa66c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:35.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 -- 172.21.15.44:0/1407544377 shutdown_connections 2020-11-20T12:10:35.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.102+0000 7fa673839700 1 -- 172.21.15.44:0/1407544377 wait complete. 2020-11-20T12:10:35.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 Processor -- start 2020-11-20T12:10:35.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 -- start start 2020-11-20T12:10:35.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa020 0x7fa66c14b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0aa400 0x7fa66c0ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c13ac60 0x7fa66c12f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0abd60 con 0x7fa66c0aa400 2020-11-20T12:10:35.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c12c1c0 con 0x7fa66c13ac60 2020-11-20T12:10:35.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.103+0000 7fa673839700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0abf90 con 0x7fa66c0aa020 2020-11-20T12:10:35.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0aa400 0x7fa66c0ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0aa400 0x7fa66c0ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39636/0 (socket says 172.21.15.44:39636) 2020-11-20T12:10:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 -- 172.21.15.44:0/4098296045 learned_addr learned my addr 172.21.15.44:0/4098296045 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa671dd6700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c13ac60 0x7fa66c12f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa6715d5700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa020 0x7fa66c14b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa020 msgr2=0x7fa66c14b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa66c0aa020 0x7fa66c14b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c13ac60 msgr2=0x7fa66c12f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c13ac60 0x7fa66c12f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 -- 172.21.15.44:0/4098296045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c0ac230 con 0x7fa66c0aa400 2020-11-20T12:10:35.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa671dd6700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa66c13ac60 0x7fa66c12f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:35.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa670dd4700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0aa400 0x7fa66c0ad3d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa66800ef50 tx=0x7fa668006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800ca80 con 0x7fa66c0aa400 2020-11-20T12:10:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800cbe0 con 0x7fa66c0aa400 2020-11-20T12:10:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c0ac4d0 con 0x7fa66c0aa400 2020-11-20T12:10:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680123f0 con 0x7fa66c0aa400 2020-11-20T12:10:35.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.104+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c0ab210 con 0x7fa66c0aa400 2020-11-20T12:10:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.105+0000 7fa673839700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0ac660 0x7fa66c134880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.105+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c14b8f0 con 0x7fa66c0ac660 2020-11-20T12:10:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.105+0000 7fa6715d5700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0ac660 0x7fa66c134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.105+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa668012760 con 0x7fa66c0aa400 2020-11-20T12:10:35.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa6715d5700 1 -- 172.21.15.44:0/4098296045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa66c061650 con 0x7fa66c0ac660 2020-11-20T12:10:35.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa6715d5700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0ac660 0x7fa66c134880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa66000b9a0 tx=0x7fa660015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa644056d20 0x7fa6440591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa6680706e0 con 0x7fa66c0aa400 2020-11-20T12:10:35.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660009d00 con 0x7fa66c0ac660 2020-11-20T12:10:35.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa671dd6700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa644056d20 0x7fa6440591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fa660011c80 con 0x7fa66c0ac660 2020-11-20T12:10:35.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0ac660 msgr2=0x7fa66c134880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0ac660 0x7fa66c134880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fa66000b9a0 tx=0x7fa660015000).stop 2020-11-20T12:10:35.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0ac660 msgr2=0x7fa66c134880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:35.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa65d7fa700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0ac660 0x7fa66c134880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.106+0000 7fa671dd6700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa644056d20 0x7fa6440591e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fa664007fd0 tx=0x7fa664009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.114+0000 7fa673839700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c134d70 0x7fa66c0671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.114+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0446d0 con 0x7fa66c134d70 2020-11-20T12:10:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.114+0000 7fa6715d5700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c134d70 0x7fa66c0671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.114+0000 7fa6715d5700 1 -- 172.21.15.44:0/4098296045 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa66c069560 con 0x7fa66c134d70 2020-11-20T12:10:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.114+0000 7fa6715d5700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c134d70 0x7fa66c0671f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa66000c010 tx=0x7fa660021070).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600298a0 con 0x7fa66c134d70 2020-11-20T12:10:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4268 (secure 0 0 0) 0x7fa660011a70 con 0x7fa66c134d70 2020-11-20T12:10:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c134d70 msgr2=0x7fa66c0671f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa65d7fa700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c134d70 0x7fa66c0671f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa66000c010 tx=0x7fa660021070).stop 2020-11-20T12:10:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa65d7fa700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c134d70 msgr2=0x7fa66c0671f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:35.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa65d7fa700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c134d70 0x7fa66c0671f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa644056d20 msgr2=0x7fa6440591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa644056d20 0x7fa6440591e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fa664007fd0 tx=0x7fa664009040).stop 2020-11-20T12:10:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0aa400 msgr2=0x7fa66c0ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 --2- 172.21.15.44:0/4098296045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa66c0aa400 0x7fa66c0ad3d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa66800ef50 tx=0x7fa668006df0).stop 2020-11-20T12:10:35.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa671dd6700 1 -- 172.21.15.44:0/4098296045 reap_dead start 2020-11-20T12:10:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa671dd6700 1 -- 172.21.15.44:0/4098296045 reap_dead start 2020-11-20T12:10:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 shutdown_connections 2020-11-20T12:10:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 >> 172.21.15.44:0/4098296045 conn(0x7fa66c062690 msgr2=0x7fa66c0a8030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 shutdown_connections 2020-11-20T12:10:35.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.115+0000 7fa673839700 1 -- 172.21.15.44:0/4098296045 wait complete. 2020-11-20T12:10:35.133 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:10:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 82, 2020-11-20T12:10:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:10:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 11, 2020-11-20T12:10:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:35.169 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:35.171 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:10:35.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.313+0000 7f94bd496700 1 Processor -- start 2020-11-20T12:10:35.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.313+0000 7f94bd496700 1 -- start start 2020-11-20T12:10:35.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94bd496700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94b80aa400 0x7f94b80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94bd496700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f94b813ac60 0x7f94b80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94bd496700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b80a9770 con 0x7f94b80aa020 2020-11-20T12:10:35.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94bd496700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b80a56c0 con 0x7f94b80aa400 2020-11-20T12:10:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94bd496700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b80a6da0 con 0x7f94b813ac60 2020-11-20T12:10:35.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94b80aa400 0x7f94b80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94b80aa400 0x7f94b80a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39646/0 (socket says 172.21.15.44:39646) 2020-11-20T12:10:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 -- 172.21.15.44:0/1617702579 learned_addr learned my addr 172.21.15.44:0/1617702579 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b77fe700 1 --2- 172.21.15.44:0/1617702579 >> v2:172.21.15.79:3300/0 conn(0x7f94b813ac60 0x7f94b80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 -- 172.21.15.44:0/1617702579 >> v2:172.21.15.79:3300/0 conn(0x7f94b813ac60 msgr2=0x7f94b80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1617702579 >> v2:172.21.15.79:3300/0 conn(0x7f94b813ac60 0x7f94b80a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 -- 172.21.15.44:0/1617702579 >> v1:172.21.15.79:6789/0 conn(0x7f94b80aa020 legacy=0x7f94b80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:35.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 -- 172.21.15.44:0/1617702579 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b80a7040 con 0x7f94b80aa400 2020-11-20T12:10:35.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b77fe700 1 --2- 172.21.15.44:0/1617702579 >> v2:172.21.15.79:3300/0 conn(0x7f94b813ac60 0x7f94b80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:10:35.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1617702579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa400 0x7f94b80a9280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f94b80a8f70 tx=0x7f94a8006eb0).ready entity=mon.0 client_cookie=474d9a4ad26408c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b5ffb700 1 -- 172.21.15.44:0/1617702579 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8009890 con 0x7f94b80aa400 2020-11-20T12:10:35.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b5ffb700 1 -- 172.21.15.44:0/1617702579 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a80099f0 con 0x7f94b80aa400 2020-11-20T12:10:35.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.314+0000 7f94b5ffb700 1 -- 172.21.15.44:0/1617702579 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8009d70 con 0x7f94b80aa400 2020-11-20T12:10:35.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 -- 172.21.15.44:0/1617702579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa400 msgr2=0x7f94b80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 --2- 172.21.15.44:0/1617702579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa400 0x7f94b80a9280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f94b80a8f70 tx=0x7f94a8006eb0).stop 2020-11-20T12:10:35.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 -- 172.21.15.44:0/1617702579 shutdown_connections 2020-11-20T12:10:35.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 --2- 172.21.15.44:0/1617702579 >> v2:172.21.15.79:3300/0 conn(0x7f94b813ac60 0x7f94b80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 --2- 172.21.15.44:0/1617702579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa400 0x7f94b80a9280 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 -- 172.21.15.44:0/1617702579 >> 172.21.15.44:0/1617702579 conn(0x7f94b8062690 msgr2=0x7f94b80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:35.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 -- 172.21.15.44:0/1617702579 shutdown_connections 2020-11-20T12:10:35.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.315+0000 7f94bd496700 1 -- 172.21.15.44:0/1617702579 wait complete. 2020-11-20T12:10:35.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.316+0000 7f94bd496700 1 Processor -- start 2020-11-20T12:10:35.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.316+0000 7f94bd496700 1 -- start start 2020-11-20T12:10:35.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.316+0000 7f94bd496700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa020 0x7f94b80ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.316+0000 7f94bd496700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94b80aa400 0x7f94b8131120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94bd496700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b813ac60 0x7f94b8131610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94bd496700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8131b00 con 0x7f94b80aa020 2020-11-20T12:10:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94bd496700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b813e840 con 0x7f94b813ac60 2020-11-20T12:10:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94bd496700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b813eae0 con 0x7f94b80aa400 2020-11-20T12:10:35.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa020 0x7f94b80ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa020 0x7f94b80ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39652/0 (socket says 172.21.15.44:39652) 2020-11-20T12:10:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 -- 172.21.15.44:0/1905675401 learned_addr learned my addr 172.21.15.44:0/1905675401 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b77fe700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b813ac60 0x7f94b8131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94b80aa400 0x7f94b8131120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94b80aa400 msgr2=0x7f94b8131120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94b80aa400 0x7f94b8131120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b813ac60 msgr2=0x7f94b8131610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b813ac60 0x7f94b8131610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 -- 172.21.15.44:0/1905675401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b813ed80 con 0x7f94b80aa020 2020-11-20T12:10:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94b80aa400 0x7f94b8131120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:10:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94b6ffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa020 0x7f94b80ad0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f94a00096e0 tx=0x7f94a0006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000d5c0 con 0x7f94b80aa020 2020-11-20T12:10:35.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a0000e90 con 0x7f94b80aa020 2020-11-20T12:10:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000d990 con 0x7f94b80aa020 2020-11-20T12:10:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b8067ff0 con 0x7f94b80aa020 2020-11-20T12:10:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.317+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b813e5e0 con 0x7f94b80aa020 2020-11-20T12:10:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.318+0000 7f94bd496700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8067a90 0x7f94b8060170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.318+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8061650 con 0x7f94b8067a90 2020-11-20T12:10:35.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.318+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8067a90 0x7f94b8060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.318+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f94a000dd40 con 0x7f94b80aa020 2020-11-20T12:10:35.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94b67fc700 1 -- 172.21.15.44:0/1905675401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94b80446d0 con 0x7f94b8067a90 2020-11-20T12:10:35.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8067a90 0x7f94b8060170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f94b80a8f70 tx=0x7f94a8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9494056d20 0x7f94940591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f94a006f9c0 con 0x7f94b80aa020 2020-11-20T12:10:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94b77fe700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9494056d20 0x7f94940591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8011de0 con 0x7f94b8067a90 2020-11-20T12:10:35.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f94a801e430 con 0x7f94b8067a90 2020-11-20T12:10:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8067a90 msgr2=0x7f94b8060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8067a90 0x7f94b8060170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f94b80a8f70 tx=0x7f94a8015000).stop 2020-11-20T12:10:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8067a90 msgr2=0x7f94b8060170 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94aeffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8067a90 0x7f94b8060170 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.319+0000 7f94b77fe700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9494056d20 0x7f94940591e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f94b81457a0 tx=0x7f94b0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.327+0000 7f94bd496700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8060660 0x7f94b80652b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.327+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8069560 con 0x7f94b8060660 2020-11-20T12:10:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.327+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8060660 0x7f94b80652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.327+0000 7f94b67fc700 1 -- 172.21.15.44:0/1905675401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f94b8061350 con 0x7f94b8060660 2020-11-20T12:10:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94b67fc700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8060660 0x7f94b80652b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f94b8061790 tx=0x7f94a801f000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a802db30 con 0x7f94b8060660 2020-11-20T12:10:35.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4436 (secure 0 0 0) 0x7f94a8011bd0 con 0x7f94b8060660 2020-11-20T12:10:35.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8060660 msgr2=0x7f94b80652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94aeffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8060660 0x7f94b80652b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f94b8061790 tx=0x7f94a801f000).stop 2020-11-20T12:10:35.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94aeffd700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8060660 msgr2=0x7f94b80652b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:35.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94aeffd700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94b8060660 0x7f94b80652b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.361 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9494056d20 msgr2=0x7f94940591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9494056d20 0x7f94940591e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f94b81457a0 tx=0x7f94b0008040).stop 2020-11-20T12:10:35.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa020 msgr2=0x7f94b80ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 --2- 172.21.15.44:0/1905675401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94b80aa020 0x7f94b80ad0c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f94a00096e0 tx=0x7f94a0006d70).stop 2020-11-20T12:10:35.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94b77fe700 1 -- 172.21.15.44:0/1905675401 reap_dead start 2020-11-20T12:10:35.362 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94b77fe700 1 -- 172.21.15.44:0/1905675401 reap_dead start 2020-11-20T12:10:35.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 shutdown_connections 2020-11-20T12:10:35.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 >> 172.21.15.44:0/1905675401 conn(0x7f94b8062690 msgr2=0x7f94b8136c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 shutdown_connections 2020-11-20T12:10:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.328+0000 7f94bd496700 1 -- 172.21.15.44:0/1905675401 wait complete. 2020-11-20T12:10:35.364 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:35.364 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:35.364 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:35.365 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:10:35.365 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 82, 2020-11-20T12:10:35.365 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:35.365 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:10:35.366 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:35.366 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:35.366 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.366 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:10:35.366 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:35.367 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:35.367 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:35.367 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.367 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.367 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.368 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.368 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.368 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.368 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.368 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.368 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.369 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:35.370 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:35.371 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.372 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.373 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.373 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.374 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.374 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.375 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.376 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.377 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.377 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:35.377 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:35.378 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:35.378 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:35.378 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:35.378 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:35.378 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:35.379 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:35.379 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:35.379 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:35.379 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:35.380 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:35.380 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:35.380 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:35.380 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:35.380 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.381 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.382 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.382 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.383 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.383 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.383 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.383 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.384 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.384 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:35.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.385 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:35.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:35.385 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:35.386 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.386 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.387 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:35.387 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.388 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:35.388 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.389 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.389 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:35.390 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:35.390 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.390 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.390 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:35.391 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:35.392 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.392 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.392 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:35.393 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.394 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:35.394 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.394 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.395 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.395 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:35.395 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:35.396 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.396 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.396 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.397 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:35.397 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:35.397 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.397 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.398 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.398 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:35.398 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.399 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.399 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:35.399 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.400 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.400 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.400 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:35.401 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:35.401 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.401 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.401 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.402 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.402 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.402 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:35.403 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:35.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.403 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.403 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.403 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:35.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.404 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.404 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:10:35.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.405 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.405 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.405 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:35.405 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.406 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.406 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:35.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.406 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.406 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.407 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:35.407 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.407 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.408 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:35.408 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:35.408 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:10:35.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07fdc0d700 1 Processor -- start 2020-11-20T12:10:35.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07fdc0d700 1 -- start start 2020-11-20T12:10:35.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07fdc0d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07fdc0d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f07f813ac60 0x7f07f80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07fdc0d700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f80a9770 con 0x7f07f80aa400 2020-11-20T12:10:35.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07fdc0d700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f80a56c0 con 0x7f07f813ac60 2020-11-20T12:10:35.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07fdc0d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f80a6da0 con 0x7f07f80aa020 2020-11-20T12:10:35.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07f7fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f07f813ac60 0x7f07f80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.537+0000 7f07f6ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f07f80aa400 0x7f07f80a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43412/0 (socket says 172.21.15.44:43412) 2020-11-20T12:10:35.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f6ffd700 1 -- 172.21.15.44:0/2430480940 learned_addr learned my addr 172.21.15.44:0/2430480940 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:35.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f77fe700 1 --2- 172.21.15.44:0/2430480940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f7fff700 1 -- 172.21.15.44:0/2430480940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 msgr2=0x7f07f80a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:35.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f7fff700 1 --2- 172.21.15.44:0/2430480940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f80a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f7fff700 1 -- 172.21.15.44:0/2430480940 >> v1:172.21.15.44:6789/0 conn(0x7f07f80aa400 legacy=0x7f07f80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f7fff700 1 -- 172.21.15.44:0/2430480940 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80a7040 con 0x7f07f813ac60 2020-11-20T12:10:35.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f67fc700 1 -- 172.21.15.44:0/2430480940 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815895072 0 0) 0x7f07e00010d0 con 0x7f07f80aa400 2020-11-20T12:10:35.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f77fe700 1 --2- 172.21.15.44:0/2430480940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f80a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:10:35.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f7fff700 1 --2- 172.21.15.44:0/2430480940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f813ac60 0x7f07f80a3700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f07e800b9a0 tx=0x7f07e8006df0).ready entity=mon.0 client_cookie=1c3c78b89b41ef18 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f67fc700 1 -- 172.21.15.44:0/2430480940 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80097b0 con 0x7f07f813ac60 2020-11-20T12:10:35.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f67fc700 1 -- 172.21.15.44:0/2430480940 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e8009910 con 0x7f07f813ac60 2020-11-20T12:10:35.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.538+0000 7f07f67fc700 1 -- 172.21.15.44:0/2430480940 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8009c90 con 0x7f07f813ac60 2020-11-20T12:10:35.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2430480940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f813ac60 msgr2=0x7f07f80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 --2- 172.21.15.44:0/2430480940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f813ac60 0x7f07f80a3700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f07e800b9a0 tx=0x7f07e8006df0).stop 2020-11-20T12:10:35.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2430480940 shutdown_connections 2020-11-20T12:10:35.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 --2- 172.21.15.44:0/2430480940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f813ac60 0x7f07f80a3700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 --2- 172.21.15.44:0/2430480940 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f80a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2430480940 >> 172.21.15.44:0/2430480940 conn(0x7f07f8062690 msgr2=0x7f07f80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:35.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2430480940 shutdown_connections 2020-11-20T12:10:35.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.539+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2430480940 wait complete. 2020-11-20T12:10:35.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 Processor -- start 2020-11-20T12:10:35.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 -- start start 2020-11-20T12:10:35.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f8139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f80aa400 0x7f07f80656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f813ac60 0x7f07f814d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f813a390 con 0x7f07f80aa400 2020-11-20T12:10:35.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8065280 con 0x7f07f813ac60 2020-11-20T12:10:35.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.540+0000 7f07fdc0d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8066b70 con 0x7f07f80aa020 2020-11-20T12:10:35.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f80aa400 0x7f07f80656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f80aa400 0x7f07f80656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39670/0 (socket says 172.21.15.44:39670) 2020-11-20T12:10:35.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 -- 172.21.15.44:0/2981631807 learned_addr learned my addr 172.21.15.44:0/2981631807 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:35.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f77fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f8139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f7fff700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f813ac60 0x7f07f814d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 msgr2=0x7f07f8139ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80aa020 0x7f07f8139ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f813ac60 msgr2=0x7f07f814d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f813ac60 0x7f07f814d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 -- 172.21.15.44:0/2981631807 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80a2b90 con 0x7f07f80aa400 2020-11-20T12:10:35.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07f6ffd700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f80aa400 0x7f07f80656e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f07e000c730 tx=0x7f07e00078a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00072e0 con 0x7f07f80aa400 2020-11-20T12:10:35.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e0007440 con 0x7f07f80aa400 2020-11-20T12:10:35.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0005800 con 0x7f07f80aa400 2020-11-20T12:10:35.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f814d2d0 con 0x7f07f80aa400 2020-11-20T12:10:35.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.541+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f8065bd0 con 0x7f07f80aa400 2020-11-20T12:10:35.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.542+0000 7f07fdc0d700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80ab330 0x7f07f814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.542+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8065440 con 0x7f07f80ab330 2020-11-20T12:10:35.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.542+0000 7f07f77fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80ab330 0x7f07f814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.542+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f07e0010070 con 0x7f07f80aa400 2020-11-20T12:10:35.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.542+0000 7f07f77fe700 1 -- 172.21.15.44:0/2981631807 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07f8065dc0 con 0x7f07f80ab330 2020-11-20T12:10:35.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07f77fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80ab330 0x7f07f814de30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f07f0010c50 tx=0x7f07f0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f07d4056d20 0x7f07d40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f07e006f550 con 0x7f07f80aa400 2020-11-20T12:10:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07f7fff700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f07d4056d20 0x7f07d40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000ecc0 con 0x7f07f80ab330 2020-11-20T12:10:35.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f07f000ee20 con 0x7f07f80ab330 2020-11-20T12:10:35.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80ab330 msgr2=0x7f07f814de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80ab330 0x7f07f814de30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f07f0010c50 tx=0x7f07f0008040).stop 2020-11-20T12:10:35.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80ab330 msgr2=0x7f07f814de30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:35.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07ef7fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80ab330 0x7f07f814de30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.543+0000 7f07f7fff700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f07d4056d20 0x7f07d40591e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f07e800b150 tx=0x7f07e8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.551+0000 7f07fdc0d700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80665f0 0x7f07f80a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.551+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8066a00 con 0x7f07f80665f0 2020-11-20T12:10:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.551+0000 7f07f77fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80665f0 0x7f07f80a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07f77fe700 1 -- 172.21.15.44:0/2981631807 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f07f80a9770 con 0x7f07f80665f0 2020-11-20T12:10:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07f77fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80665f0 0x7f07f80a1880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f07f0017000 tx=0x7f07f0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0013920 con 0x7f07f80665f0 2020-11-20T12:10:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f07f000eab0 con 0x7f07f80665f0 2020-11-20T12:10:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80665f0 msgr2=0x7f07f80a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07ef7fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80665f0 0x7f07f80a1880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f07f0017000 tx=0x7f07f0008040).stop 2020-11-20T12:10:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07ef7fe700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80665f0 msgr2=0x7f07f80a1880 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07ef7fe700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f07f80665f0 0x7f07f80a1880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f07d4056d20 msgr2=0x7f07d40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07fdc0d700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f07d4056d20 0x7f07d40591e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f07e800b150 tx=0x7f07e8015000).stop 2020-11-20T12:10:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f80aa400 msgr2=0x7f07f80656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07fdc0d700 1 --2- 172.21.15.44:0/2981631807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07f80aa400 0x7f07f80656e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f07e000c730 tx=0x7f07e00078a0).stop 2020-11-20T12:10:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.552+0000 7f07f7fff700 1 -- 172.21.15.44:0/2981631807 reap_dead start 2020-11-20T12:10:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.553+0000 7f07f7fff700 1 -- 172.21.15.44:0/2981631807 reap_dead start 2020-11-20T12:10:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.553+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 shutdown_connections 2020-11-20T12:10:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.553+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 >> 172.21.15.44:0/2981631807 conn(0x7f07f8062690 msgr2=0x7f07f80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.553+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 shutdown_connections 2020-11-20T12:10:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.553+0000 7f07fdc0d700 1 -- 172.21.15.44:0/2981631807 wait complete. 2020-11-20T12:10:35.595 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:10:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 82, 2020-11-20T12:10:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:10:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:10:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:10:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:10:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:10:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:10:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:10:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:35.641 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:35.643 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:10:35.643 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:10:35.643 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:10:35.644 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:10:35.644 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:10:35.644 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:10:35.654 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:10:35.654 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:10:35.655 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:10:35.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c842b5700 1 Processor -- start 2020-11-20T12:10:35.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c842b5700 1 -- start start 2020-11-20T12:10:35.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c842b5700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c7c0a3e10 0x7f3c7c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c842b5700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c0a9770 con 0x7f3c7c0aa020 2020-11-20T12:10:35.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c842b5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c0a6da0 con 0x7f3c7c0aa400 2020-11-20T12:10:35.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c842b5700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7c0a56c0 con 0x7f3c7c0a3e10 2020-11-20T12:10:35.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c82852700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3c7c0a3e10 msgr2=0x7f3c7c0a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:35.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.820+0000 7f3c82051700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3c7c0aa020 legacy=0x7f3c7c0a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:10:35.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c82852700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c7c0a3e10 0x7f3c7c0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:35.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c81850700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3c7c0aa400 0x7f3c7c0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:46306/0 (socket says 172.21.15.44:46306) 2020-11-20T12:10:35.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c81850700 1 -- 172.21.15.44:0/2268662192 learned_addr learned my addr 172.21.15.44:0/2268662192 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:35.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900518857 0 0) 0x7f3c7c0a6da0 con 0x7f3c7c0aa400 2020-11-20T12:10:35.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c68003240 con 0x7f3c7c0aa400 2020-11-20T12:10:35.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1217596084 0 0) 0x7f3c68003240 con 0x7f3c7c0aa400 2020-11-20T12:10:35.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 >> v2:172.21.15.44:3300/0 conn(0x7f3c7c0a3e10 msgr2=0x7f3c7c0a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:35.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 --2- 172.21.15.44:0/2268662192 >> v2:172.21.15.44:3300/0 conn(0x7f3c7c0a3e10 0x7f3c7c0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:35.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 >> v1:172.21.15.44:6789/0 conn(0x7f3c7c0aa020 legacy=0x7f3c7c0a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:35.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0a7040 con 0x7f3c7c0aa400 2020-11-20T12:10:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f3c700014e0 con 0x7f3c7c0aa400 2020-11-20T12:10:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 >> v1:172.21.15.79:6789/0 conn(0x7f3c7c0aa400 legacy=0x7f3c7c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c68003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c68004090 0x7f3c68004560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c68004b60 0x7f3c6800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0a56c0 con 0x7f3c68004b60 2020-11-20T12:10:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68005200 con 0x7f3c68003730 2020-11-20T12:10:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6800bc20 con 0x7f3c68004090 2020-11-20T12:10:35.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c82852700 1 -- 172.21.15.44:0/2268662192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c68004b60 msgr2=0x7f3c6800b730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:35.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.821+0000 7f3c82852700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c68004b60 0x7f3c6800b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:35.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82051700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c68003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c81850700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c68004090 0x7f3c68004560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82051700 1 -- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c68004090 msgr2=0x7f3c68004560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:35.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82051700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c68004090 0x7f3c68004560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82051700 1 -- 172.21.15.44:0/2268662192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c68004b60 msgr2=0x7f3c6800b730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82051700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c68004b60 0x7f3c6800b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82051700 1 -- 172.21.15.44:0/2268662192 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68003240 con 0x7f3c68003730 2020-11-20T12:10:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82852700 1 -- 172.21.15.44:0/2268662192 reap_dead start 2020-11-20T12:10:35.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c82051700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c68003b30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f3c74009cd0 tx=0x7f3c74009270).ready entity=mon.1 client_cookie=1112d3589197b43f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74003c60 con 0x7f3c68003730 2020-11-20T12:10:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74003dc0 con 0x7f3c68003730 2020-11-20T12:10:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.822+0000 7f3c8104f700 1 -- 172.21.15.44:0/2268662192 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7400bb10 con 0x7f3c68003730 2020-11-20T12:10:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.823+0000 7f3c842b5700 1 -- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 msgr2=0x7f3c68003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.823+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c68003b30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f3c74009cd0 tx=0x7f3c74009270).stop 2020-11-20T12:10:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.823+0000 7f3c842b5700 1 -- 172.21.15.44:0/2268662192 shutdown_connections 2020-11-20T12:10:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.823+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2268662192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c68003b30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.823+0000 7f3c842b5700 1 -- 172.21.15.44:0/2268662192 >> 172.21.15.44:0/2268662192 conn(0x7f3c7c062690 msgr2=0x7f3c7c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.823+0000 7f3c842b5700 1 -- 172.21.15.44:0/2268662192 shutdown_connections 2020-11-20T12:10:35.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.823+0000 7f3c842b5700 1 -- 172.21.15.44:0/2268662192 wait complete. 2020-11-20T12:10:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 Processor -- start 2020-11-20T12:10:35.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 -- start start 2020-11-20T12:10:35.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c7c0acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7c0665b0 0x7f3c7c067b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0650c0 0x7f3c7c065540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c065a30 con 0x7f3c7c0650c0 2020-11-20T12:10:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c066a50 con 0x7f3c68003730 2020-11-20T12:10:35.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c842b5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c066cf0 con 0x7f3c7c0665b0 2020-11-20T12:10:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c82852700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0650c0 msgr2=0x7f3c7c065540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c82852700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0650c0 0x7f3c7c065540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.824+0000 7f3c82051700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c7c0acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c81850700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7c0665b0 0x7f3c7c067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c7c0acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59976/0 (socket says 172.21.15.44:59976) 2020-11-20T12:10:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 -- 172.21.15.44:0/2204691957 learned_addr learned my addr 172.21.15.44:0/2204691957 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 -- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7c0665b0 msgr2=0x7f3c7c067b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7c0665b0 0x7f3c7c067b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 -- 172.21.15.44:0/2204691957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0650c0 msgr2=0x7f3c7c065540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:35.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0650c0 0x7f3c7c065540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:35.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 -- 172.21.15.44:0/2204691957 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c066f90 con 0x7f3c68003730 2020-11-20T12:10:35.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c81850700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7c0665b0 0x7f3c7c067b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:35.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c82051700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c7c0acc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f3c7400b7d0 tx=0x7f3c74015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c7a7fc700 1 -- 172.21.15.44:0/2204691957 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74005810 con 0x7f3c68003730 2020-11-20T12:10:35.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c7a7fc700 1 -- 172.21.15.44:0/2204691957 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7401e070 con 0x7f3c68003730 2020-11-20T12:10:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c068080 con 0x7f3c68003730 2020-11-20T12:10:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c7a7fc700 1 -- 172.21.15.44:0/2204691957 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7401a460 con 0x7f3c68003730 2020-11-20T12:10:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.825+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c14a460 con 0x7f3c68003730 2020-11-20T12:10:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.826+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c067120 con 0x7f3c68003730 2020-11-20T12:10:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.827+0000 7f3c7a7fc700 1 -- 172.21.15.44:0/2204691957 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3c7402a070 con 0x7f3c68003730 2020-11-20T12:10:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.827+0000 7f3c7a7fc700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c54056cd0 0x7f3c54059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.827+0000 7f3c7a7fc700 1 -- 172.21.15.44:0/2204691957 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3c740731b0 con 0x7f3c68003730 2020-11-20T12:10:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.827+0000 7f3c81850700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c54056cd0 0x7f3c54059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.827+0000 7f3c81850700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c54056cd0 0x7f3c54059190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3c70001470 tx=0x7f3c70009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:35.831+0000 7f3c7a7fc700 1 -- 172.21.15.44:0/2204691957 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3c74048480 con 0x7f3c68003730 2020-11-20T12:10:36.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.074+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3c7c061650 con 0x7f3c68003730 2020-11-20T12:10:36.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.074+0000 7f3c7a7fc700 1 -- 172.21.15.44:0/2204691957 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f3c74006da0 con 0x7f3c68003730 2020-11-20T12:10:36.078 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:10:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c54056cd0 msgr2=0x7f3c54059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:36.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c54056cd0 0x7f3c54059190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3c70001470 tx=0x7f3c70009040).stop 2020-11-20T12:10:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 msgr2=0x7f3c7c0acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c7c0acc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f3c7400b7d0 tx=0x7f3c74015000).stop 2020-11-20T12:10:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 shutdown_connections 2020-11-20T12:10:36.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c54056cd0 0x7f3c54059190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c0650c0 0x7f3c7c065540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c68003730 0x7f3c7c0acc90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 --2- 172.21.15.44:0/2204691957 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7c0665b0 0x7f3c7c067b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 >> 172.21.15.44:0/2204691957 conn(0x7f3c7c062690 msgr2=0x7f3c7c135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:36.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 shutdown_connections 2020-11-20T12:10:36.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:36.076+0000 7f3c842b5700 1 -- 172.21.15.44:0/2204691957 wait complete. 2020-11-20T12:10:37.224 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:10:37.227 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:10:39.092 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:10:39.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.227+0000 7fcfadd41700 1 Processor -- start 2020-11-20T12:10:39.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfadd41700 1 -- start start 2020-11-20T12:10:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfadd41700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcfa80aa400 0x7fcfa80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfadd41700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcfa813ac60 0x7fcfa80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfadd41700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa80a9770 con 0x7fcfa80aa020 2020-11-20T12:10:39.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfadd41700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa80a56c0 con 0x7fcfa80aa400 2020-11-20T12:10:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfadd41700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa80a6da0 con 0x7fcfa813ac60 2020-11-20T12:10:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfa6ffd700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fcfa80aa400 msgr2=0x7fcfa80a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.228+0000 7fcfa6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcfa80aa400 0x7fcfa80a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:39.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcfa813ac60 0x7fcfa80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa77fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fcfa80aa020 0x7fcfa80a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:46326/0 (socket says 172.21.15.44:46326) 2020-11-20T12:10:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa77fe700 1 -- 172.21.15.44:0/3018700931 learned_addr learned my addr 172.21.15.44:0/3018700931 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa7fff700 1 -- 172.21.15.44:0/3018700931 >> v2:172.21.15.44:3300/0 conn(0x7fcfa80aa400 msgr2=0x7fcfa80a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa7fff700 1 --2- 172.21.15.44:0/3018700931 >> v2:172.21.15.44:3300/0 conn(0x7fcfa80aa400 0x7fcfa80a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:39.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa7fff700 1 -- 172.21.15.44:0/3018700931 >> v1:172.21.15.79:6789/0 conn(0x7fcfa80aa020 legacy=0x7fcfa80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa67fc700 1 -- 172.21.15.44:0/3018700931 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025234421 0 0) 0x7fcf980010d0 con 0x7fcfa80aa020 2020-11-20T12:10:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa7fff700 1 -- 172.21.15.44:0/3018700931 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa80a7040 con 0x7fcfa813ac60 2020-11-20T12:10:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa7fff700 1 --2- 172.21.15.44:0/3018700931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa813ac60 0x7fcfa80a3700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fcfa80a8f70 tx=0x7fcfa00091d0).ready entity=mon.1 client_cookie=67841651839bda7c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:39.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa67fc700 1 -- 172.21.15.44:0/3018700931 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000bbc0 con 0x7fcfa813ac60 2020-11-20T12:10:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa67fc700 1 -- 172.21.15.44:0/3018700931 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa000bd20 con 0x7fcfa813ac60 2020-11-20T12:10:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.229+0000 7fcfa67fc700 1 -- 172.21.15.44:0/3018700931 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011610 con 0x7fcfa813ac60 2020-11-20T12:10:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 -- 172.21.15.44:0/3018700931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa813ac60 msgr2=0x7fcfa80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:39.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 --2- 172.21.15.44:0/3018700931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa813ac60 0x7fcfa80a3700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fcfa80a8f70 tx=0x7fcfa00091d0).stop 2020-11-20T12:10:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 -- 172.21.15.44:0/3018700931 shutdown_connections 2020-11-20T12:10:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 --2- 172.21.15.44:0/3018700931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa813ac60 0x7fcfa80a3700 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 --2- 172.21.15.44:0/3018700931 >> v2:172.21.15.44:3300/0 conn(0x7fcfa80aa400 0x7fcfa80a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 -- 172.21.15.44:0/3018700931 >> 172.21.15.44:0/3018700931 conn(0x7fcfa8062690 msgr2=0x7fcfa80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:39.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 -- 172.21.15.44:0/3018700931 shutdown_connections 2020-11-20T12:10:39.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.230+0000 7fcfadd41700 1 -- 172.21.15.44:0/3018700931 wait complete. 2020-11-20T12:10:39.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.231+0000 7fcfadd41700 1 Processor -- start 2020-11-20T12:10:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.231+0000 7fcfadd41700 1 -- start start 2020-11-20T12:10:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfadd41700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa80aa020 0x7fcfa8065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfadd41700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa80aa400 0x7fcfa8130480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfadd41700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfa813ac60 0x7fcfa80abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:39.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfadd41700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8067530 con 0x7fcfa80aa020 2020-11-20T12:10:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfadd41700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80660a0 con 0x7fcfa80aa400 2020-11-20T12:10:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfadd41700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8065380 con 0x7fcfa813ac60 2020-11-20T12:10:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa77fe700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa80aa020 msgr2=0x7fcfa8065ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:39.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa80aa020 0x7fcfa8065ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:39.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa80aa400 0x7fcfa8130480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:39.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa80aa400 0x7fcfa8130480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:59992/0 (socket says 172.21.15.44:59992) 2020-11-20T12:10:39.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 -- 172.21.15.44:0/1146453881 learned_addr learned my addr 172.21.15.44:0/1146453881 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:39.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa7fff700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfa813ac60 0x7fcfa80abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:39.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 -- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfa813ac60 msgr2=0x7fcfa80abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:39.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfa813ac60 0x7fcfa80abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:39.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 -- 172.21.15.44:0/1146453881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa80aa020 msgr2=0x7fcfa8065ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:39.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa80aa020 0x7fcfa8065ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:39.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 -- 172.21.15.44:0/1146453881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8065620 con 0x7fcfa80aa400 2020-11-20T12:10:39.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.232+0000 7fcfa6ffd700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa80aa400 0x7fcfa8130480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fcf9c009710 tx=0x7fcf9c006e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:39.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.233+0000 7fcf877fe700 1 -- 172.21.15.44:0/1146453881 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c006170 con 0x7fcfa80aa400 2020-11-20T12:10:39.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.233+0000 7fcf877fe700 1 -- 172.21.15.44:0/1146453881 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c0062d0 con 0x7fcfa80aa400 2020-11-20T12:10:39.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.233+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa8066310 con 0x7fcfa80aa400 2020-11-20T12:10:39.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.233+0000 7fcf877fe700 1 -- 172.21.15.44:0/1146453881 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0038e0 con 0x7fcfa80aa400 2020-11-20T12:10:39.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.233+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa8066c20 con 0x7fcfa80aa400 2020-11-20T12:10:39.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.234+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa80657b0 con 0x7fcfa80aa400 2020-11-20T12:10:39.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.234+0000 7fcf877fe700 1 -- 172.21.15.44:0/1146453881 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcf9c01b030 con 0x7fcfa80aa400 2020-11-20T12:10:39.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.234+0000 7fcf877fe700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf80056d00 0x7fcf800591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:39.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.234+0000 7fcf877fe700 1 -- 172.21.15.44:0/1146453881 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fcf9c06fdf0 con 0x7fcfa80aa400 2020-11-20T12:10:39.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.234+0000 7fcfa77fe700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf80056d00 0x7fcf800591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:39.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.235+0000 7fcfa77fe700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf80056d00 0x7fcf800591c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fcf98001230 tx=0x7fcf98007700).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:39.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.238+0000 7fcf877fe700 1 -- 172.21.15.44:0/1146453881 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcf9c0450c0 con 0x7fcfa80aa400 2020-11-20T12:10:39.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.477+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcfa8066500 con 0x7fcfa80aa400 2020-11-20T12:10:39.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.478+0000 7fcf877fe700 1 -- 172.21.15.44:0/1146453881 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fcf9c0486e0 con 0x7fcfa80aa400 2020-11-20T12:10:39.482 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:10:39.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf80056d00 msgr2=0x7fcf800591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf80056d00 0x7fcf800591c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fcf98001230 tx=0x7fcf98007700).stop 2020-11-20T12:10:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa80aa400 msgr2=0x7fcfa8130480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:39.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa80aa400 0x7fcfa8130480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fcf9c009710 tx=0x7fcf9c006e20).stop 2020-11-20T12:10:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 shutdown_connections 2020-11-20T12:10:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf80056d00 0x7fcf800591c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfa813ac60 0x7fcfa80abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfa80aa020 0x7fcfa8065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:39.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 --2- 172.21.15.44:0/1146453881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfa80aa400 0x7fcfa8130480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 >> 172.21.15.44:0/1146453881 conn(0x7fcfa8062690 msgr2=0x7fcfa8135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 shutdown_connections 2020-11-20T12:10:39.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:39.479+0000 7fcfadd41700 1 -- 172.21.15.44:0/1146453881 wait complete. 2020-11-20T12:10:42.495 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:10:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.632+0000 7f09b62d7700 1 Processor -- start 2020-11-20T12:10:42.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.632+0000 7f09b62d7700 1 -- start start 2020-11-20T12:10:42.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.633+0000 7f09b62d7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f09b00a9480 0x7f09b0142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:42.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.633+0000 7f09b62d7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00a7880 con 0x7f09b00aa1e0 2020-11-20T12:10:42.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.633+0000 7f09b62d7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00a5b00 con 0x7f09b00a4330 2020-11-20T12:10:42.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.633+0000 7f09b62d7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b00a5da0 con 0x7f09b00a9480 2020-11-20T12:10:42.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.633+0000 7f09affff700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f09b00aa1e0 legacy=0x7f09b00a37c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:10:42.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.633+0000 7f09b4874700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f09b00a9480 0x7f09b0142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:42.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09a7fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f09b00a4330 0x7f09b01427e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:46342/0 (socket says 172.21.15.44:46342) 2020-11-20T12:10:42.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09b4874700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f09b00a9480 0x7f09b0142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60000/0 (socket says 172.21.15.44:60000) 2020-11-20T12:10:42.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09a7fff700 1 -- 172.21.15.44:0/3198905850 learned_addr learned my addr 172.21.15.44:0/3198905850 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:42.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09b4874700 1 -- 172.21.15.44:0/3198905850 >> v1:172.21.15.79:6789/0 conn(0x7f09b00a4330 legacy=0x7f09b01427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:42.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09b4874700 1 -- 172.21.15.44:0/3198905850 >> v1:172.21.15.44:6789/0 conn(0x7f09b00aa1e0 legacy=0x7f09b00a37c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:42.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09b4874700 1 -- 172.21.15.44:0/3198905850 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00a6040 con 0x7f09b00a9480 2020-11-20T12:10:42.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09b4874700 1 --2- 172.21.15.44:0/3198905850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00a9480 0x7f09b0142ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f09a8009c90 tx=0x7f09a80091d0).ready entity=mon.1 client_cookie=4c23b16d461d6534 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:42.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09af7fe700 1 -- 172.21.15.44:0/3198905850 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800bc10 con 0x7f09b00a9480 2020-11-20T12:10:42.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09af7fe700 1 -- 172.21.15.44:0/3198905850 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800bd70 con 0x7f09b00a9480 2020-11-20T12:10:42.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.634+0000 7f09af7fe700 1 -- 172.21.15.44:0/3198905850 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8011610 con 0x7f09b00a9480 2020-11-20T12:10:42.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.635+0000 7f09b62d7700 1 -- 172.21.15.44:0/3198905850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00a9480 msgr2=0x7f09b0142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:42.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.635+0000 7f09b62d7700 1 --2- 172.21.15.44:0/3198905850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00a9480 0x7f09b0142ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f09a8009c90 tx=0x7f09a80091d0).stop 2020-11-20T12:10:42.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.635+0000 7f09b62d7700 1 -- 172.21.15.44:0/3198905850 shutdown_connections 2020-11-20T12:10:42.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.635+0000 7f09b62d7700 1 --2- 172.21.15.44:0/3198905850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00a9480 0x7f09b0142ef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:42.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.635+0000 7f09b62d7700 1 -- 172.21.15.44:0/3198905850 >> 172.21.15.44:0/3198905850 conn(0x7f09b0062690 msgr2=0x7f09b00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:42.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.635+0000 7f09b62d7700 1 -- 172.21.15.44:0/3198905850 shutdown_connections 2020-11-20T12:10:42.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.635+0000 7f09b62d7700 1 -- 172.21.15.44:0/3198905850 wait complete. 2020-11-20T12:10:42.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 Processor -- start 2020-11-20T12:10:42.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 -- start start 2020-11-20T12:10:42.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f09b00a4330 0x7f09b00a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:42.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09b00a9480 0x7f09b00ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:42.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00aa1e0 0x7f09b0152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:42.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00ad6d0 con 0x7f09b00a9480 2020-11-20T12:10:42.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00ad920 con 0x7f09b00aa1e0 2020-11-20T12:10:42.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09b62d7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b01519f0 con 0x7f09b00a4330 2020-11-20T12:10:42.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09a7fff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09b00a9480 msgr2=0x7f09b00ad1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:42.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.636+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09b00a9480 0x7f09b00ad1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:42.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09affff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f09b00a4330 0x7f09b00a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:42.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b4874700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00aa1e0 0x7f09b0152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09affff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f09b00a4330 0x7f09b00a8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:51160/0 (socket says 172.21.15.44:51160) 2020-11-20T12:10:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09affff700 1 -- 172.21.15.44:0/4138619102 learned_addr learned my addr 172.21.15.44:0/4138619102 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b4874700 1 -- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f09b00a4330 msgr2=0x7f09b00a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b4874700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f09b00a4330 0x7f09b00a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:42.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b4874700 1 -- 172.21.15.44:0/4138619102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09b00a9480 msgr2=0x7f09b00ad1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:42.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b4874700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09b00a9480 0x7f09b00ad1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:42.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b4874700 1 -- 172.21.15.44:0/4138619102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0151c90 con 0x7f09b00aa1e0 2020-11-20T12:10:42.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09affff700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f09b00a4330 0x7f09b00a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:42.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b4874700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00aa1e0 0x7f09b0152130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f09a8009c90 tx=0x7f09a8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:42.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09acff9700 1 -- 172.21.15.44:0/4138619102 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800bee0 con 0x7f09b00aa1e0 2020-11-20T12:10:42.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09acff9700 1 -- 172.21.15.44:0/4138619102 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a8010b10 con 0x7f09b00aa1e0 2020-11-20T12:10:42.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09acff9700 1 -- 172.21.15.44:0/4138619102 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a801d480 con 0x7f09b00aa1e0 2020-11-20T12:10:42.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b014e420 con 0x7f09b00aa1e0 2020-11-20T12:10:42.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.637+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b0152860 con 0x7f09b00aa1e0 2020-11-20T12:10:42.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.638+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b0061650 con 0x7f09b00aa1e0 2020-11-20T12:10:42.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.639+0000 7f09acff9700 1 -- 172.21.15.44:0/4138619102 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f09a8017070 con 0x7f09b00aa1e0 2020-11-20T12:10:42.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.639+0000 7f09acff9700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0984056cd0 0x7f0984059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:42.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.639+0000 7f09acff9700 1 -- 172.21.15.44:0/4138619102 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f09a80729a0 con 0x7f09b00aa1e0 2020-11-20T12:10:42.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.639+0000 7f09affff700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0984056cd0 0x7f0984059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:42.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.640+0000 7f09affff700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0984056cd0 0x7f0984059190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f09a00038d0 tx=0x7f09a0006e60).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:42.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.643+0000 7f09acff9700 1 -- 172.21.15.44:0/4138619102 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f09a8047c70 con 0x7f09b00aa1e0 2020-11-20T12:10:42.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.882+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f09b00a6820 con 0x7f09b00aa1e0 2020-11-20T12:10:42.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.882+0000 7f09acff9700 1 -- 172.21.15.44:0/4138619102 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f09a8015070 con 0x7f09b00aa1e0 2020-11-20T12:10:42.886 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:10:42.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0984056cd0 msgr2=0x7f0984059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:42.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0984056cd0 0x7f0984059190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f09a00038d0 tx=0x7f09a0006e60).stop 2020-11-20T12:10:42.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00aa1e0 msgr2=0x7f09b0152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:42.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00aa1e0 0x7f09b0152130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f09a8009c90 tx=0x7f09a8016000).stop 2020-11-20T12:10:42.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 shutdown_connections 2020-11-20T12:10:42.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0984056cd0 0x7f0984059190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:42.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09b00aa1e0 0x7f09b0152130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:42.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f09b00a4330 0x7f09b00a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:42.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 --2- 172.21.15.44:0/4138619102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09b00a9480 0x7f09b00ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:42.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 >> 172.21.15.44:0/4138619102 conn(0x7f09b0062690 msgr2=0x7f09b013e680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:42.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 shutdown_connections 2020-11-20T12:10:42.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:42.884+0000 7f09b62d7700 1 -- 172.21.15.44:0/4138619102 wait complete. 2020-11-20T12:10:45.900 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:10:46.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.036+0000 7fce27ff7700 1 Processor -- start 2020-11-20T12:10:46.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.036+0000 7fce27ff7700 1 -- start start 2020-11-20T12:10:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce27ff7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 0x7fce20142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce27ff7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce200a6260 con 0x7fce2013a990 2020-11-20T12:10:46.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce27ff7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce200a6490 con 0x7fce20060330 2020-11-20T12:10:46.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce27ff7700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce200a52e0 con 0x7fce20140810 2020-11-20T12:10:46.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce25592700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fce2013a990 legacy=0x7fce2013ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:10:46.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce26594700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 msgr2=0x7fce20142c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:46.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce26594700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 0x7fce20142c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:46.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce25d93700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fce20060330 0x7fce200a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:46354/0 (socket says 172.21.15.44:46354) 2020-11-20T12:10:46.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.037+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 learned_addr learned my addr 172.21.15.44:0/3602026626 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:46.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.238+0000 7fce26594700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 msgr2=0x7fce20142c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.238+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v1:172.21.15.44:6789/0 conn(0x7fce2013a990 legacy=0x7fce2013ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:10:46.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.238+0000 7fce26594700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 0x7fce20142c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:46.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.639+0000 7fce26594700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 msgr2=0x7fce20142c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.639+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v1:172.21.15.44:6789/0 conn(0x7fce2013a990 legacy=0x7fce2013ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:10:46.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:46.639+0000 7fce26594700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 0x7fce20142c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:47.440+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v1:172.21.15.44:6789/0 conn(0x7fce2013a990 legacy=0x7fce2013ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:47.440+0000 7fce26594700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 msgr2=0x7fce20142c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:47.440+0000 7fce26594700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 0x7fce20142c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:10:49.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 msgr2=0x7fce20142c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:49.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.44:3300/0 conn(0x7fce20140810 0x7fce20142c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:49.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 -- 172.21.15.44:0/3602026626 >> v1:172.21.15.79:6789/0 conn(0x7fce20060330 legacy=0x7fce200a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:49.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 -- 172.21.15.44:0/3602026626 >> v1:172.21.15.44:6789/0 conn(0x7fce2013a990 legacy=0x7fce2013ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:49.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:49.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:49.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 -- 172.21.15.44:0/3602026626 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce200a52e0 con 0x7fce00003e10 2020-11-20T12:10:49.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce0f7fe700 1 -- 172.21.15.44:0/3602026626 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0000ab60 con 0x7fce000033b0 2020-11-20T12:10:49.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:49.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.037+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:49.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 msgr2=0x7fce000037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:49.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 msgr2=0x7fce000037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:49.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 msgr2=0x7fce0000a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:49.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 msgr2=0x7fce0000a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:49.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:49.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:49.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:49.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.038+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:49.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:49.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:49.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 msgr2=0x7fce0000a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:49.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 msgr2=0x7fce0000a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:49.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:49.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 msgr2=0x7fce000037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:49.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 msgr2=0x7fce000037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:49.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:49.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:49.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.239+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:10:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:49.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 msgr2=0x7fce0000a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:10:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 msgr2=0x7fce000037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:10:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25592700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 msgr2=0x7fce0000a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 msgr2=0x7fce000037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:10:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:49.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:10:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:49.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:49.640+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:10:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.441+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.441+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 msgr2=0x7fce0000a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:10:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce25d93700 1 -- 172.21.15.44:0/3602026626 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce200a6730 con 0x7fce000033b0 2020-11-20T12:10:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce25592700 1 --2- 172.21.15.44:0/3602026626 >> v2:172.21.15.79:3300/0 conn(0x7fce00003e10 0x7fce0000a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:50.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce25d93700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fce1c007a90 tx=0x7fce1c009040).ready entity=mon.2 client_cookie=8ad686bbe8f7804d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce24d91700 1 -- 172.21.15.44:0/3602026626 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c014550 con 0x7fce000033b0 2020-11-20T12:10:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce24d91700 1 -- 172.21.15.44:0/3602026626 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c0146b0 con 0x7fce000033b0 2020-11-20T12:10:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.442+0000 7fce24d91700 1 -- 172.21.15.44:0/3602026626 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c014a30 con 0x7fce000033b0 2020-11-20T12:10:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.443+0000 7fce27ff7700 1 -- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 msgr2=0x7fce000037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.443+0000 7fce27ff7700 1 --2- 172.21.15.44:0/3602026626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce000033b0 0x7fce000037f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fce1c007a90 tx=0x7fce1c009040).stop 2020-11-20T12:10:50.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.443+0000 7fce26594700 1 -- 172.21.15.44:0/3602026626 reap_dead start 2020-11-20T12:10:50.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.443+0000 7fce27ff7700 1 -- 172.21.15.44:0/3602026626 shutdown_connections 2020-11-20T12:10:50.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.443+0000 7fce27ff7700 1 -- 172.21.15.44:0/3602026626 >> 172.21.15.44:0/3602026626 conn(0x7fce20062690 msgr2=0x7fce200a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:50.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.443+0000 7fce27ff7700 1 -- 172.21.15.44:0/3602026626 shutdown_connections 2020-11-20T12:10:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.443+0000 7fce27ff7700 1 -- 172.21.15.44:0/3602026626 wait complete. 2020-11-20T12:10:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.444+0000 7fce27ff7700 1 Processor -- start 2020-11-20T12:10:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.444+0000 7fce27ff7700 1 -- start start 2020-11-20T12:10:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce27ff7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000033b0 0x7fce201360a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce27ff7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce201329b0 0x7fce20133a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce27ff7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce20134aa0 0x7fce20136660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce27ff7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce200672e0 con 0x7fce20134aa0 2020-11-20T12:10:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce27ff7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2012d230 con 0x7fce000033b0 2020-11-20T12:10:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce27ff7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2012d4d0 con 0x7fce201329b0 2020-11-20T12:10:50.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce26594700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce20134aa0 msgr2=0x7fce20136660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce26594700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce20134aa0 0x7fce20136660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce25592700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce201329b0 0x7fce20133a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce25d93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000033b0 0x7fce201360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce25592700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce201329b0 0x7fce20133a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:51202/0 (socket says 172.21.15.44:51202) 2020-11-20T12:10:50.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce25d93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000033b0 0x7fce201360a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60048/0 (socket says 172.21.15.44:60048) 2020-11-20T12:10:50.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.445+0000 7fce25592700 1 -- 172.21.15.44:0/4208506639 learned_addr learned my addr 172.21.15.44:0/4208506639 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:50.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce25592700 1 -- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000033b0 msgr2=0x7fce201360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce25592700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000033b0 0x7fce201360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce25592700 1 -- 172.21.15.44:0/4208506639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce20134aa0 msgr2=0x7fce20136660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce25592700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce20134aa0 0x7fce20136660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce25592700 1 -- 172.21.15.44:0/4208506639 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20135c00 con 0x7fce201329b0 2020-11-20T12:10:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce25d93700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000033b0 0x7fce201360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce25592700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce201329b0 0x7fce20133a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fce10013e40 tx=0x7fce10008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce0dffb700 1 -- 172.21.15.44:0/4208506639 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10011d80 con 0x7fce201329b0 2020-11-20T12:10:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce0dffb700 1 -- 172.21.15.44:0/4208506639 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce100050d0 con 0x7fce201329b0 2020-11-20T12:10:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce20151d00 con 0x7fce201329b0 2020-11-20T12:10:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.446+0000 7fce0dffb700 1 -- 172.21.15.44:0/4208506639 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10011ee0 con 0x7fce201329b0 2020-11-20T12:10:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.447+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce20152270 con 0x7fce201329b0 2020-11-20T12:10:50.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.448+0000 7fce0dffb700 1 -- 172.21.15.44:0/4208506639 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fce1001b020 con 0x7fce201329b0 2020-11-20T12:10:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.448+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2012d090 con 0x7fce201329b0 2020-11-20T12:10:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.448+0000 7fce0dffb700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdf8056cd0 0x7fcdf8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.448+0000 7fce0dffb700 1 -- 172.21.15.44:0/4208506639 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fce10070d00 con 0x7fce201329b0 2020-11-20T12:10:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.448+0000 7fce25d93700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdf8056cd0 0x7fcdf8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.449+0000 7fce25d93700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdf8056cd0 0x7fcdf8059190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fce1c004560 tx=0x7fce1c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.453+0000 7fce0dffb700 1 -- 172.21.15.44:0/4208506639 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fce2012d090 con 0x7fce201329b0 2020-11-20T12:10:50.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.698+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fce2012d090 con 0x7fce201329b0 2020-11-20T12:10:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.699+0000 7fce0dffb700 1 -- 172.21.15.44:0/4208506639 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fce2012d090 con 0x7fce201329b0 2020-11-20T12:10:50.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdf8056cd0 msgr2=0x7fcdf8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdf8056cd0 0x7fcdf8059190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fce1c004560 tx=0x7fce1c009040).stop 2020-11-20T12:10:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce201329b0 msgr2=0x7fce20133a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce201329b0 0x7fce20133a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fce10013e40 tx=0x7fce10008040).stop 2020-11-20T12:10:50.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 shutdown_connections 2020-11-20T12:10:50.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcdf8056cd0 0x7fcdf8059190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce20134aa0 0x7fce20136660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce000033b0 0x7fce201360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 --2- 172.21.15.44:0/4208506639 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fce201329b0 0x7fce20133a10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 >> 172.21.15.44:0/4208506639 conn(0x7fce20062690 msgr2=0x7fce200a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:50.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 shutdown_connections 2020-11-20T12:10:50.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.701+0000 7fce27ff7700 1 -- 172.21.15.44:0/4208506639 wait complete. 2020-11-20T12:10:50.761 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":84,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:10:50.762 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:10:50.762 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:10:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.891+0000 7efc69da9700 1 Processor -- start 2020-11-20T12:10:50.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.891+0000 7efc69da9700 1 -- start start 2020-11-20T12:10:50.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc69da9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc640aa1e0 0x7efc64065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc69da9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efc6413c880 0x7efc64066030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc69da9700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc640a8bf0 con 0x7efc640a7810 2020-11-20T12:10:50.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc69da9700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc64066520 con 0x7efc6413c880 2020-11-20T12:10:50.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc69da9700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc640a86b0 con 0x7efc640aa1e0 2020-11-20T12:10:50.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc62ffd700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7efc640a7810 legacy=0x7efc6413c170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:10:50.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc63fff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7efc6413c880 msgr2=0x7efc64066030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:50.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc63fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efc6413c880 0x7efc64066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:50.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc637fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc640aa1e0 0x7efc64065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc637fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc640aa1e0 0x7efc64065b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60054/0 (socket says 172.21.15.44:60054) 2020-11-20T12:10:50.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc637fe700 1 -- 172.21.15.44:0/1818400921 learned_addr learned my addr 172.21.15.44:0/1818400921 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:50.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc637fe700 1 -- 172.21.15.44:0/1818400921 >> v2:172.21.15.44:3300/0 conn(0x7efc6413c880 msgr2=0x7efc64066030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:50.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc637fe700 1 --2- 172.21.15.44:0/1818400921 >> v2:172.21.15.44:3300/0 conn(0x7efc6413c880 0x7efc64066030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:50.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc637fe700 1 -- 172.21.15.44:0/1818400921 >> v1:172.21.15.44:6789/0 conn(0x7efc640a7810 legacy=0x7efc6413c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:50.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.892+0000 7efc637fe700 1 -- 172.21.15.44:0/1818400921 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc640a8950 con 0x7efc640aa1e0 2020-11-20T12:10:50.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc637fe700 1 --2- 172.21.15.44:0/1818400921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640aa1e0 0x7efc64065b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7efc640adec0 tx=0x7efc54006eb0).ready entity=mon.1 client_cookie=6836ffc3c4c8ac27 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc627fc700 1 -- 172.21.15.44:0/1818400921 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54009920 con 0x7efc640aa1e0 2020-11-20T12:10:50.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc627fc700 1 -- 172.21.15.44:0/1818400921 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc54009a80 con 0x7efc640aa1e0 2020-11-20T12:10:50.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc627fc700 1 -- 172.21.15.44:0/1818400921 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54009e00 con 0x7efc640aa1e0 2020-11-20T12:10:50.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 -- 172.21.15.44:0/1818400921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640aa1e0 msgr2=0x7efc64065b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 --2- 172.21.15.44:0/1818400921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640aa1e0 0x7efc64065b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7efc640adec0 tx=0x7efc54006eb0).stop 2020-11-20T12:10:50.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 -- 172.21.15.44:0/1818400921 shutdown_connections 2020-11-20T12:10:50.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 --2- 172.21.15.44:0/1818400921 >> v2:172.21.15.44:3300/0 conn(0x7efc6413c880 0x7efc64066030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:50.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 --2- 172.21.15.44:0/1818400921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640aa1e0 0x7efc64065b40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 -- 172.21.15.44:0/1818400921 >> 172.21.15.44:0/1818400921 conn(0x7efc64062690 msgr2=0x7efc640a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 -- 172.21.15.44:0/1818400921 shutdown_connections 2020-11-20T12:10:50.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.893+0000 7efc69da9700 1 -- 172.21.15.44:0/1818400921 wait complete. 2020-11-20T12:10:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.894+0000 7efc69da9700 1 Processor -- start 2020-11-20T12:10:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.894+0000 7efc69da9700 1 -- start start 2020-11-20T12:10:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc69da9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a7810 0x7efc640a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc69da9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640aa1e0 0x7efc64147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc69da9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc6413c880 0x7efc641157f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc69da9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6414b610 con 0x7efc640aa1e0 2020-11-20T12:10:50.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc69da9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6414b880 con 0x7efc640a7810 2020-11-20T12:10:50.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc69da9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6414bb20 con 0x7efc6413c880 2020-11-20T12:10:50.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc62ffd700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640aa1e0 msgr2=0x7efc64147d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:50.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc62ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640aa1e0 0x7efc64147d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:50.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc637fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a7810 0x7efc640a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc637fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a7810 0x7efc640a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60058/0 (socket says 172.21.15.44:60058) 2020-11-20T12:10:50.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc637fe700 1 -- 172.21.15.44:0/458435472 learned_addr learned my addr 172.21.15.44:0/458435472 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:50.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.895+0000 7efc63fff700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc6413c880 0x7efc641157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc637fe700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc6413c880 msgr2=0x7efc641157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc637fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc6413c880 0x7efc641157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc637fe700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640aa1e0 msgr2=0x7efc64147d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:50.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc637fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc640aa1e0 0x7efc64147d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc637fe700 1 -- 172.21.15.44:0/458435472 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc640a35f0 con 0x7efc640a7810 2020-11-20T12:10:50.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc63fff700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc6413c880 0x7efc641157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc637fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a7810 0x7efc640a3100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7efc54006ef0 tx=0x7efc54015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5401e460 con 0x7efc640a7810 2020-11-20T12:10:50.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5401e5c0 con 0x7efc640a7810 2020-11-20T12:10:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc640a0ea0 con 0x7efc640a7810 2020-11-20T12:10:50.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5401e980 con 0x7efc640a7810 2020-11-20T12:10:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.896+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc641481f0 con 0x7efc640a7810 2020-11-20T12:10:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.897+0000 7efc69da9700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc6414bce0 0x7efc64148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.897+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641483e0 con 0x7efc6414bce0 2020-11-20T12:10:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.897+0000 7efc62ffd700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc6414bce0 0x7efc64148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.897+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7efc54025030 con 0x7efc640a7810 2020-11-20T12:10:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.897+0000 7efc62ffd700 1 -- 172.21.15.44:0/458435472 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc64061650 con 0x7efc6414bce0 2020-11-20T12:10:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.897+0000 7efc62ffd700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc6414bce0 0x7efc64148590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7efc6413e360 tx=0x7efc58006e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc3c056cd0 0x7efc3c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7efc540719b0 con 0x7efc640a7810 2020-11-20T12:10:50.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc63fff700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc3c056cd0 0x7efc3c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc580050d0 con 0x7efc6414bce0 2020-11-20T12:10:50.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7efc58005230 con 0x7efc6414bce0 2020-11-20T12:10:50.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc6414bce0 msgr2=0x7efc64148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc6414bce0 0x7efc64148590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7efc6413e360 tx=0x7efc58006e60).stop 2020-11-20T12:10:50.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc6414bce0 msgr2=0x7efc64148590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc437fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc6414bce0 0x7efc64148590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.898+0000 7efc63fff700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc3c056cd0 0x7efc3c059190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7efc64115710 tx=0x7efc5c009060).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.906+0000 7efc69da9700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a1980 0x7efc640a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:50.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.906+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc64139ad0 con 0x7efc640a1980 2020-11-20T12:10:50.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.906+0000 7efc62ffd700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a1980 0x7efc640a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.906+0000 7efc62ffd700 1 -- 172.21.15.44:0/458435472 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efc640446d0 con 0x7efc640a1980 2020-11-20T12:10:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.906+0000 7efc62ffd700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a1980 0x7efc640a1d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7efc6413e360 tx=0x7efc58013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:50.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58015c60 con 0x7efc640a1980 2020-11-20T12:10:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4426 (secure 0 0 0) 0x7efc58015dc0 con 0x7efc640a1980 2020-11-20T12:10:50.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a1980 msgr2=0x7efc640a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc437fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a1980 0x7efc640a1d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7efc6413e360 tx=0x7efc58013000).stop 2020-11-20T12:10:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc437fe700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a1980 msgr2=0x7efc640a1d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:50.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc437fe700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a1980 0x7efc640a1d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc3c056cd0 msgr2=0x7efc3c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc3c056cd0 0x7efc3c059190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7efc64115710 tx=0x7efc5c009060).stop 2020-11-20T12:10:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a7810 msgr2=0x7efc640a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 --2- 172.21.15.44:0/458435472 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc640a7810 0x7efc640a3100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7efc54006ef0 tx=0x7efc54015000).stop 2020-11-20T12:10:50.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc63fff700 1 -- 172.21.15.44:0/458435472 reap_dead start 2020-11-20T12:10:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc63fff700 1 -- 172.21.15.44:0/458435472 reap_dead start 2020-11-20T12:10:50.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 shutdown_connections 2020-11-20T12:10:50.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 >> 172.21.15.44:0/458435472 conn(0x7efc64062690 msgr2=0x7efc640ac600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:50.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 shutdown_connections 2020-11-20T12:10:50.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:50.907+0000 7efc69da9700 1 -- 172.21.15.44:0/458435472 wait complete. 2020-11-20T12:10:50.945 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:50.945 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:50.946 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:50.946 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:10:50.947 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 84, 2020-11-20T12:10:50.947 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:50.947 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:50.948 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:50.948 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:50.949 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:10:50.949 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:50.949 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:50.950 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:50.950 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:50.951 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:50.951 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:50.951 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:50.952 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:50.952 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:50.953 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:50.953 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:50.954 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:50.954 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:50.954 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:50.955 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:50.955 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:50.955 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:50.956 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:50.956 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:50.956 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:50.956 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:50.956 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:50.957 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:50.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.957 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:50.957 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:50.958 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:50.958 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:50.958 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:50.958 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:50.958 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:50.959 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:50.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:50.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:50.959 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:50.959 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:50.960 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:50.960 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:50.960 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:50.960 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:50.960 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:50.961 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:50.961 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:50.961 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:50.961 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:50.962 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:50.962 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:50.962 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:50.962 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:50.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.963 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:50.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.963 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:50.963 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:50.963 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:50.964 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:50.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:50.964 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:50.964 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:50.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:50.965 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:50.965 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:50.966 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:50.966 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:50.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.966 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:50.966 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:50.967 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:50.967 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:50.967 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:50.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.968 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:50.968 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:50.968 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:50.968 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:50.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:50.969 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:50.969 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:50.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:50.970 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:50.970 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:50.970 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:50.970 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:50.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.971 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:50.971 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:50.971 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:50.972 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:50.972 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:50.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.972 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:50.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:50.973 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:50.973 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:50.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:50.974 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:50.974 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:50.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:50.975 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:50.975 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:50.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:50.975 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:50.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.976 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:50.976 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:50.976 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:50.976 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:50.977 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:50.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:50.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:50.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.977 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:50.978 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:50.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.978 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:50.978 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:50.978 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:50.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:50.979 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:50.979 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:10:50.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.979 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:50.980 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:50.980 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:50.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:50.980 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:50.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:50.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:50.981 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:50.981 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:50.981 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:50.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:50.982 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:50.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:50.982 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:50.982 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:50.983 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:10:51.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.106+0000 7f3c805d4700 1 Processor -- start 2020-11-20T12:10:51.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c805d4700 1 -- start start 2020-11-20T12:10:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c805d4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c780aa400 0x7f3c780a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c805d4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3c7813ac60 0x7f3c780a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c805d4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c780a9770 con 0x7f3c780aa020 2020-11-20T12:10:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c805d4700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780a56c0 con 0x7f3c780aa400 2020-11-20T12:10:51.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c805d4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780a6da0 con 0x7f3c7813ac60 2020-11-20T12:10:51.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c7db6f700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3c780aa400 msgr2=0x7f3c780a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:51.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.107+0000 7f3c7db6f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3c780aa400 0x7f3c780a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:51.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7eb71700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3c7813ac60 0x7f3c780a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:51.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7e370700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3c780aa020 0x7f3c780a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:46412/0 (socket says 172.21.15.44:46412) 2020-11-20T12:10:51.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7e370700 1 -- 172.21.15.44:0/2267541589 learned_addr learned my addr 172.21.15.44:0/2267541589 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:10:51.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7d36e700 1 -- 172.21.15.44:0/2267541589 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023749204 0 0) 0x7f3c6c0010d0 con 0x7f3c780aa020 2020-11-20T12:10:51.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7d36e700 1 -- 172.21.15.44:0/2267541589 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c60003240 con 0x7f3c780aa020 2020-11-20T12:10:51.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7eb71700 1 -- 172.21.15.44:0/2267541589 >> v2:172.21.15.44:3300/0 conn(0x7f3c780aa400 msgr2=0x7f3c780a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:10:51.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7eb71700 1 --2- 172.21.15.44:0/2267541589 >> v2:172.21.15.44:3300/0 conn(0x7f3c780aa400 0x7f3c780a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:51.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7eb71700 1 -- 172.21.15.44:0/2267541589 >> v1:172.21.15.79:6789/0 conn(0x7f3c780aa020 legacy=0x7f3c780a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:51.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7eb71700 1 -- 172.21.15.44:0/2267541589 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780a7040 con 0x7f3c7813ac60 2020-11-20T12:10:51.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7eb71700 1 --2- 172.21.15.44:0/2267541589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c780a3700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f3c780a8f70 tx=0x7f3c700091d0).ready entity=mon.1 client_cookie=fa4a8e6a00713b83 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:51.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7d36e700 1 -- 172.21.15.44:0/2267541589 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000ba70 con 0x7f3c7813ac60 2020-11-20T12:10:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7d36e700 1 -- 172.21.15.44:0/2267541589 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7000bbd0 con 0x7f3c7813ac60 2020-11-20T12:10:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.108+0000 7f3c7d36e700 1 -- 172.21.15.44:0/2267541589 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70011400 con 0x7f3c7813ac60 2020-11-20T12:10:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 -- 172.21.15.44:0/2267541589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 msgr2=0x7f3c780a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 --2- 172.21.15.44:0/2267541589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c780a3700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f3c780a8f70 tx=0x7f3c700091d0).stop 2020-11-20T12:10:51.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 -- 172.21.15.44:0/2267541589 shutdown_connections 2020-11-20T12:10:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 --2- 172.21.15.44:0/2267541589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c780a3700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 --2- 172.21.15.44:0/2267541589 >> v2:172.21.15.44:3300/0 conn(0x7f3c780aa400 0x7f3c780a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 -- 172.21.15.44:0/2267541589 >> 172.21.15.44:0/2267541589 conn(0x7f3c78062690 msgr2=0x7f3c780a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 -- 172.21.15.44:0/2267541589 shutdown_connections 2020-11-20T12:10:51.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.109+0000 7f3c805d4700 1 -- 172.21.15.44:0/2267541589 wait complete. 2020-11-20T12:10:51.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 Processor -- start 2020-11-20T12:10:51.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 -- start start 2020-11-20T12:10:51.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c780aa020 0x7f3c7813ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c780aa400 0x7f3c781327b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c7813c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7813a100 con 0x7f3c780aa020 2020-11-20T12:10:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7813d4e0 con 0x7f3c7813ac60 2020-11-20T12:10:51.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.110+0000 7f3c805d4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7813d780 con 0x7f3c780aa400 2020-11-20T12:10:51.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7e370700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c780aa020 msgr2=0x7f3c7813ce40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:10:51.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7e370700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c780aa020 0x7f3c7813ce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:10:51.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7eb71700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c7813c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:51.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7db6f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c780aa400 0x7f3c781327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:51.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7eb71700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c7813c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60078/0 (socket says 172.21.15.44:60078) 2020-11-20T12:10:51.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7eb71700 1 -- 172.21.15.44:0/1425407704 learned_addr learned my addr 172.21.15.44:0/1425407704 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:10:51.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7db6f700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 msgr2=0x7f3c7813c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:51.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7db6f700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c7813c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:51.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7db6f700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c780aa020 msgr2=0x7f3c7813ce40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:10:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7db6f700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c780aa020 0x7f3c7813ce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:10:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7db6f700 1 -- 172.21.15.44:0/1425407704 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780655b0 con 0x7f3c780aa400 2020-11-20T12:10:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7eb71700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c7813ac60 0x7f3c7813c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:10:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c7db6f700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c780aa400 0x7f3c781327b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f3c740096b0 tx=0x7f3c74006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:51.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74006170 con 0x7f3c780aa400 2020-11-20T12:10:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c740062f0 con 0x7f3c780aa400 2020-11-20T12:10:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c78065800 con 0x7f3c780aa400 2020-11-20T12:10:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740038e0 con 0x7f3c780aa400 2020-11-20T12:10:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.111+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7813c7d0 con 0x7f3c780aa400 2020-11-20T12:10:51.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.112+0000 7f3c805d4700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7812f030 0x7f3c7812f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.112+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7813cab0 con 0x7f3c7812f030 2020-11-20T12:10:51.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.112+0000 7f3c7e370700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7812f030 0x7f3c7812f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:51.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3c7400f070 con 0x7f3c780aa400 2020-11-20T12:10:51.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c7e370700 1 -- 172.21.15.44:0/1425407704 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c7813d980 con 0x7f3c7812f030 2020-11-20T12:10:51.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c7e370700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7812f030 0x7f3c7812f410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f3c6c004590 tx=0x7f3c6c007980).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:51.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c50056cd0 0x7f3c50059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3c7406f890 con 0x7f3c780aa400 2020-11-20T12:10:51.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0072e0 con 0x7f3c7812f030 2020-11-20T12:10:51.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c7eb71700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c50056cd0 0x7f3c50059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:51.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f3c6c007440 con 0x7f3c7812f030 2020-11-20T12:10:51.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7812f030 msgr2=0x7f3c7812f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:51.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7812f030 0x7f3c7812f410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f3c6c004590 tx=0x7f3c6c007980).stop 2020-11-20T12:10:51.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7812f030 msgr2=0x7f3c7812f410 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:51.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.113+0000 7f3c6a7fc700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7812f030 0x7f3c7812f410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:51.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.114+0000 7f3c7eb71700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c50056cd0 0x7f3c50059190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f3c7813ec80 tx=0x7f3c70016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:51.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.121+0000 7f3c805d4700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7814b020 0x7f3c7814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:10:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.121+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7813d380 con 0x7f3c7814b020 2020-11-20T12:10:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.121+0000 7f3c7e370700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7814b020 0x7f3c7814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:10:51.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.122+0000 7f3c7e370700 1 -- 172.21.15.44:0/1425407704 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3c78061650 con 0x7f3c7814b020 2020-11-20T12:10:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.122+0000 7f3c7e370700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7814b020 0x7f3c7814b430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f3c6c012000 tx=0x7f3c6c013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:10:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.122+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c013bb0 con 0x7f3c7814b020 2020-11-20T12:10:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.122+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4764 (secure 0 0 0) 0x7f3c6c0070d0 con 0x7f3c7814b020 2020-11-20T12:10:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.122+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7814b020 msgr2=0x7f3c7814b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:51.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.122+0000 7f3c6a7fc700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7814b020 0x7f3c7814b430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f3c6c012000 tx=0x7f3c6c013000).stop 2020-11-20T12:10:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c6a7fc700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7814b020 msgr2=0x7f3c7814b430 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:10:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c6a7fc700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c7814b020 0x7f3c7814b430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:10:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c50056cd0 msgr2=0x7f3c50059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3c50056cd0 0x7f3c50059190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f3c7813ec80 tx=0x7f3c70016000).stop 2020-11-20T12:10:51.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c780aa400 msgr2=0x7f3c781327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:10:51.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 --2- 172.21.15.44:0/1425407704 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3c780aa400 0x7f3c781327b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f3c740096b0 tx=0x7f3c74006dd0).stop 2020-11-20T12:10:51.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c7eb71700 1 -- 172.21.15.44:0/1425407704 reap_dead start 2020-11-20T12:10:51.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c7eb71700 1 -- 172.21.15.44:0/1425407704 reap_dead start 2020-11-20T12:10:51.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 shutdown_connections 2020-11-20T12:10:51.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 >> 172.21.15.44:0/1425407704 conn(0x7f3c78062690 msgr2=0x7f3c780a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:10:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 shutdown_connections 2020-11-20T12:10:51.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:10:51.123+0000 7f3c805d4700 1 -- 172.21.15.44:0/1425407704 wait complete. 2020-11-20T12:10:51.137 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:10:51.137 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:51.138 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:51.138 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:10:51.138 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 84, 2020-11-20T12:10:51.138 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:10:51.138 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:10:51.139 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:10:51.139 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:51.139 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:10:51.139 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:10:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:10:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:10:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:10:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:51.144 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:51.144 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:51.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.144 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:10:51.145 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:10:51.145 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:10:51.145 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:10:51.145 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:10:51.145 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:10:51.146 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:51.146 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:10:51.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:10:51.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:10:51.146 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:10:51.147 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:10:51.147 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:10:51.147 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:10:51.147 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:10:51.147 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:10:51.148 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:10:51.148 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:10:51.148 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:10:51.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:10:51.148 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:10:51.149 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:10:51.149 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:10:51.149 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:51.149 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:10:51.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:10:51.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.150 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:10:51.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:10:51.151 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:51.151 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:51.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:51.151 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:10:51.152 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:51.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:51.152 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:10:51.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:51.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.153 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:51.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:10:51.154 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:10:51.154 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:51.154 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:51.154 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:51.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.155 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:10:51.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:10:51.155 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:51.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:51.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:51.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:10:51.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:51.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:51.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:10:51.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:51.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:51.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:10:51.158 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:10:51.159 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:51.159 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:51.159 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:51.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.160 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:10:51.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:10:51.160 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:10:51.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:10:51.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:10:51.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:10:51.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:51.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:10:51.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:10:51.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:10:51.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:51.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:10:51.163 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:10:51.163 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:10:51.164 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:10:51.164 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:10:51.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:51.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:10:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:10:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:51.165 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:10:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:10:51.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.167 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:51.167 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:51.167 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:51.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.167 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:51.168 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:10:51.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.168 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:51.168 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:51.168 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:10:51.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.169 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:51.169 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:10:51.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.170 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:51.170 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:51.170 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:10:51.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.170 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:10:51.171 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:10:51.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:10:51.171 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:10:51.171 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:10:51.171 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:10:51.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:10:51.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:10:51.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:10:51.172 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:10:51.172 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:10:51.174 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:11:07.296 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:11:07.299 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:11:11.174 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:11:11.175 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:11:11.175 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:11:11.175 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:11:11.179 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:11:11.179 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:11:11.180 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.317+0000 7f2a46e41700 1 Processor -- start 2020-11-20T12:11:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a46e41700 1 -- start start 2020-11-20T12:11:11.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a46e41700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 0x7f2a40060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:11.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a46e41700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a4013ebf0 0x7f2a40137b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:11.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a46e41700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4013e080 con 0x7f2a400669d0 2020-11-20T12:11:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a46e41700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a40060b40 con 0x7f2a400ae050 2020-11-20T12:11:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a46e41700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40060d60 con 0x7f2a4013ebf0 2020-11-20T12:11:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a44bdd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 0x7f2a40060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a3ffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2a400669d0 0x7f2a4013e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43562/0 (socket says 172.21.15.44:43562) 2020-11-20T12:11:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a3ffff700 1 -- 172.21.15.44:0/2605162610 learned_addr learned my addr 172.21.15.44:0/2605162610 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:11.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a44bdd700 1 -- 172.21.15.44:0/2605162610 >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 msgr2=0x7f2a40060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:11.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a44bdd700 1 -- 172.21.15.44:0/2605162610 >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 msgr2=0x7f2a40060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:11.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.318+0000 7f2a44bdd700 1 --2- 172.21.15.44:0/2605162610 >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 0x7f2a40060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:11.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a44bdd700 1 --2- 172.21.15.44:0/2605162610 >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 0x7f2a40060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a453de700 1 --2- 172.21.15.44:0/2605162610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a4013ebf0 0x7f2a40137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a453de700 1 -- 172.21.15.44:0/2605162610 >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 msgr2=0x7f2a40060650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:11.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a453de700 1 --2- 172.21.15.44:0/2605162610 >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 0x7f2a40060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a453de700 1 -- 172.21.15.44:0/2605162610 >> v1:172.21.15.44:6789/0 conn(0x7f2a400669d0 legacy=0x7f2a4013e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:11.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a453de700 1 -- 172.21.15.44:0/2605162610 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400ae430 con 0x7f2a4013ebf0 2020-11-20T12:11:11.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a453de700 1 --2- 172.21.15.44:0/2605162610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a4013ebf0 0x7f2a40137b10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f2a38009c90 tx=0x7f2a380091d0).ready entity=mon.2 client_cookie=cca246bcb11e4a61 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:11.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a3f7fe700 1 -- 172.21.15.44:0/2605162610 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800bbc0 con 0x7f2a4013ebf0 2020-11-20T12:11:11.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a3f7fe700 1 -- 172.21.15.44:0/2605162610 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3800bd20 con 0x7f2a4013ebf0 2020-11-20T12:11:11.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.319+0000 7f2a3f7fe700 1 -- 172.21.15.44:0/2605162610 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38011610 con 0x7f2a4013ebf0 2020-11-20T12:11:11.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 -- 172.21.15.44:0/2605162610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a4013ebf0 msgr2=0x7f2a40137b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:11.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2605162610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a4013ebf0 0x7f2a40137b10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f2a38009c90 tx=0x7f2a380091d0).stop 2020-11-20T12:11:11.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 -- 172.21.15.44:0/2605162610 shutdown_connections 2020-11-20T12:11:11.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2605162610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a4013ebf0 0x7f2a40137b10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2605162610 >> v2:172.21.15.44:3300/0 conn(0x7f2a400ae050 0x7f2a40060650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 -- 172.21.15.44:0/2605162610 >> 172.21.15.44:0/2605162610 conn(0x7f2a40062bc0 msgr2=0x7f2a40062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:11.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 -- 172.21.15.44:0/2605162610 shutdown_connections 2020-11-20T12:11:11.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.320+0000 7f2a46e41700 1 -- 172.21.15.44:0/2605162610 wait complete. 2020-11-20T12:11:11.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.321+0000 7f2a46e41700 1 Processor -- start 2020-11-20T12:11:11.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.321+0000 7f2a46e41700 1 -- start start 2020-11-20T12:11:11.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.321+0000 7f2a46e41700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a400669d0 0x7f2a4013a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:11.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.321+0000 7f2a46e41700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 0x7f2a400acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:11.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a46e41700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a4013ebf0 0x7f2a4013cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a46e41700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400abf50 con 0x7f2a400ae050 2020-11-20T12:11:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a46e41700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400ac1b0 con 0x7f2a4013ebf0 2020-11-20T12:11:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a46e41700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400ac450 con 0x7f2a400669d0 2020-11-20T12:11:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a3ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 0x7f2a400acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:11.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a3ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 0x7f2a400acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39820/0 (socket says 172.21.15.44:39820) 2020-11-20T12:11:11.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a3ffff700 1 -- 172.21.15.44:0/2935396191 learned_addr learned my addr 172.21.15.44:0/2935396191 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:11.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a3ffff700 1 -- 172.21.15.44:0/2935396191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 msgr2=0x7f2a400acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:11:11.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a3ffff700 1 -- 172.21.15.44:0/2935396191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 msgr2=0x7f2a400acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:11:11.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a3ffff700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 0x7f2a400acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:11.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a3ffff700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 0x7f2a400acd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:11.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a44bdd700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a400669d0 0x7f2a4013a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:11.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a453de700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a4013ebf0 0x7f2a4013cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:11.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a44bdd700 1 -- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a4013ebf0 msgr2=0x7f2a4013cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:11.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a44bdd700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a4013ebf0 0x7f2a4013cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a44bdd700 1 -- 172.21.15.44:0/2935396191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 msgr2=0x7f2a400acd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:11.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a44bdd700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 0x7f2a400acd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a44bdd700 1 -- 172.21.15.44:0/2935396191 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a40136dc0 con 0x7f2a400669d0 2020-11-20T12:11:11.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a453de700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a4013ebf0 0x7f2a4013cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:11:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.322+0000 7f2a44bdd700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a400669d0 0x7f2a4013a340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f2a34006c00 tx=0x7f2a34008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.323+0000 7f2a3cff9700 1 -- 172.21.15.44:0/2935396191 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340125f0 con 0x7f2a400669d0 2020-11-20T12:11:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.323+0000 7f2a3cff9700 1 -- 172.21.15.44:0/2935396191 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a34012750 con 0x7f2a400669d0 2020-11-20T12:11:11.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.323+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a400ad240 con 0x7f2a400669d0 2020-11-20T12:11:11.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.323+0000 7f2a3cff9700 1 -- 172.21.15.44:0/2935396191 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34012af0 con 0x7f2a400669d0 2020-11-20T12:11:11.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.323+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4013d400 con 0x7f2a400669d0 2020-11-20T12:11:11.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.324+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40138080 con 0x7f2a400669d0 2020-11-20T12:11:11.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.324+0000 7f2a3cff9700 1 -- 172.21.15.44:0/2935396191 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2a34013480 con 0x7f2a400669d0 2020-11-20T12:11:11.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.324+0000 7f2a3cff9700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a18056d20 0x7f2a180591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:11.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.324+0000 7f2a3cff9700 1 -- 172.21.15.44:0/2935396191 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2a340704c0 con 0x7f2a400669d0 2020-11-20T12:11:11.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.324+0000 7f2a3ffff700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a18056d20 0x7f2a180591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:11.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.325+0000 7f2a3ffff700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a18056d20 0x7f2a180591e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f2a401479c0 tx=0x7f2a2c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:11.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.328+0000 7f2a3cff9700 1 -- 172.21.15.44:0/2935396191 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2a34045620 con 0x7f2a400669d0 2020-11-20T12:11:11.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.576+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2a400a50c0 con 0x7f2a400669d0 2020-11-20T12:11:11.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.577+0000 7f2a3cff9700 1 -- 172.21.15.44:0/2935396191 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f2a34048c40 con 0x7f2a400669d0 2020-11-20T12:11:11.580 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":84,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:11.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a18056d20 msgr2=0x7f2a180591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:11.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a18056d20 0x7f2a180591e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f2a401479c0 tx=0x7f2a2c009040).stop 2020-11-20T12:11:11.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a400669d0 msgr2=0x7f2a4013a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:11.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a400669d0 0x7f2a4013a340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f2a34006c00 tx=0x7f2a34008040).stop 2020-11-20T12:11:11.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 shutdown_connections 2020-11-20T12:11:11.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a18056d20 0x7f2a180591e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a4013ebf0 0x7f2a4013cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a400669d0 0x7f2a4013a340 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 --2- 172.21.15.44:0/2935396191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a400ae050 0x7f2a400acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:11.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 >> 172.21.15.44:0/2935396191 conn(0x7f2a40062bc0 msgr2=0x7f2a4005ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:11.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 shutdown_connections 2020-11-20T12:11:11.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:11.578+0000 7f2a46e41700 1 -- 172.21.15.44:0/2935396191 wait complete. 2020-11-20T12:11:14.594 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:14.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.743+0000 7fb6f6ccb700 1 Processor -- start 2020-11-20T12:11:14.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.743+0000 7fb6f6ccb700 1 -- start start 2020-11-20T12:11:14.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f6ccb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:14.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f6ccb700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f00a9770 con 0x7fb6f00aa020 2020-11-20T12:11:14.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f6ccb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f00a6da0 con 0x7fb6f00aa400 2020-11-20T12:11:14.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f6ccb700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f00a56c0 con 0x7fb6f00a3e10 2020-11-20T12:11:14.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f5268700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:14.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f4a67700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb6f00aa020 0x7fb6f00a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43590/0 (socket says 172.21.15.44:43590) 2020-11-20T12:11:14.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 learned_addr learned my addr 172.21.15.44:0/3203425241 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:14.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:11:14.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:14.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:14.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.744+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:14.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.945+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:14.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.945+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:11:14.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.945+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:14.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.945+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:14.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:14.945+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:15.346+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:15.346+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:11:15.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:15.346+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:15.346+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:15.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:15.346+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:16.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:16.147+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:16.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:16.147+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:11:16.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:16.147+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:16.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:16.147+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:16.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:16.147+0000 7fb6f5268700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:11:17.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 msgr2=0x7fb6f00a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.44:3300/0 conn(0x7fb6f00a3e10 0x7fb6f00a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 >> v1:172.21.15.79:6789/0 conn(0x7fb6f00aa400 legacy=0x7fb6f00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 >> v1:172.21.15.44:6789/0 conn(0x7fb6f00aa020 legacy=0x7fb6f00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:17.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f00a56c0 con 0x7fb6d00033b0 2020-11-20T12:11:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00089a0 con 0x7fb6d0005e20 2020-11-20T12:11:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:17.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:17.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.744+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:17.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:17.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:17.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:17.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:17.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:17.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:17.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:17.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:17.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:17.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:17.946+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:18.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:18.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:18.347+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:19.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:19.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:19.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:19.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:19.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:19.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:19.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:11:19.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:19.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:19.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:19.148+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:11:20.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 msgr2=0x7fb6d0005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d00033b0 0x7fb6d0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6f5268700 1 -- 172.21.15.44:0/3203425241 reap_dead start 2020-11-20T12:11:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f00a7040 con 0x7fb6d00092f0 2020-11-20T12:11:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f00a56c0 con 0x7fb6d0003820 2020-11-20T12:11:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6ee7fc700 1 -- 172.21.15.44:0/3203425241 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00089a0 con 0x7fb6d0005e20 2020-11-20T12:11:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:20.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 msgr2=0x7fb6d0008e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 msgr2=0x7fb6d0008e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.745+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:20.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:20.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 msgr2=0x7fb6d0008e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 msgr2=0x7fb6d0008e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:20.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:20.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:20.946+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:21.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:21.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6f4a67700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:21.356 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 msgr2=0x7fb6d0008e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 msgr2=0x7fb6d0008e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:21.359 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:21.360 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:21.347+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:22.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.148+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:22.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.148+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> v2:172.21.15.79:3300/0 conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:22.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.148+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6d00082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:22.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.148+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.148+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 >> v1:172.21.15.44:6789/0 conn(0x7fb6d00092f0 legacy=0x7fb6d000b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:22.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.148+0000 7fb6effff700 1 -- 172.21.15.44:0/3203425241 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d000e060 con 0x7fb6d0003820 2020-11-20T12:11:22.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.148+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:11:22.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.149+0000 7fb6effff700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6d0008e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fb6e40079a0 tx=0x7fb6e4009040).ready entity=mon.1 client_cookie=9c83233ad8f1cc24 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:22.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.149+0000 7fb6ef7fe700 1 -- 172.21.15.44:0/3203425241 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4007240 con 0x7fb6d0003820 2020-11-20T12:11:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.149+0000 7fb6ef7fe700 1 -- 172.21.15.44:0/3203425241 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e40073a0 con 0x7fb6d0003820 2020-11-20T12:11:22.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.149+0000 7fb6ef7fe700 1 -- 172.21.15.44:0/3203425241 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4018980 con 0x7fb6d0003820 2020-11-20T12:11:22.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 msgr2=0x7fb6d0008e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:22.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6d0008e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fb6e40079a0 tx=0x7fb6e4009040).stop 2020-11-20T12:11:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/3203425241 shutdown_connections 2020-11-20T12:11:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6d0008e00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/3203425241 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6d00082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/3203425241 >> 172.21.15.44:0/3203425241 conn(0x7fb6f0062690 msgr2=0x7fb6f00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/3203425241 shutdown_connections 2020-11-20T12:11:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.150+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/3203425241 wait complete. 2020-11-20T12:11:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.151+0000 7fb6f6ccb700 1 Processor -- start 2020-11-20T12:11:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.151+0000 7fb6f6ccb700 1 -- start start 2020-11-20T12:11:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f6ccb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6f0065400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f6ccb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6f0066b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f6ccb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 0x7fb6f012b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f6ccb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f0135300 con 0x7fb6d00092f0 2020-11-20T12:11:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f6ccb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00aae50 con 0x7fb6d0003820 2020-11-20T12:11:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f6ccb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00ab0f0 con 0x7fb6d0005e20 2020-11-20T12:11:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f5268700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 0x7fb6f012b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f5268700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 0x7fb6f012b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39888/0 (socket says 172.21.15.44:39888) 2020-11-20T12:11:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f5268700 1 -- 172.21.15.44:0/2858484732 learned_addr learned my addr 172.21.15.44:0/2858484732 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f5268700 1 -- 172.21.15.44:0/2858484732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 msgr2=0x7fb6f012b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:11:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6effff700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6f0066b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f5268700 1 -- 172.21.15.44:0/2858484732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 msgr2=0x7fb6f012b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:11:22.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6f0065400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:22.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f5268700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 0x7fb6f012b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:22.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.152+0000 7fb6f5268700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 0x7fb6f012b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:22.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6effff700 1 -- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 msgr2=0x7fb6f0065400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:22.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6effff700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6f0065400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6effff700 1 -- 172.21.15.44:0/2858484732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 msgr2=0x7fb6f012b250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:22.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6effff700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 0x7fb6f012b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6effff700 1 -- 172.21.15.44:0/2858484732 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f00ab390 con 0x7fb6d0005e20 2020-11-20T12:11:22.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6f0065400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:11:22.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6effff700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6f0066b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb6e40079a0 tx=0x7fb6e4023000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:22.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6ecff9700 1 -- 172.21.15.44:0/2858484732 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4023430 con 0x7fb6d0005e20 2020-11-20T12:11:22.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6ecff9700 1 -- 172.21.15.44:0/2858484732 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e4023590 con 0x7fb6d0005e20 2020-11-20T12:11:22.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6ecff9700 1 -- 172.21.15.44:0/2858484732 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4023930 con 0x7fb6d0005e20 2020-11-20T12:11:22.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f00619a0 con 0x7fb6d0005e20 2020-11-20T12:11:22.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.153+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f012b900 con 0x7fb6d0005e20 2020-11-20T12:11:22.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.155+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f00ab520 con 0x7fb6d0005e20 2020-11-20T12:11:22.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.155+0000 7fb6ecff9700 1 -- 172.21.15.44:0/2858484732 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb6e402c030 con 0x7fb6d0005e20 2020-11-20T12:11:22.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.155+0000 7fb6ecff9700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb6c8056d50 0x7fb6c8059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:22.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.155+0000 7fb6ecff9700 1 -- 172.21.15.44:0/2858484732 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb6e4072ff0 con 0x7fb6d0005e20 2020-11-20T12:11:22.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.155+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb6c8056d50 0x7fb6c8059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:22.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.156+0000 7fb6f4a67700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb6c8056d50 0x7fb6c8059210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb6e8005920 tx=0x7fb6e8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:22.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.159+0000 7fb6ecff9700 1 -- 172.21.15.44:0/2858484732 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb6e4023d60 con 0x7fb6d0005e20 2020-11-20T12:11:22.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.401+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb6f0061650 con 0x7fb6d0005e20 2020-11-20T12:11:22.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.402+0000 7fb6ecff9700 1 -- 172.21.15.44:0/2858484732 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7fb6e4023d60 con 0x7fb6d0005e20 2020-11-20T12:11:22.406 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":90,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb6c8056d50 msgr2=0x7fb6c8059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb6c8056d50 0x7fb6c8059210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb6e8005920 tx=0x7fb6e8009040).stop 2020-11-20T12:11:22.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 msgr2=0x7fb6f0066b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6f0066b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb6e40079a0 tx=0x7fb6e4023000).stop 2020-11-20T12:11:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 shutdown_connections 2020-11-20T12:11:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb6c8056d50 0x7fb6c8059210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6d00092f0 0x7fb6f012b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6d0003820 0x7fb6f0065400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 --2- 172.21.15.44:0/2858484732 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb6d0005e20 0x7fb6f0066b40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 >> 172.21.15.44:0/2858484732 conn(0x7fb6f0062690 msgr2=0x7fb6f0135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 shutdown_connections 2020-11-20T12:11:22.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:22.404+0000 7fb6f6ccb700 1 -- 172.21.15.44:0/2858484732 wait complete. 2020-11-20T12:11:25.419 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:25.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.555+0000 7f6d7f9e1700 1 Processor -- start 2020-11-20T12:11:25.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.555+0000 7f6d7f9e1700 1 -- start start 2020-11-20T12:11:25.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7f9e1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6d780aa020 0x7f6d780a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7f9e1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6d780aa400 0x7f6d780a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7f9e1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d7813ac60 0x7f6d780a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7f9e1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d780a9770 con 0x7f6d780aa400 2020-11-20T12:11:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7f9e1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d780a6da0 con 0x7f6d780aa020 2020-11-20T12:11:25.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7f9e1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d780a7040 con 0x7f6d7813ac60 2020-11-20T12:11:25.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6d780aa400 0x7f6d780a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:25.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6d780aa400 0x7f6d780a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39896/0 (socket says 172.21.15.44:39896) 2020-11-20T12:11:25.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 -- 172.21.15.44:0/2721249394 learned_addr learned my addr 172.21.15.44:0/2721249394 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:25.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7d77d700 1 --2- 172.21.15.44:0/2721249394 >> v2:172.21.15.79:3300/0 conn(0x7f6d780aa020 0x7f6d780a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:25.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7df7e700 1 --2- 172.21.15.44:0/2721249394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d7813ac60 0x7f6d780a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 -- 172.21.15.44:0/2721249394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d7813ac60 msgr2=0x7f6d780a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 --2- 172.21.15.44:0/2721249394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d7813ac60 0x7f6d780a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 -- 172.21.15.44:0/2721249394 >> v2:172.21.15.79:3300/0 conn(0x7f6d780aa020 msgr2=0x7f6d780a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 --2- 172.21.15.44:0/2721249394 >> v2:172.21.15.79:3300/0 conn(0x7f6d780aa020 0x7f6d780a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:25.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 -- 172.21.15.44:0/2721249394 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d780a6a90 con 0x7f6d780aa400 2020-11-20T12:11:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.556+0000 7f6d7cf7c700 1 --2- 172.21.15.44:0/2721249394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d780aa400 0x7f6d780a4fb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6d780adec0 tx=0x7f6d74009290).ready entity=mon.0 client_cookie=c7498882bc1f9a21 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d67fff700 1 -- 172.21.15.44:0/2721249394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400bb10 con 0x7f6d780aa400 2020-11-20T12:11:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d67fff700 1 -- 172.21.15.44:0/2721249394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7400bc70 con 0x7f6d780aa400 2020-11-20T12:11:25.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d67fff700 1 -- 172.21.15.44:0/2721249394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74011400 con 0x7f6d780aa400 2020-11-20T12:11:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/2721249394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d780aa400 msgr2=0x7f6d780a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/2721249394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d780aa400 0x7f6d780a4fb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6d780adec0 tx=0x7f6d74009290).stop 2020-11-20T12:11:25.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/2721249394 shutdown_connections 2020-11-20T12:11:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/2721249394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d7813ac60 0x7f6d780a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/2721249394 >> v2:172.21.15.79:3300/0 conn(0x7f6d780aa020 0x7f6d780a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:25.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/2721249394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d780aa400 0x7f6d780a4fb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:25.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/2721249394 >> 172.21.15.44:0/2721249394 conn(0x7f6d78062690 msgr2=0x7f6d780a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:25.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/2721249394 shutdown_connections 2020-11-20T12:11:25.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.557+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/2721249394 wait complete. 2020-11-20T12:11:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.558+0000 7f6d7f9e1700 1 Processor -- start 2020-11-20T12:11:25.575 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.558+0000 7f6d7f9e1700 1 -- start start 2020-11-20T12:11:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7f9e1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d780aa020 0x7f6d780a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:25.576 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7f9e1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d780aa400 0x7f6d7809e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7f9e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d7813ac60 0x7f6d7809eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7f9e1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7809f190 con 0x7f6d7813ac60 2020-11-20T12:11:25.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7f9e1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7809ddd0 con 0x7f6d780aa020 2020-11-20T12:11:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7f9e1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7809e070 con 0x7f6d780aa400 2020-11-20T12:11:25.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7df7e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d7813ac60 0x7f6d7809eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7df7e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d7813ac60 0x7f6d7809eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39906/0 (socket says 172.21.15.44:39906) 2020-11-20T12:11:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7df7e700 1 -- 172.21.15.44:0/1749663120 learned_addr learned my addr 172.21.15.44:0/1749663120 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:25.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7d77d700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d780aa020 0x7f6d780a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.559+0000 7f6d7cf7c700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d780aa400 0x7f6d7809e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7df7e700 1 -- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d780aa400 msgr2=0x7f6d7809e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7df7e700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d780aa400 0x7f6d7809e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7df7e700 1 -- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d780aa020 msgr2=0x7f6d780a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:25.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7df7e700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d780aa020 0x7f6d780a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7df7e700 1 -- 172.21.15.44:0/1749663120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7809e310 con 0x7f6d7813ac60 2020-11-20T12:11:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7df7e700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d7813ac60 0x7f6d7809eca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6d7000eba0 tx=0x7f6d70006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d657fa700 1 -- 172.21.15.44:0/1749663120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7000b810 con 0x7f6d7813ac60 2020-11-20T12:11:25.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d657fa700 1 -- 172.21.15.44:0/1749663120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7000b970 con 0x7f6d7813ac60 2020-11-20T12:11:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d78067590 con 0x7f6d7813ac60 2020-11-20T12:11:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d657fa700 1 -- 172.21.15.44:0/1749663120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7000bd10 con 0x7f6d7813ac60 2020-11-20T12:11:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.560+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7809d800 con 0x7f6d7813ac60 2020-11-20T12:11:25.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.561+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7809e4f0 con 0x7f6d7813ac60 2020-11-20T12:11:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.561+0000 7f6d657fa700 1 -- 172.21.15.44:0/1749663120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f6d70012600 con 0x7f6d7813ac60 2020-11-20T12:11:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.562+0000 7f6d657fa700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d50056d50 0x7f6d50059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.562+0000 7f6d657fa700 1 -- 172.21.15.44:0/1749663120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f6d70070350 con 0x7f6d7813ac60 2020-11-20T12:11:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.562+0000 7f6d7d77d700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d50056d50 0x7f6d50059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:25.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.562+0000 7f6d7d77d700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d50056d50 0x7f6d50059210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f6d68006fd0 tx=0x7f6d68006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:25.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.566+0000 7f6d657fa700 1 -- 172.21.15.44:0/1749663120 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6d7809e4f0 con 0x7f6d7813ac60 2020-11-20T12:11:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.808+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6d7809e4f0 con 0x7f6d7813ac60 2020-11-20T12:11:27.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.808+0000 7f6d657fa700 1 -- 172.21.15.44:0/1749663120 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1469 (secure 0 0 0) 0x7f6d7809e4f0 con 0x7f6d7813ac60 2020-11-20T12:11:27.101 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d50056d50 msgr2=0x7f6d50059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d50056d50 0x7f6d50059210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f6d68006fd0 tx=0x7f6d68006df0).stop 2020-11-20T12:11:27.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d7813ac60 msgr2=0x7f6d7809eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d7813ac60 0x7f6d7809eca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6d7000eba0 tx=0x7f6d70006df0).stop 2020-11-20T12:11:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 shutdown_connections 2020-11-20T12:11:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6d50056d50 0x7f6d50059210 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d7813ac60 0x7f6d7809eca0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6d780aa020 0x7f6d780a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 --2- 172.21.15.44:0/1749663120 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6d780aa400 0x7f6d7809e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 >> 172.21.15.44:0/1749663120 conn(0x7f6d78062690 msgr2=0x7f6d780a3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 shutdown_connections 2020-11-20T12:11:27.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:25.810+0000 7f6d7f9e1700 1 -- 172.21.15.44:0/1749663120 wait complete. 2020-11-20T12:11:27.106 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:11:27.106 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:11:27.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.248+0000 7fb787d05700 1 Processor -- start 2020-11-20T12:11:27.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.248+0000 7fb787d05700 1 -- start start 2020-11-20T12:11:27.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb787d05700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb7800aa400 0x7fb7800a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb787d05700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb78013ac60 0x7fb7800a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb787d05700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7800a9770 con 0x7fb7800aa020 2020-11-20T12:11:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb787d05700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800a56c0 con 0x7fb7800aa400 2020-11-20T12:11:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb787d05700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800a6da0 con 0x7fb78013ac60 2020-11-20T12:11:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7852a0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb7800aa400 0x7fb7800a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb785aa1700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb7800aa020 0x7fb7800a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43658/0 (socket says 172.21.15.44:43658) 2020-11-20T12:11:27.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb785aa1700 1 -- 172.21.15.44:0/1696070571 learned_addr learned my addr 172.21.15.44:0/1696070571 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:27.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7862a2700 1 --2- 172.21.15.44:0/1696070571 >> v2:172.21.15.79:3300/0 conn(0x7fb78013ac60 0x7fb7800a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7852a0700 1 -- 172.21.15.44:0/1696070571 >> v2:172.21.15.79:3300/0 conn(0x7fb78013ac60 msgr2=0x7fb7800a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:27.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1696070571 >> v2:172.21.15.79:3300/0 conn(0x7fb78013ac60 0x7fb7800a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7852a0700 1 -- 172.21.15.44:0/1696070571 >> v1:172.21.15.44:6789/0 conn(0x7fb7800aa020 legacy=0x7fb7800a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb784a9f700 1 -- 172.21.15.44:0/1696070571 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927261055 0 0) 0x7fb7700010d0 con 0x7fb7800aa020 2020-11-20T12:11:27.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7852a0700 1 -- 172.21.15.44:0/1696070571 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800a7040 con 0x7fb7800aa400 2020-11-20T12:11:27.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7862a2700 1 --2- 172.21.15.44:0/1696070571 >> v2:172.21.15.79:3300/0 conn(0x7fb78013ac60 0x7fb7800a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:11:27.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.249+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1696070571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa400 0x7fb7800a9280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb7800a8f70 tx=0x7fb77c0091d0).ready entity=mon.0 client_cookie=d59a976c5a2fb675 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb784a9f700 1 -- 172.21.15.44:0/1696070571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00bbc0 con 0x7fb7800aa400 2020-11-20T12:11:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb784a9f700 1 -- 172.21.15.44:0/1696070571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c00bd20 con 0x7fb7800aa400 2020-11-20T12:11:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb784a9f700 1 -- 172.21.15.44:0/1696070571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c011610 con 0x7fb7800aa400 2020-11-20T12:11:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 -- 172.21.15.44:0/1696070571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa400 msgr2=0x7fb7800a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 --2- 172.21.15.44:0/1696070571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa400 0x7fb7800a9280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb7800a8f70 tx=0x7fb77c0091d0).stop 2020-11-20T12:11:27.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 -- 172.21.15.44:0/1696070571 shutdown_connections 2020-11-20T12:11:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 --2- 172.21.15.44:0/1696070571 >> v2:172.21.15.79:3300/0 conn(0x7fb78013ac60 0x7fb7800a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 --2- 172.21.15.44:0/1696070571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa400 0x7fb7800a9280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 -- 172.21.15.44:0/1696070571 >> 172.21.15.44:0/1696070571 conn(0x7fb780062690 msgr2=0x7fb7800a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 -- 172.21.15.44:0/1696070571 shutdown_connections 2020-11-20T12:11:27.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.250+0000 7fb787d05700 1 -- 172.21.15.44:0/1696070571 wait complete. 2020-11-20T12:11:27.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.251+0000 7fb787d05700 1 Processor -- start 2020-11-20T12:11:27.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.251+0000 7fb787d05700 1 -- start start 2020-11-20T12:11:27.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb787d05700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa020 0x7fb78009d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb787d05700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7800aa400 0x7fb780060330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb787d05700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb78013ac60 0x7fb780147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb787d05700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78009de20 con 0x7fb7800aa020 2020-11-20T12:11:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb787d05700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780133b60 con 0x7fb7800aa400 2020-11-20T12:11:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb787d05700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb780067f60 con 0x7fb78013ac60 2020-11-20T12:11:27.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa020 0x7fb78009d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa020 0x7fb78009d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39918/0 (socket says 172.21.15.44:39918) 2020-11-20T12:11:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 -- 172.21.15.44:0/1572681836 learned_addr learned my addr 172.21.15.44:0/1572681836 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb7862a2700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb78013ac60 0x7fb780147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7800aa400 0x7fb780060330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb78013ac60 msgr2=0x7fb780147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb78013ac60 0x7fb780147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7800aa400 msgr2=0x7fb780060330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7800aa400 0x7fb780060330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 -- 172.21.15.44:0/1572681836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78013a970 con 0x7fb7800aa020 2020-11-20T12:11:27.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7800aa400 0x7fb780060330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:11:27.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.252+0000 7fb785aa1700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa020 0x7fb78009d780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb77000c760 tx=0x7fb770007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.253+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700072e0 con 0x7fb7800aa020 2020-11-20T12:11:27.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.253+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb770007440 con 0x7fb7800aa020 2020-11-20T12:11:27.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.253+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb780148340 con 0x7fb7800aa020 2020-11-20T12:11:27.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.253+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770005800 con 0x7fb7800aa020 2020-11-20T12:11:27.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.253+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7801341c0 con 0x7fb7800aa020 2020-11-20T12:11:27.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb787d05700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb780132cf0 0x7fb7801330d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78009e020 con 0x7fb780132cf0 2020-11-20T12:11:27.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb770010070 con 0x7fb7800aa020 2020-11-20T12:11:27.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb780132cf0 0x7fb7801330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb76dffb700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb758056d20 0x7fb7580591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb7852a0700 1 -- 172.21.15.44:0/1572681836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb78009dc70 con 0x7fb780132cf0 2020-11-20T12:11:27.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb77006f6c0 con 0x7fb7800aa020 2020-11-20T12:11:27.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb780132cf0 0x7fb7801330d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb780135130 tx=0x7fb77c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb7862a2700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb758056d20 0x7fb7580591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.254+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c01d660 con 0x7fb780132cf0 2020-11-20T12:11:27.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.255+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb77c01d7c0 con 0x7fb780132cf0 2020-11-20T12:11:27.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.255+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb780132cf0 msgr2=0x7fb7801330d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.255+0000 7fb76dffb700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb780132cf0 0x7fb7801330d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fb780135130 tx=0x7fb77c016000).stop 2020-11-20T12:11:27.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.255+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb780132cf0 msgr2=0x7fb7801330d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:27.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.255+0000 7fb76dffb700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb780132cf0 0x7fb7801330d0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0x7fb780135130 tx=0x7fb77c016000).stop 2020-11-20T12:11:27.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.255+0000 7fb7862a2700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb758056d20 0x7fb7580591e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fb77800b7b0 tx=0x7fb778008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb787d05700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb78014b420 0x7fb78014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800a9770 con 0x7fb78014b420 2020-11-20T12:11:27.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb78014b420 0x7fb78014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb7852a0700 1 -- 172.21.15.44:0/1572681836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb780138c00 con 0x7fb78014b420 2020-11-20T12:11:27.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb7852a0700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb78014b420 0x7fb78014b800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb77c009210 tx=0x7fb77c01e000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c025600 con 0x7fb78014b420 2020-11-20T12:11:27.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7fb77c01d450 con 0x7fb78014b420 2020-11-20T12:11:27.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb78014b420 msgr2=0x7fb78014b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb76dffb700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb78014b420 0x7fb78014b800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb77c009210 tx=0x7fb77c01e000).stop 2020-11-20T12:11:27.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb76dffb700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb78014b420 msgr2=0x7fb78014b800 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:27.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.263+0000 7fb76dffb700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb78014b420 0x7fb78014b800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb758056d20 msgr2=0x7fb7580591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb758056d20 0x7fb7580591e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fb77800b7b0 tx=0x7fb778008040).stop 2020-11-20T12:11:27.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa020 msgr2=0x7fb78009d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 --2- 172.21.15.44:0/1572681836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7800aa020 0x7fb78009d780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb77000c760 tx=0x7fb770007ce0).stop 2020-11-20T12:11:27.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb7862a2700 1 -- 172.21.15.44:0/1572681836 reap_dead start 2020-11-20T12:11:27.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb7862a2700 1 -- 172.21.15.44:0/1572681836 reap_dead start 2020-11-20T12:11:27.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 shutdown_connections 2020-11-20T12:11:27.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 >> 172.21.15.44:0/1572681836 conn(0x7fb780062690 msgr2=0x7fb7800a4130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 shutdown_connections 2020-11-20T12:11:27.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.264+0000 7fb787d05700 1 -- 172.21.15.44:0/1572681836 wait complete. 2020-11-20T12:11:27.277 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:27.277 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:11:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 94, 2020-11-20T12:11:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:27.278 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:11:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:11:27.279 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.281 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:27.282 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.283 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.283 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.283 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.283 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.284 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.284 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.284 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.284 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:27.285 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:27.285 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:27.285 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:27.285 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:27.285 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:27.286 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:27.286 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:27.286 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:27.286 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:27.286 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:27.287 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:27.287 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:27.287 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:27.287 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:27.287 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.288 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.288 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.288 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.289 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.289 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.289 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.289 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:27.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:27.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.290 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:27.290 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:27.291 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.291 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.291 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:27.292 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.292 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:27.292 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.293 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.293 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.293 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:27.294 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:27.294 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.294 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.294 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.295 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:27.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:27.295 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.295 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.296 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:27.296 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.297 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:27.297 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.298 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.298 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:27.298 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:27.298 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.299 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.299 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.299 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:27.300 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:27.300 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.300 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.301 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:27.301 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.302 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:27.302 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.302 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.303 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:27.303 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:27.303 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.303 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.303 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.304 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.304 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.304 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:27.304 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:27.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.305 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.305 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.305 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:27.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.306 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:27.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.306 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.306 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.307 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:27.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.307 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.307 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:27.308 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:27.309 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:11:27.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.434+0000 7f4250425700 1 Processor -- start 2020-11-20T12:11:27.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.434+0000 7f4250425700 1 -- start start 2020-11-20T12:11:27.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f4250425700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4248140810 0x7f4248142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f4250425700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480a6260 con 0x7f424813a990 2020-11-20T12:11:27.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f4250425700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480a6490 con 0x7f4248060330 2020-11-20T12:11:27.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f4250425700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42480a52e0 con 0x7f4248140810 2020-11-20T12:11:27.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f424e9c2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4248140810 0x7f4248142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f424d9c0700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f424813a990 0x7f424813ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43676/0 (socket says 172.21.15.44:43676) 2020-11-20T12:11:27.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f424d9c0700 1 -- 172.21.15.44:0/1242308692 learned_addr learned my addr 172.21.15.44:0/1242308692 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:27.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.435+0000 7f424e9c2700 1 --2- 172.21.15.44:0/1242308692 >> v2:172.21.15.44:3300/0 conn(0x7f4248140810 0x7f4248142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39932/0 (socket says 172.21.15.44:39932) 2020-11-20T12:11:27.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424e9c2700 1 -- 172.21.15.44:0/1242308692 >> v1:172.21.15.79:6789/0 conn(0x7f4248060330 legacy=0x7f42480a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:27.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424e9c2700 1 -- 172.21.15.44:0/1242308692 >> v1:172.21.15.44:6789/0 conn(0x7f424813a990 legacy=0x7f424813ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424e9c2700 1 -- 172.21.15.44:0/1242308692 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480a6730 con 0x7f4248140810 2020-11-20T12:11:27.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424d1bf700 1 -- 172.21.15.44:0/1242308692 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750307407 0 0) 0x7f42480a6260 con 0x7f424813a990 2020-11-20T12:11:27.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424e9c2700 1 --2- 172.21.15.44:0/1242308692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248140810 0x7f4248142c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f424000b9a0 tx=0x7f4240006df0).ready entity=mon.0 client_cookie=fa02aba2a29dd8b4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424d1bf700 1 -- 172.21.15.44:0/1242308692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400097b0 con 0x7f4248140810 2020-11-20T12:11:27.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424d1bf700 1 -- 172.21.15.44:0/1242308692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240009910 con 0x7f4248140810 2020-11-20T12:11:27.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f424d1bf700 1 -- 172.21.15.44:0/1242308692 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240009c90 con 0x7f4248140810 2020-11-20T12:11:27.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f4250425700 1 -- 172.21.15.44:0/1242308692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248140810 msgr2=0x7f4248142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f4250425700 1 --2- 172.21.15.44:0/1242308692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248140810 0x7f4248142c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f424000b9a0 tx=0x7f4240006df0).stop 2020-11-20T12:11:27.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f4250425700 1 -- 172.21.15.44:0/1242308692 shutdown_connections 2020-11-20T12:11:27.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f4250425700 1 --2- 172.21.15.44:0/1242308692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248140810 0x7f4248142c00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.436+0000 7f4250425700 1 -- 172.21.15.44:0/1242308692 >> 172.21.15.44:0/1242308692 conn(0x7f4248062690 msgr2=0x7f42480a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.437+0000 7f4250425700 1 -- 172.21.15.44:0/1242308692 shutdown_connections 2020-11-20T12:11:27.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.437+0000 7f4250425700 1 -- 172.21.15.44:0/1242308692 wait complete. 2020-11-20T12:11:27.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.437+0000 7f4250425700 1 Processor -- start 2020-11-20T12:11:27.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f4250425700 1 -- start start 2020-11-20T12:11:27.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f4250425700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248060330 0x7f4248065260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f4250425700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424813a990 0x7f4248134130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f4250425700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4248140810 0x7f4248128460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f4250425700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248065750 con 0x7f4248060330 2020-11-20T12:11:27.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f4250425700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248134ca0 con 0x7f424813a990 2020-11-20T12:11:27.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f4250425700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4248067810 con 0x7f4248140810 2020-11-20T12:11:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248060330 0x7f4248065260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248060330 0x7f4248065260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39934/0 (socket says 172.21.15.44:39934) 2020-11-20T12:11:27.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 -- 172.21.15.44:0/1403084183 learned_addr learned my addr 172.21.15.44:0/1403084183 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:27.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e9c2700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4248140810 0x7f4248128460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424d9c0700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424813a990 0x7f4248134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4248140810 msgr2=0x7f4248128460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4248140810 0x7f4248128460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424813a990 msgr2=0x7f4248134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424813a990 0x7f4248134130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.438+0000 7f424e1c1700 1 -- 172.21.15.44:0/1403084183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248134e90 con 0x7f4248060330 2020-11-20T12:11:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.439+0000 7f424e1c1700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248060330 0x7f4248065260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f423800c760 tx=0x7f4238007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.439+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423800e560 con 0x7f4248060330 2020-11-20T12:11:27.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.439+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423800e6e0 con 0x7f4248060330 2020-11-20T12:11:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.439+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4248135130 con 0x7f4248060330 2020-11-20T12:11:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.439+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423800eaa0 con 0x7f4248060330 2020-11-20T12:11:27.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.439+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42481347e0 con 0x7f4248060330 2020-11-20T12:11:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.439+0000 7f4250425700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4248128a10 0x7f424812a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42481355d0 con 0x7f4248128a10 2020-11-20T12:11:27.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f424d9c0700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4248128a10 0x7f424812a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4238019030 con 0x7f4248060330 2020-11-20T12:11:27.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f424d9c0700 1 -- 172.21.15.44:0/1403084183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4248140df0 con 0x7f4248128a10 2020-11-20T12:11:27.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f42367fc700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4220056d20 0x7f42200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f423806f620 con 0x7f4248060330 2020-11-20T12:11:27.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f424d9c0700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4248128a10 0x7f424812a450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f424400d9d0 tx=0x7f4244009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f424e9c2700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4220056d20 0x7f42200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.440+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244005b80 con 0x7f4248128a10 2020-11-20T12:11:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.441+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4244005ce0 con 0x7f4248128a10 2020-11-20T12:11:27.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.441+0000 7f424e9c2700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4220056d20 0x7f42200591e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f42480a6180 tx=0x7f4240015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.441+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4248128a10 msgr2=0x7f424812a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.441+0000 7f42367fc700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4248128a10 0x7f424812a450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f424400d9d0 tx=0x7f4244009040).stop 2020-11-20T12:11:27.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.441+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4248128a10 msgr2=0x7f424812a450 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:27.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.441+0000 7f42367fc700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4248128a10 0x7f424812a450 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0x7f424400d9d0 tx=0x7f4244009040).stop 2020-11-20T12:11:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.448+0000 7f4250425700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424812a940 0x7f4248129320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.449+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248061650 con 0x7f424812a940 2020-11-20T12:11:27.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.449+0000 7f424d9c0700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424812a940 0x7f4248129320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.449+0000 7f424d9c0700 1 -- 172.21.15.44:0/1403084183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4248060710 con 0x7f424812a940 2020-11-20T12:11:27.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.449+0000 7f424d9c0700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424812a940 0x7f4248129320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f424400d9d0 tx=0x7f4244009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424400be90 con 0x7f424812a940 2020-11-20T12:11:27.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4435 (secure 0 0 0) 0x7f4244005970 con 0x7f424812a940 2020-11-20T12:11:27.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424812a940 msgr2=0x7f4248129320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f42367fc700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424812a940 0x7f4248129320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f424400d9d0 tx=0x7f4244009040).stop 2020-11-20T12:11:27.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f42367fc700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424812a940 msgr2=0x7f4248129320 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:27.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f42367fc700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f424812a940 0x7f4248129320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4220056d20 msgr2=0x7f42200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4220056d20 0x7f42200591e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f42480a6180 tx=0x7f4240015000).stop 2020-11-20T12:11:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248060330 msgr2=0x7f4248065260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 --2- 172.21.15.44:0/1403084183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248060330 0x7f4248065260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f423800c760 tx=0x7f4238007c30).stop 2020-11-20T12:11:27.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f424e9c2700 1 -- 172.21.15.44:0/1403084183 reap_dead start 2020-11-20T12:11:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f424e9c2700 1 -- 172.21.15.44:0/1403084183 reap_dead start 2020-11-20T12:11:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 shutdown_connections 2020-11-20T12:11:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 >> 172.21.15.44:0/1403084183 conn(0x7f4248062690 msgr2=0x7f42480a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 shutdown_connections 2020-11-20T12:11:27.475 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.450+0000 7f4250425700 1 -- 172.21.15.44:0/1403084183 wait complete. 2020-11-20T12:11:27.475 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:11:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 94, 2020-11-20T12:11:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:11:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:11:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:11:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:27.513 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:27.514 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:11:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.637+0000 7f80afe63700 1 Processor -- start 2020-11-20T12:11:27.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.638+0000 7f80afe63700 1 -- start start 2020-11-20T12:11:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.638+0000 7f80afe63700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa020 0x7f80a80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.638+0000 7f80afe63700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80a813ac60 0x7f80a80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.638+0000 7f80afe63700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a80a9770 con 0x7f80a80aa400 2020-11-20T12:11:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.638+0000 7f80afe63700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a80a56c0 con 0x7f80a813ac60 2020-11-20T12:11:27.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.638+0000 7f80afe63700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80a6da0 con 0x7f80a80aa020 2020-11-20T12:11:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80ad3fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f80a80aa400 0x7f80a80a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43694/0 (socket says 172.21.15.44:43694) 2020-11-20T12:11:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80ad3fe700 1 -- 172.21.15.44:0/1100858545 learned_addr learned my addr 172.21.15.44:0/1100858545 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80ae400700 1 --2- 172.21.15.44:0/1100858545 >> v2:172.21.15.79:3300/0 conn(0x7f80a813ac60 0x7f80a80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80adbff700 1 --2- 172.21.15.44:0/1100858545 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa020 0x7f80a80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80acbfd700 1 -- 172.21.15.44:0/1100858545 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156606086 0 0) 0x7f809c0010d0 con 0x7f80a80aa400 2020-11-20T12:11:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80acbfd700 1 -- 172.21.15.44:0/1100858545 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f808c003240 con 0x7f80a80aa400 2020-11-20T12:11:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80adbff700 1 -- 172.21.15.44:0/1100858545 >> v2:172.21.15.79:3300/0 conn(0x7f80a813ac60 msgr2=0x7f80a80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80adbff700 1 --2- 172.21.15.44:0/1100858545 >> v2:172.21.15.79:3300/0 conn(0x7f80a813ac60 0x7f80a80a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80adbff700 1 -- 172.21.15.44:0/1100858545 >> v1:172.21.15.44:6789/0 conn(0x7f80a80aa400 legacy=0x7f80a80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80adbff700 1 -- 172.21.15.44:0/1100858545 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80a7040 con 0x7f80a80aa020 2020-11-20T12:11:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80ae400700 1 --2- 172.21.15.44:0/1100858545 >> v2:172.21.15.79:3300/0 conn(0x7f80a813ac60 0x7f80a80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:11:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80adbff700 1 --2- 172.21.15.44:0/1100858545 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa020 0x7f80a80a9280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f80a400b9a0 tx=0x7f80a4006df0).ready entity=mon.2 client_cookie=68ff0c13dce96dba server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80acbfd700 1 -- 172.21.15.44:0/1100858545 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4009680 con 0x7f80a80aa020 2020-11-20T12:11:27.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80acbfd700 1 -- 172.21.15.44:0/1100858545 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a40097e0 con 0x7f80a80aa020 2020-11-20T12:11:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.639+0000 7f80acbfd700 1 -- 172.21.15.44:0/1100858545 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4009b60 con 0x7f80a80aa020 2020-11-20T12:11:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 -- 172.21.15.44:0/1100858545 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa020 msgr2=0x7f80a80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 --2- 172.21.15.44:0/1100858545 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa020 0x7f80a80a9280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f80a400b9a0 tx=0x7f80a4006df0).stop 2020-11-20T12:11:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 -- 172.21.15.44:0/1100858545 shutdown_connections 2020-11-20T12:11:27.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 --2- 172.21.15.44:0/1100858545 >> v2:172.21.15.79:3300/0 conn(0x7f80a813ac60 0x7f80a80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 --2- 172.21.15.44:0/1100858545 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa020 0x7f80a80a9280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 -- 172.21.15.44:0/1100858545 >> 172.21.15.44:0/1100858545 conn(0x7f80a8062690 msgr2=0x7f80a80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 -- 172.21.15.44:0/1100858545 shutdown_connections 2020-11-20T12:11:27.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.640+0000 7f80afe63700 1 -- 172.21.15.44:0/1100858545 wait complete. 2020-11-20T12:11:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.641+0000 7f80afe63700 1 Processor -- start 2020-11-20T12:11:27.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.641+0000 7f80afe63700 1 -- start start 2020-11-20T12:11:27.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80a80aa020 0x7f80a80aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa400 0x7f80a80ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80a813ac60 0x7f80a8060570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80ac2a0 con 0x7f80a813ac60 2020-11-20T12:11:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80ab150 con 0x7f80a80aa020 2020-11-20T12:11:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8135300 con 0x7f80a80aa400 2020-11-20T12:11:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80a813ac60 0x7f80a8060570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80a813ac60 0x7f80a8060570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39956/0 (socket says 172.21.15.44:39956) 2020-11-20T12:11:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 -- 172.21.15.44:0/2307835915 learned_addr learned my addr 172.21.15.44:0/2307835915 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ad3fe700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa400 0x7f80a80ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80adbff700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80a80aa020 0x7f80a80aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa400 msgr2=0x7f80a80ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a80aa400 0x7f80a80ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80a80aa020 msgr2=0x7f80a80aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80a80aa020 0x7f80a80aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 -- 172.21.15.44:0/2307835915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a8133e30 con 0x7f80a813ac60 2020-11-20T12:11:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80ae400700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80a813ac60 0x7f80a8060570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f80a000ef50 tx=0x7f80a00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000cc10 con 0x7f80a813ac60 2020-11-20T12:11:27.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000cd70 con 0x7f80a813ac60 2020-11-20T12:11:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a80ab370 con 0x7f80a813ac60 2020-11-20T12:11:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0012620 con 0x7f80a813ac60 2020-11-20T12:11:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.642+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a8134530 con 0x7f80a813ac60 2020-11-20T12:11:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.643+0000 7f80afe63700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8133fc0 0x7f80a8131120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.643+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8061650 con 0x7f80a8133fc0 2020-11-20T12:11:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.643+0000 7f80adbff700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8133fc0 0x7f80a8131120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f80adbff700 1 -- 172.21.15.44:0/2307835915 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80a80446d0 con 0x7f80a8133fc0 2020-11-20T12:11:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f80a0022070 con 0x7f80a813ac60 2020-11-20T12:11:27.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f80adbff700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8133fc0 0x7f80a8131120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f80a400b9a0 tx=0x7f80a4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f807c056d50 0x7f807c059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f80a0070950 con 0x7f80a813ac60 2020-11-20T12:11:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4023030 con 0x7f80a8133fc0 2020-11-20T12:11:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f80ad3fe700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f807c056d50 0x7f807c059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f80a4011a70 con 0x7f80a8133fc0 2020-11-20T12:11:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8133fc0 msgr2=0x7f80a8131120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8133fc0 0x7f80a8131120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f80a400b9a0 tx=0x7f80a4015000).stop 2020-11-20T12:11:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8133fc0 msgr2=0x7f80a8131120 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.644+0000 7f809a7fc700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8133fc0 0x7f80a8131120 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0x7f80a400b9a0 tx=0x7f80a4015000).stop 2020-11-20T12:11:27.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.645+0000 7f80ad3fe700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f807c056d50 0x7f807c059210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f809c001230 tx=0x7f809c0078c0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.652+0000 7f80afe63700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8131610 0x7f80a80672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.652+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8131a20 con 0x7f80a8131610 2020-11-20T12:11:27.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.652+0000 7f80adbff700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8131610 0x7f80a80672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f80adbff700 1 -- 172.21.15.44:0/2307835915 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f80a8069560 con 0x7f80a8131610 2020-11-20T12:11:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f80adbff700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8131610 0x7f80a80672b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f80a4006e30 tx=0x7f80a4021070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4033060 con 0x7f80a8131610 2020-11-20T12:11:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4775 (secure 0 0 0) 0x7f80a4011860 con 0x7f80a8131610 2020-11-20T12:11:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8131610 msgr2=0x7f80a80672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f809a7fc700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8131610 0x7f80a80672b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f80a4006e30 tx=0x7f80a4021070).stop 2020-11-20T12:11:27.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f809a7fc700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8131610 msgr2=0x7f80a80672b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:27.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.653+0000 7f809a7fc700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f80a8131610 0x7f80a80672b0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0x7f80a4006e30 tx=0x7f80a4021070).stop 2020-11-20T12:11:27.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f807c056d50 msgr2=0x7f807c059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f807c056d50 0x7f807c059210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f809c001230 tx=0x7f809c0078c0).stop 2020-11-20T12:11:27.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80a813ac60 msgr2=0x7f80a8060570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 --2- 172.21.15.44:0/2307835915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80a813ac60 0x7f80a8060570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f80a000ef50 tx=0x7f80a00091d0).stop 2020-11-20T12:11:27.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80ae400700 1 -- 172.21.15.44:0/2307835915 reap_dead start 2020-11-20T12:11:27.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80ae400700 1 -- 172.21.15.44:0/2307835915 reap_dead start 2020-11-20T12:11:27.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 shutdown_connections 2020-11-20T12:11:27.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 >> 172.21.15.44:0/2307835915 conn(0x7f80a8062690 msgr2=0x7f80a8139180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 shutdown_connections 2020-11-20T12:11:27.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.654+0000 7f80afe63700 1 -- 172.21.15.44:0/2307835915 wait complete. 2020-11-20T12:11:27.667 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:27.667 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:27.668 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:27.668 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:11:27.668 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 94, 2020-11-20T12:11:27.668 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:27.668 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:27.669 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:11:27.669 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:27.669 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.669 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:11:27.669 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:27.670 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:27.670 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:27.670 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.670 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.671 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.671 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.671 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.671 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.671 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.672 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.672 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:27.672 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:27.672 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.672 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:11:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:11:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:11:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:11:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:11:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:27.702 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:27.703 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:11:27.703 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:11:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.825+0000 7f5946154700 1 Processor -- start 2020-11-20T12:11:27.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.825+0000 7f5946154700 1 -- start start 2020-11-20T12:11:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f5946154700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa020 0x7f59400a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f5946154700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f59400aa400 0x7f59400a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f5946154700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400a9770 con 0x7f594013ac60 2020-11-20T12:11:27.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f5946154700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400a6da0 con 0x7f59400aa400 2020-11-20T12:11:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f5946154700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59400a7040 con 0x7f59400aa020 2020-11-20T12:11:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f59400aa400 0x7f59400a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f59400aa400 0x7f59400a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39968/0 (socket says 172.21.15.44:39968) 2020-11-20T12:11:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 -- 172.21.15.44:0/3995790696 learned_addr learned my addr 172.21.15.44:0/3995790696 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:27.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593f7fe700 1 --2- 172.21.15.44:0/3995790696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa020 0x7f59400a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 -- 172.21.15.44:0/3995790696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa020 msgr2=0x7f59400a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 --2- 172.21.15.44:0/3995790696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa020 0x7f59400a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 -- 172.21.15.44:0/3995790696 >> v1:172.21.15.79:6789/0 conn(0x7f594013ac60 legacy=0x7f59400a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 -- 172.21.15.44:0/3995790696 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400a6a90 con 0x7f59400aa400 2020-11-20T12:11:27.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593e7fc700 1 -- 172.21.15.44:0/3995790696 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012422586 0 0) 0x7f5938001450 con 0x7f594013ac60 2020-11-20T12:11:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.826+0000 7f593effd700 1 --2- 172.21.15.44:0/3995790696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa400 0x7f59400a4fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5934006fd0 tx=0x7f5934006eb0).ready entity=mon.0 client_cookie=745c32aad095e608 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f593e7fc700 1 -- 172.21.15.44:0/3995790696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934009810 con 0x7f59400aa400 2020-11-20T12:11:27.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f593e7fc700 1 -- 172.21.15.44:0/3995790696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934009970 con 0x7f59400aa400 2020-11-20T12:11:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f593e7fc700 1 -- 172.21.15.44:0/3995790696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934009cf0 con 0x7f59400aa400 2020-11-20T12:11:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 -- 172.21.15.44:0/3995790696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa400 msgr2=0x7f59400a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 --2- 172.21.15.44:0/3995790696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa400 0x7f59400a4fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5934006fd0 tx=0x7f5934006eb0).stop 2020-11-20T12:11:27.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 -- 172.21.15.44:0/3995790696 shutdown_connections 2020-11-20T12:11:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 --2- 172.21.15.44:0/3995790696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa020 0x7f59400a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 --2- 172.21.15.44:0/3995790696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa400 0x7f59400a4fb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 -- 172.21.15.44:0/3995790696 >> 172.21.15.44:0/3995790696 conn(0x7f5940062690 msgr2=0x7f59400a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:27.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 -- 172.21.15.44:0/3995790696 shutdown_connections 2020-11-20T12:11:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.827+0000 7f5946154700 1 -- 172.21.15.44:0/3995790696 wait complete. 2020-11-20T12:11:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.828+0000 7f5946154700 1 Processor -- start 2020-11-20T12:11:27.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.828+0000 7f5946154700 1 -- start start 2020-11-20T12:11:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f5946154700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa020 0x7f59400a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f5946154700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa400 0x7f5940066fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f5946154700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f594013ac60 0x7f59400ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f5946154700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594014ae80 con 0x7f59400aa020 2020-11-20T12:11:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f5946154700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59400a3280 con 0x7f594013ac60 2020-11-20T12:11:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f5946154700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5940065c90 con 0x7f59400aa400 2020-11-20T12:11:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa020 0x7f59400a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa020 0x7f59400a2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39970/0 (socket says 172.21.15.44:39970) 2020-11-20T12:11:27.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 -- 172.21.15.44:0/4264384585 learned_addr learned my addr 172.21.15.44:0/4264384585 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593effd700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa400 0x7f5940066fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593ffff700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f594013ac60 0x7f59400ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 -- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa400 msgr2=0x7f5940066fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa400 0x7f5940066fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 -- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f594013ac60 msgr2=0x7f59400ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f594013ac60 0x7f59400ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 -- 172.21.15.44:0/4264384585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940065f30 con 0x7f59400aa020 2020-11-20T12:11:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593ffff700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f594013ac60 0x7f59400ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:11:27.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593effd700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa400 0x7f5940066fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:11:27.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f593f7fe700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa020 0x7f59400a2890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f593000eba0 tx=0x7f5930006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f591f7fe700 1 -- 172.21.15.44:0/4264384585 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000ca60 con 0x7f59400aa020 2020-11-20T12:11:27.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f591f7fe700 1 -- 172.21.15.44:0/4264384585 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593000cbc0 con 0x7f59400aa020 2020-11-20T12:11:27.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f591f7fe700 1 -- 172.21.15.44:0/4264384585 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300123f0 con 0x7f59400aa020 2020-11-20T12:11:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.829+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59400661d0 con 0x7f59400aa020 2020-11-20T12:11:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.830+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59400a2d80 con 0x7f59400aa020 2020-11-20T12:11:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.830+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5940061650 con 0x7f59400aa020 2020-11-20T12:11:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.831+0000 7f591f7fe700 1 -- 172.21.15.44:0/4264384585 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f59300127a0 con 0x7f59400aa020 2020-11-20T12:11:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.831+0000 7f591f7fe700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5918056d20 0x7f59180591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.831+0000 7f591f7fe700 1 -- 172.21.15.44:0/4264384585 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f59300705d0 con 0x7f59400aa020 2020-11-20T12:11:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.831+0000 7f593effd700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5918056d20 0x7f59180591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.832+0000 7f593effd700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5918056d20 0x7f59180591e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f5940068720 tx=0x7f5934015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:27.835+0000 7f591f7fe700 1 -- 172.21.15.44:0/4264384585 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f59300458a0 con 0x7f59400aa020 2020-11-20T12:11:28.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.073+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f59400446d0 con 0x7f59400aa020 2020-11-20T12:11:28.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.074+0000 7f591f7fe700 1 -- 172.21.15.44:0/4264384585 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f5930048ec0 con 0x7f59400aa020 2020-11-20T12:11:28.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5918056d20 msgr2=0x7f59180591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:28.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5918056d20 0x7f59180591e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f5940068720 tx=0x7f5934015000).stop 2020-11-20T12:11:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa020 msgr2=0x7f59400a2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa020 0x7f59400a2890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f593000eba0 tx=0x7f5930006df0).stop 2020-11-20T12:11:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 shutdown_connections 2020-11-20T12:11:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5918056d20 0x7f59180591e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:28.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f594013ac60 0x7f59400ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:28.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59400aa020 0x7f59400a2890 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:28.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 --2- 172.21.15.44:0/4264384585 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f59400aa400 0x7f5940066fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:28.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 >> 172.21.15.44:0/4264384585 conn(0x7f5940062690 msgr2=0x7f5940134900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:28.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 shutdown_connections 2020-11-20T12:11:28.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:28.076+0000 7f5946154700 1 -- 172.21.15.44:0/4264384585 wait complete. 2020-11-20T12:11:28.097 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:11:37.388 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:11:37.398 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:11:38.098 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:11:38.099 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:38.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.234+0000 7fee7cab2700 1 Processor -- start 2020-11-20T12:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.234+0000 7fee7cab2700 1 -- start start 2020-11-20T12:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.234+0000 7fee7cab2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fee780aa020 0x7fee780a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7cab2700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780a9770 con 0x7fee7813ac60 2020-11-20T12:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7cab2700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780a6da0 con 0x7fee780aa400 2020-11-20T12:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7cab2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780a56c0 con 0x7fee780aa020 2020-11-20T12:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7659c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fee780aa020 0x7fee780a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee76d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fee7813ac60 0x7fee780a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43728/0 (socket says 172.21.15.44:43728) 2020-11-20T12:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee76d9d700 1 -- 172.21.15.44:0/3142059560 learned_addr learned my addr 172.21.15.44:0/3142059560 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7659c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fee780aa020 0x7fee780a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39978/0 (socket says 172.21.15.44:39978) 2020-11-20T12:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7659c700 1 -- 172.21.15.44:0/3142059560 >> v1:172.21.15.79:6789/0 conn(0x7fee780aa400 legacy=0x7fee780a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7659c700 1 -- 172.21.15.44:0/3142059560 >> v1:172.21.15.44:6789/0 conn(0x7fee7813ac60 legacy=0x7fee780a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7559a700 1 -- 172.21.15.44:0/3142059560 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477569378 0 0) 0x7fee680010d0 con 0x7fee7813ac60 2020-11-20T12:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7659c700 1 -- 172.21.15.44:0/3142059560 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780a7040 con 0x7fee780aa020 2020-11-20T12:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7659c700 1 --2- 172.21.15.44:0/3142059560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee780aa020 0x7fee780a9280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fee70009c90 tx=0x7fee700091d0).ready entity=mon.0 client_cookie=ee84ea77f66af57c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7559a700 1 -- 172.21.15.44:0/3142059560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7000bbc0 con 0x7fee780aa020 2020-11-20T12:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7559a700 1 -- 172.21.15.44:0/3142059560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7000bd20 con 0x7fee780aa020 2020-11-20T12:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.235+0000 7fee7559a700 1 -- 172.21.15.44:0/3142059560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70011610 con 0x7fee780aa020 2020-11-20T12:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.236+0000 7fee7cab2700 1 -- 172.21.15.44:0/3142059560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee780aa020 msgr2=0x7fee780a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.236+0000 7fee7cab2700 1 --2- 172.21.15.44:0/3142059560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee780aa020 0x7fee780a9280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fee70009c90 tx=0x7fee700091d0).stop 2020-11-20T12:11:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.236+0000 7fee7cab2700 1 -- 172.21.15.44:0/3142059560 shutdown_connections 2020-11-20T12:11:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.236+0000 7fee7cab2700 1 --2- 172.21.15.44:0/3142059560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee780aa020 0x7fee780a9280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.236+0000 7fee7cab2700 1 -- 172.21.15.44:0/3142059560 >> 172.21.15.44:0/3142059560 conn(0x7fee78062690 msgr2=0x7fee780a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.236+0000 7fee7cab2700 1 -- 172.21.15.44:0/3142059560 shutdown_connections 2020-11-20T12:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.236+0000 7fee7cab2700 1 -- 172.21.15.44:0/3142059560 wait complete. 2020-11-20T12:11:38.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.237+0000 7fee7cab2700 1 Processor -- start 2020-11-20T12:11:38.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.237+0000 7fee7cab2700 1 -- start start 2020-11-20T12:11:38.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7cab2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee780aa020 0x7fee78065f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7cab2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee780aa400 0x7fee7813a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7cab2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee7813ac60 0x7fee780667f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7cab2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780ade30 con 0x7fee7813ac60 2020-11-20T12:11:38.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7cab2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780aac60 con 0x7fee780aa020 2020-11-20T12:11:38.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7cab2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee780aae90 con 0x7fee780aa400 2020-11-20T12:11:38.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee7813ac60 0x7fee780667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee780aa020 0x7fee78065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee7813ac60 0x7fee780667f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39988/0 (socket says 172.21.15.44:39988) 2020-11-20T12:11:38.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee780aa020 0x7fee78065f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60266/0 (socket says 172.21.15.44:60266) 2020-11-20T12:11:38.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 -- 172.21.15.44:0/1982059487 learned_addr learned my addr 172.21.15.44:0/1982059487 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:38.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee75d9b700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee780aa400 0x7fee7813a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 -- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee780aa400 msgr2=0x7fee7813a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee780aa400 0x7fee7813a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 -- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee780aa020 msgr2=0x7fee78065f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee780aa020 0x7fee78065f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 -- 172.21.15.44:0/1982059487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780ab130 con 0x7fee7813ac60 2020-11-20T12:11:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee76d9d700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee7813ac60 0x7fee780667f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fee6800c760 tx=0x7fee68007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee6e7fc700 1 -- 172.21.15.44:0/1982059487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680072e0 con 0x7fee7813ac60 2020-11-20T12:11:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee6e7fc700 1 -- 172.21.15.44:0/1982059487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee68007440 con 0x7fee7813ac60 2020-11-20T12:11:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.238+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee780680b0 con 0x7fee7813ac60 2020-11-20T12:11:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.239+0000 7fee6e7fc700 1 -- 172.21.15.44:0/1982059487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68005820 con 0x7fee7813ac60 2020-11-20T12:11:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.239+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7812d310 con 0x7fee7813ac60 2020-11-20T12:11:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.239+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee78066ce0 con 0x7fee7813ac60 2020-11-20T12:11:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.240+0000 7fee6e7fc700 1 -- 172.21.15.44:0/1982059487 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fee6801a060 con 0x7fee7813ac60 2020-11-20T12:11:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.240+0000 7fee6e7fc700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee54056d20 0x7fee540591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.240+0000 7fee6e7fc700 1 -- 172.21.15.44:0/1982059487 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fee6806f2d0 con 0x7fee7813ac60 2020-11-20T12:11:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.240+0000 7fee7659c700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee54056d20 0x7fee540591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.241+0000 7fee7659c700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee54056d20 0x7fee540591e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fee70009fd0 tx=0x7fee70016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.244+0000 7fee6e7fc700 1 -- 172.21.15.44:0/1982059487 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fee680445a0 con 0x7fee7813ac60 2020-11-20T12:11:38.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.483+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fee780a9770 con 0x7fee7813ac60 2020-11-20T12:11:38.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.483+0000 7fee6e7fc700 1 -- 172.21.15.44:0/1982059487 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fee780a9770 con 0x7fee7813ac60 2020-11-20T12:11:38.487 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:38.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee54056d20 msgr2=0x7fee540591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee54056d20 0x7fee540591e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fee70009fd0 tx=0x7fee70016000).stop 2020-11-20T12:11:38.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee7813ac60 msgr2=0x7fee780667f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee7813ac60 0x7fee780667f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fee6800c760 tx=0x7fee68007c60).stop 2020-11-20T12:11:38.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 shutdown_connections 2020-11-20T12:11:38.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee54056d20 0x7fee540591e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee7813ac60 0x7fee780667f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fee780aa020 0x7fee78065f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 --2- 172.21.15.44:0/1982059487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fee780aa400 0x7fee7813a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 >> 172.21.15.44:0/1982059487 conn(0x7fee78062690 msgr2=0x7fee78135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:38.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 shutdown_connections 2020-11-20T12:11:38.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.485+0000 7fee7cab2700 1 -- 172.21.15.44:0/1982059487 wait complete. 2020-11-20T12:11:38.499 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:11:38.500 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:11:38.500 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:11:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.623+0000 7fde7ee97700 1 Processor -- start 2020-11-20T12:11:38.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.623+0000 7fde7ee97700 1 -- start start 2020-11-20T12:11:38.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7ee97700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa400 0x7fde780a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7ee97700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fde7813ac60 0x7fde780a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7ee97700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde780a9770 con 0x7fde780aa020 2020-11-20T12:11:38.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7ee97700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde780a56c0 con 0x7fde7813ac60 2020-11-20T12:11:38.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7ee97700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780a6da0 con 0x7fde780aa400 2020-11-20T12:11:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7d434700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fde7813ac60 0x7fde780a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7cc33700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fde780aa020 0x7fde780a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43740/0 (socket says 172.21.15.44:43740) 2020-11-20T12:11:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7cc33700 1 -- 172.21.15.44:0/3870185742 learned_addr learned my addr 172.21.15.44:0/3870185742 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde77fff700 1 --2- 172.21.15.44:0/3870185742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa400 0x7fde780a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7d434700 1 -- 172.21.15.44:0/3870185742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa400 msgr2=0x7fde780a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:38.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7d434700 1 --2- 172.21.15.44:0/3870185742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa400 0x7fde780a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7d434700 1 -- 172.21.15.44:0/3870185742 >> v1:172.21.15.44:6789/0 conn(0x7fde780aa020 legacy=0x7fde780a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde777fe700 1 -- 172.21.15.44:0/3870185742 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093329693 0 0) 0x7fde680010d0 con 0x7fde780aa020 2020-11-20T12:11:38.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.624+0000 7fde7d434700 1 -- 172.21.15.44:0/3870185742 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780a7040 con 0x7fde7813ac60 2020-11-20T12:11:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde77fff700 1 --2- 172.21.15.44:0/3870185742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa400 0x7fde780a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:11:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7d434700 1 --2- 172.21.15.44:0/3870185742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde780a3700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fde70009c90 tx=0x7fde700091d0).ready entity=mon.0 client_cookie=8c5cbd7f87c960e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde777fe700 1 -- 172.21.15.44:0/3870185742 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7000bbc0 con 0x7fde7813ac60 2020-11-20T12:11:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde777fe700 1 -- 172.21.15.44:0/3870185742 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7000bd20 con 0x7fde7813ac60 2020-11-20T12:11:38.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde777fe700 1 -- 172.21.15.44:0/3870185742 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70011610 con 0x7fde7813ac60 2020-11-20T12:11:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 -- 172.21.15.44:0/3870185742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 msgr2=0x7fde780a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 --2- 172.21.15.44:0/3870185742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde780a3700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fde70009c90 tx=0x7fde700091d0).stop 2020-11-20T12:11:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 -- 172.21.15.44:0/3870185742 shutdown_connections 2020-11-20T12:11:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 --2- 172.21.15.44:0/3870185742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde780a3700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 --2- 172.21.15.44:0/3870185742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa400 0x7fde780a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 -- 172.21.15.44:0/3870185742 >> 172.21.15.44:0/3870185742 conn(0x7fde78062690 msgr2=0x7fde780a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 -- 172.21.15.44:0/3870185742 shutdown_connections 2020-11-20T12:11:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.625+0000 7fde7ee97700 1 -- 172.21.15.44:0/3870185742 wait complete. 2020-11-20T12:11:38.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.626+0000 7fde7ee97700 1 Processor -- start 2020-11-20T12:11:38.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.626+0000 7fde7ee97700 1 -- start start 2020-11-20T12:11:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7ee97700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa020 0x7fde780ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7ee97700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde780aa400 0x7fde780661f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7ee97700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde7814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7ee97700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780a2f80 con 0x7fde7813ac60 2020-11-20T12:11:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7ee97700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780ac100 con 0x7fde780aa400 2020-11-20T12:11:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7ee97700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780ac3a0 con 0x7fde780aa020 2020-11-20T12:11:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde7814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde7814d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39998/0 (socket says 172.21.15.44:39998) 2020-11-20T12:11:38.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 -- 172.21.15.44:0/3111378643 learned_addr learned my addr 172.21.15.44:0/3111378643 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7cc33700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa020 0x7fde780ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde77fff700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde780aa400 0x7fde780661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa020 msgr2=0x7fde780ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fde780aa020 0x7fde780ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde780aa400 msgr2=0x7fde780661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde780aa400 0x7fde780661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.627+0000 7fde7d434700 1 -- 172.21.15.44:0/3111378643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780ac640 con 0x7fde7813ac60 2020-11-20T12:11:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde7d434700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde7814d1b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fde70009f80 tx=0x7fde70016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7000be90 con 0x7fde7813ac60 2020-11-20T12:11:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70011e20 con 0x7fde7813ac60 2020-11-20T12:11:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde780ab4a0 con 0x7fde7813ac60 2020-11-20T12:11:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70010e50 con 0x7fde7813ac60 2020-11-20T12:11:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde780666e0 con 0x7fde7813ac60 2020-11-20T12:11:38.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde7ee97700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde780aca00 0x7fde7814d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.628+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780abf80 con 0x7fde780aca00 2020-11-20T12:11:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde7cc33700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde780aca00 0x7fde7814d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde7cc33700 1 -- 172.21.15.44:0/3111378643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde78061650 con 0x7fde780aca00 2020-11-20T12:11:38.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fde70017070 con 0x7fde7813ac60 2020-11-20T12:11:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde7cc33700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde780aca00 0x7fde7814d6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fde6800e690 tx=0x7fde68009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde50056d20 0x7fde500591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fde70072780 con 0x7fde7813ac60 2020-11-20T12:11:38.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68007320 con 0x7fde780aca00 2020-11-20T12:11:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fde68007480 con 0x7fde780aca00 2020-11-20T12:11:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde780aca00 msgr2=0x7fde7814d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde780aca00 0x7fde7814d6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fde6800e690 tx=0x7fde68009040).stop 2020-11-20T12:11:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde77fff700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde50056d20 0x7fde500591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde780aca00 msgr2=0x7fde7814d6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.629+0000 7fde74ff9700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde780aca00 0x7fde7814d6a0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0x7fde6800e690 tx=0x7fde68009040).stop 2020-11-20T12:11:38.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.630+0000 7fde77fff700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde50056d20 0x7fde500591e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fde6c006bd0 tx=0x7fde6c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.637+0000 7fde7ee97700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7814db90 0x7fde78149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.637+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780446d0 con 0x7fde7814db90 2020-11-20T12:11:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.637+0000 7fde7cc33700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7814db90 0x7fde78149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.637+0000 7fde7cc33700 1 -- 172.21.15.44:0/3111378643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fde780ac7d0 con 0x7fde7814db90 2020-11-20T12:11:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7cc33700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7814db90 0x7fde78149be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fde6800e690 tx=0x7fde68009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68014c00 con 0x7fde7814db90 2020-11-20T12:11:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4268 (secure 0 0 0) 0x7fde68007110 con 0x7fde7814db90 2020-11-20T12:11:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7814db90 msgr2=0x7fde78149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde74ff9700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7814db90 0x7fde78149be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fde6800e690 tx=0x7fde68009040).stop 2020-11-20T12:11:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde74ff9700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7814db90 msgr2=0x7fde78149be0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde74ff9700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7814db90 0x7fde78149be0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde50056d20 msgr2=0x7fde500591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fde50056d20 0x7fde500591e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fde6c006bd0 tx=0x7fde6c008040).stop 2020-11-20T12:11:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 msgr2=0x7fde7814d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 --2- 172.21.15.44:0/3111378643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde7813ac60 0x7fde7814d1b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fde70009f80 tx=0x7fde70016000).stop 2020-11-20T12:11:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7d434700 1 -- 172.21.15.44:0/3111378643 reap_dead start 2020-11-20T12:11:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7d434700 1 -- 172.21.15.44:0/3111378643 reap_dead start 2020-11-20T12:11:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 shutdown_connections 2020-11-20T12:11:38.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 >> 172.21.15.44:0/3111378643 conn(0x7fde78062690 msgr2=0x7fde780a8100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:38.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 shutdown_connections 2020-11-20T12:11:38.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.638+0000 7fde7ee97700 1 -- 172.21.15.44:0/3111378643 wait complete. 2020-11-20T12:11:38.656 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:38.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:38.657 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:38.657 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:11:38.657 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 94, 2020-11-20T12:11:38.657 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:38.657 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:38.658 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:11:38.658 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:38.658 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.658 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:11:38.658 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:38.659 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:38.659 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:38.659 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:38.659 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:38.659 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:38.660 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:38.660 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:38.660 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:38.660 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:38.660 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:38.661 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.661 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:38.661 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:38.661 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:38.661 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:38.662 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:38.662 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:38.662 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:38.662 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:38.662 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:38.663 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:38.663 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.663 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:38.664 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:38.664 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:38.664 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:38.664 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:38.664 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:38.665 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:38.665 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:38.665 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:38.665 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:38.665 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:38.666 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:38.666 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:38.666 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:38.666 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:38.666 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:38.667 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:38.667 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:38.667 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:38.667 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:38.667 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:38.668 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:38.668 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:38.668 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.668 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:38.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:38.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.669 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:38.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:38.669 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:38.670 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:38.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:38.670 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:38.670 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.671 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:38.671 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:38.671 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.672 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.672 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:38.672 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:38.673 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:38.673 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:38.673 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:38.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.674 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:38.674 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:38.674 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:38.675 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:38.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:38.675 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:38.675 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.676 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:38.676 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:38.677 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.677 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.677 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:38.678 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:38.678 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:38.678 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:38.678 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:38.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.679 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:38.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:38.679 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:38.679 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:38.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.680 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:38.680 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:38.680 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.681 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:38.681 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:38.681 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.682 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.682 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:38.682 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:38.682 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:38.683 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:38.683 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:38.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.683 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.684 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:38.684 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:38.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.684 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:38.684 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:38.684 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:38.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.685 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:38.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.686 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:38.686 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:38.686 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:38.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.687 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:38.687 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:38.688 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:11:38.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.812+0000 7fbb9883c700 1 Processor -- start 2020-11-20T12:11:38.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.812+0000 7fbb9883c700 1 -- start start 2020-11-20T12:11:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.813+0000 7fbb9883c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbb940ae050 0x7fbb94060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.813+0000 7fbb9883c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9413e080 con 0x7fbb940669d0 2020-11-20T12:11:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.813+0000 7fbb9883c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb94060d60 con 0x7fbb9413ebf0 2020-11-20T12:11:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.813+0000 7fbb9883c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb94060b40 con 0x7fbb940ae050 2020-11-20T12:11:38.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.813+0000 7fbb91d9b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbb940669d0 0x7fbb9413e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43758/0 (socket says 172.21.15.44:43758) 2020-11-20T12:11:38.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.813+0000 7fbb9259c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbb940ae050 0x7fbb94060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.813+0000 7fbb91d9b700 1 -- 172.21.15.44:0/3220875987 learned_addr learned my addr 172.21.15.44:0/3220875987 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:38.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9259c700 1 -- 172.21.15.44:0/3220875987 >> v1:172.21.15.79:6789/0 conn(0x7fbb9413ebf0 legacy=0x7fbb94137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:38.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9259c700 1 -- 172.21.15.44:0/3220875987 >> v1:172.21.15.44:6789/0 conn(0x7fbb940669d0 legacy=0x7fbb9413e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9259c700 1 -- 172.21.15.44:0/3220875987 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb940ae430 con 0x7fbb940ae050 2020-11-20T12:11:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9159a700 1 -- 172.21.15.44:0/3220875987 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528157398 0 0) 0x7fbb9413e080 con 0x7fbb940669d0 2020-11-20T12:11:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9259c700 1 --2- 172.21.15.44:0/3220875987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940ae050 0x7fbb94060650 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fbb7c00b9a0 tx=0x7fbb7c006df0).ready entity=mon.0 client_cookie=362fb42c12fd5399 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9159a700 1 -- 172.21.15.44:0/3220875987 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0097b0 con 0x7fbb940ae050 2020-11-20T12:11:38.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9159a700 1 -- 172.21.15.44:0/3220875987 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c009910 con 0x7fbb940ae050 2020-11-20T12:11:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9159a700 1 -- 172.21.15.44:0/3220875987 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c009c90 con 0x7fbb940ae050 2020-11-20T12:11:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9883c700 1 -- 172.21.15.44:0/3220875987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940ae050 msgr2=0x7fbb94060650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9883c700 1 --2- 172.21.15.44:0/3220875987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940ae050 0x7fbb94060650 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fbb7c00b9a0 tx=0x7fbb7c006df0).stop 2020-11-20T12:11:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9883c700 1 -- 172.21.15.44:0/3220875987 shutdown_connections 2020-11-20T12:11:38.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9883c700 1 --2- 172.21.15.44:0/3220875987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940ae050 0x7fbb94060650 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9883c700 1 -- 172.21.15.44:0/3220875987 >> 172.21.15.44:0/3220875987 conn(0x7fbb94062bc0 msgr2=0x7fbb94062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.814+0000 7fbb9883c700 1 -- 172.21.15.44:0/3220875987 shutdown_connections 2020-11-20T12:11:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.815+0000 7fbb9883c700 1 -- 172.21.15.44:0/3220875987 wait complete. 2020-11-20T12:11:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.815+0000 7fbb9883c700 1 Processor -- start 2020-11-20T12:11:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.815+0000 7fbb9883c700 1 -- start start 2020-11-20T12:11:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9883c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940669d0 0x7fbb940ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9883c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb940ae050 0x7fbb9413c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9883c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbb9413ebf0 0x7fbb9413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9883c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9413cb00 con 0x7fbb940669d0 2020-11-20T12:11:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9883c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94139fa0 con 0x7fbb940ae050 2020-11-20T12:11:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9883c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94139a60 con 0x7fbb9413ebf0 2020-11-20T12:11:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940669d0 0x7fbb940ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940669d0 0x7fbb940ad630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40014/0 (socket says 172.21.15.44:40014) 2020-11-20T12:11:38.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 -- 172.21.15.44:0/1327452845 learned_addr learned my addr 172.21.15.44:0/1327452845 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb91d9b700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb940ae050 0x7fbb9413c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb92d9d700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbb9413ebf0 0x7fbb9413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbb9413ebf0 msgr2=0x7fbb9413c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbb9413ebf0 0x7fbb9413c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb940ae050 msgr2=0x7fbb9413c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb940ae050 0x7fbb9413c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb9259c700 1 -- 172.21.15.44:0/1327452845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94139d00 con 0x7fbb940669d0 2020-11-20T12:11:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.816+0000 7fbb91d9b700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb940ae050 0x7fbb9413c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:11:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.817+0000 7fbb9259c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940669d0 0x7fbb940ad630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbb7c006fd0 tx=0x7fbb7c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.817+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c011e20 con 0x7fbb940669d0 2020-11-20T12:11:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.817+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c01d5d0 con 0x7fbb940669d0 2020-11-20T12:11:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.817+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c01d970 con 0x7fbb940669d0 2020-11-20T12:11:38.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.817+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb94139750 con 0x7fbb940669d0 2020-11-20T12:11:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.817+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9413d850 con 0x7fbb940669d0 2020-11-20T12:11:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.818+0000 7fbb9883c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb941321f0 0x7fbb941325d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.818+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9413be80 con 0x7fbb941321f0 2020-11-20T12:11:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.818+0000 7fbb91d9b700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb941321f0 0x7fbb941325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.818+0000 7fbb91d9b700 1 -- 172.21.15.44:0/1327452845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb940a50c0 con 0x7fbb941321f0 2020-11-20T12:11:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.818+0000 7fbb91d9b700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb941321f0 0x7fbb941325d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbb9413b980 tx=0x7fbb84009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.818+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbb7c025030 con 0x7fbb940669d0 2020-11-20T12:11:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbb70056d20 0x7fbb700591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbb7c071b90 con 0x7fbb940669d0 2020-11-20T12:11:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84007320 con 0x7fbb941321f0 2020-11-20T12:11:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb92d9d700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbb70056d20 0x7fbb700591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fbb84007480 con 0x7fbb941321f0 2020-11-20T12:11:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb941321f0 msgr2=0x7fbb941325d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb941321f0 0x7fbb941325d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbb9413b980 tx=0x7fbb84009040).stop 2020-11-20T12:11:38.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb941321f0 msgr2=0x7fbb941325d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:38.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb8a7fc700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb941321f0 0x7fbb941325d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.819+0000 7fbb92d9d700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbb70056d20 0x7fbb700591e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fbb9413be50 tx=0x7fbb8c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.827+0000 7fbb9883c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb94149d10 0x7fbb9414a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:38.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.827+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9413bfc0 con 0x7fbb94149d10 2020-11-20T12:11:38.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.827+0000 7fbb91d9b700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb94149d10 0x7fbb9414a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:38.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.827+0000 7fbb91d9b700 1 -- 172.21.15.44:0/1327452845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb940446d0 con 0x7fbb94149d10 2020-11-20T12:11:38.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.827+0000 7fbb91d9b700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb94149d10 0x7fbb9414a120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fbb9413e050 tx=0x7fbb84009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:38.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.827+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84023480 con 0x7fbb94149d10 2020-11-20T12:11:38.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4436 (secure 0 0 0) 0x7fbb84007110 con 0x7fbb94149d10 2020-11-20T12:11:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb94149d10 msgr2=0x7fbb9414a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb8a7fc700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb94149d10 0x7fbb9414a120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fbb9413e050 tx=0x7fbb84009040).stop 2020-11-20T12:11:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb8a7fc700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb94149d10 msgr2=0x7fbb9414a120 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb8a7fc700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb94149d10 0x7fbb9414a120 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbb70056d20 msgr2=0x7fbb700591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbb70056d20 0x7fbb700591e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fbb9413be50 tx=0x7fbb8c009040).stop 2020-11-20T12:11:38.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940669d0 msgr2=0x7fbb940ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:38.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940669d0 0x7fbb940ad630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbb7c006fd0 tx=0x7fbb7c015000).stop 2020-11-20T12:11:38.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb92d9d700 1 -- 172.21.15.44:0/1327452845 reap_dead start 2020-11-20T12:11:38.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 shutdown_connections 2020-11-20T12:11:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 --2- 172.21.15.44:0/1327452845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb940669d0 0x7fbb940ad630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 >> 172.21.15.44:0/1327452845 conn(0x7fbb94062bc0 msgr2=0x7fbb94134000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 shutdown_connections 2020-11-20T12:11:38.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.828+0000 7fbb9883c700 1 -- 172.21.15.44:0/1327452845 wait complete. 2020-11-20T12:11:38.842 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:38.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:38.842 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:38.842 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:11:38.843 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 94, 2020-11-20T12:11:38.843 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:38.843 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:38.843 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:11:38.843 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:38.844 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.844 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:11:38.844 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:38.844 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:38.845 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:38.845 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:38.845 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:38.845 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:38.846 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:38.846 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:38.846 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:38.846 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:38.846 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:38.847 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.847 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:38.847 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:38.847 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:38.848 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:38.848 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:38.848 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:38.848 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:38.848 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:38.849 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:38.849 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:38.849 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.849 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:38.850 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:38.850 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:38.850 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:38.850 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:38.850 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:38.851 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:38.851 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:38.851 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:38.851 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:38.851 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:38.852 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:38.852 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:38.852 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:38.852 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:38.853 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:38.853 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:38.853 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:38.853 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:38.853 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:38.854 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:38.854 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:38.854 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:38.854 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.854 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:38.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.855 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:38.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.855 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:38.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:38.856 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:38.856 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:38.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.856 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:38.857 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:38.857 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.858 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:38.858 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:38.858 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.858 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.858 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.859 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:38.859 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:38.859 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:38.859 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:38.860 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:38.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.860 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:38.860 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:38.861 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:38.861 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:38.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.861 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:38.861 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:38.862 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.862 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:38.863 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:38.863 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.863 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.863 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.864 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:38.864 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:38.864 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:38.864 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:38.864 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:38.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.865 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:38.865 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:38.866 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:38.866 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:38.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.866 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:38.867 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:38.867 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.867 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:38.868 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:38.868 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:38.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.868 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.869 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:38.869 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:38.869 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:38.869 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:38.869 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:38.870 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.870 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.870 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:38.870 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:38.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.871 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:38.871 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:38.871 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:38.872 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.872 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.872 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:11:38.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.872 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:38.873 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:38.873 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:38.873 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.873 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:38.873 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:38.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:38.874 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:38.874 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:38.874 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:38.875 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:38.875 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:38.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:38.875 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:38.875 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:38.876 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:11:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.999+0000 7f0e3844a700 1 Processor -- start 2020-11-20T12:11:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:38.999+0000 7f0e3844a700 1 -- start start 2020-11-20T12:11:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e3844a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0e300aa020 0x7f0e300a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e3844a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0e3013ac60 0x7f0e300a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e3844a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e300a9770 con 0x7f0e300aa400 2020-11-20T12:11:39.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e3844a700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e300a56c0 con 0x7f0e300aa020 2020-11-20T12:11:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e3844a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e300a6da0 con 0x7f0e3013ac60 2020-11-20T12:11:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e361e6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0e300aa020 0x7f0e300a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e359e5700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0e300aa400 0x7f0e300a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43774/0 (socket says 172.21.15.44:43774) 2020-11-20T12:11:39.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e359e5700 1 -- 172.21.15.44:0/219823148 learned_addr learned my addr 172.21.15.44:0/219823148 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e369e7700 1 --2- 172.21.15.44:0/219823148 >> v2:172.21.15.79:3300/0 conn(0x7f0e3013ac60 0x7f0e300a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e361e6700 1 -- 172.21.15.44:0/219823148 >> v2:172.21.15.79:3300/0 conn(0x7f0e3013ac60 msgr2=0x7f0e300a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:39.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e361e6700 1 --2- 172.21.15.44:0/219823148 >> v2:172.21.15.79:3300/0 conn(0x7f0e3013ac60 0x7f0e300a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e361e6700 1 -- 172.21.15.44:0/219823148 >> v1:172.21.15.44:6789/0 conn(0x7f0e300aa400 legacy=0x7f0e300a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e351e4700 1 -- 172.21.15.44:0/219823148 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964503700 0 0) 0x7f0e20001450 con 0x7f0e300aa400 2020-11-20T12:11:39.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.000+0000 7f0e361e6700 1 -- 172.21.15.44:0/219823148 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e300a7040 con 0x7f0e300aa020 2020-11-20T12:11:39.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e361e6700 1 --2- 172.21.15.44:0/219823148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e300aa020 0x7f0e300a9280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0e300a8f70 tx=0x7f0e24006df0).ready entity=mon.0 client_cookie=52dd311da413a69f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e351e4700 1 -- 172.21.15.44:0/219823148 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240097b0 con 0x7f0e300aa020 2020-11-20T12:11:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e351e4700 1 -- 172.21.15.44:0/219823148 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24009910 con 0x7f0e300aa020 2020-11-20T12:11:39.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e351e4700 1 -- 172.21.15.44:0/219823148 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24009c90 con 0x7f0e300aa020 2020-11-20T12:11:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 -- 172.21.15.44:0/219823148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e300aa020 msgr2=0x7f0e300a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 --2- 172.21.15.44:0/219823148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e300aa020 0x7f0e300a9280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0e300a8f70 tx=0x7f0e24006df0).stop 2020-11-20T12:11:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 -- 172.21.15.44:0/219823148 shutdown_connections 2020-11-20T12:11:39.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 --2- 172.21.15.44:0/219823148 >> v2:172.21.15.79:3300/0 conn(0x7f0e3013ac60 0x7f0e300a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 --2- 172.21.15.44:0/219823148 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e300aa020 0x7f0e300a9280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 -- 172.21.15.44:0/219823148 >> 172.21.15.44:0/219823148 conn(0x7f0e30062690 msgr2=0x7f0e300a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:39.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 -- 172.21.15.44:0/219823148 shutdown_connections 2020-11-20T12:11:39.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.001+0000 7f0e3844a700 1 -- 172.21.15.44:0/219823148 wait complete. 2020-11-20T12:11:39.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.002+0000 7f0e3844a700 1 Processor -- start 2020-11-20T12:11:39.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.002+0000 7f0e3844a700 1 -- start start 2020-11-20T12:11:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e3844a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e300aa020 0x7f0e300a8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e3844a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e300aa400 0x7f0e30134070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e3844a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e3013ac60 0x7f0e3012e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e3844a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3012ff90 con 0x7f0e3013ac60 2020-11-20T12:11:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e3844a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30135300 con 0x7f0e300aa020 2020-11-20T12:11:39.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e3844a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301326f0 con 0x7f0e300aa400 2020-11-20T12:11:39.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e3013ac60 0x7f0e3012e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e3013ac60 0x7f0e3012e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40036/0 (socket says 172.21.15.44:40036) 2020-11-20T12:11:39.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 -- 172.21.15.44:0/3187512727 learned_addr learned my addr 172.21.15.44:0/3187512727 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:39.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e361e6700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e300aa020 0x7f0e300a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e359e5700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e300aa400 0x7f0e30134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e300aa400 msgr2=0x7f0e30134070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e300aa400 0x7f0e30134070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e300aa020 msgr2=0x7f0e300a8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e300aa020 0x7f0e300a8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 -- 172.21.15.44:0/3187512727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30133690 con 0x7f0e3013ac60 2020-11-20T12:11:39.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.003+0000 7f0e369e7700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e3013ac60 0x7f0e3012e690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0e2800ebd0 tx=0x7f0e28006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.004+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800c820 con 0x7f0e3013ac60 2020-11-20T12:11:39.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.004+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2800c980 con 0x7f0e3013ac60 2020-11-20T12:11:39.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.004+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e30133930 con 0x7f0e3013ac60 2020-11-20T12:11:39.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.004+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800cd20 con 0x7f0e3013ac60 2020-11-20T12:11:39.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.004+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e300acf10 con 0x7f0e3013ac60 2020-11-20T12:11:39.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e3844a700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e301330a0 0x7f0e3012eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300a72c0 con 0x7f0e301330a0 2020-11-20T12:11:39.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0e28012600 con 0x7f0e3013ac60 2020-11-20T12:11:39.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e361e6700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e301330a0 0x7f0e3012eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e2e7fc700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e08056d20 0x7f0e080591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0e280703d0 con 0x7f0e3013ac60 2020-11-20T12:11:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e359e5700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e08056d20 0x7f0e080591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e361e6700 1 -- 172.21.15.44:0/3187512727 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e30133d90 con 0x7f0e301330a0 2020-11-20T12:11:39.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.005+0000 7f0e361e6700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e301330a0 0x7f0e3012eb80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0e300a8f70 tx=0x7f0e24015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.006+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24023030 con 0x7f0e301330a0 2020-11-20T12:11:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.006+0000 7f0e359e5700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e08056d20 0x7f0e080591e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0e300adbc0 tx=0x7f0e20007be0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.006+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f0e24011d50 con 0x7f0e301330a0 2020-11-20T12:11:39.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.006+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e301330a0 msgr2=0x7f0e3012eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.006+0000 7f0e2e7fc700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e301330a0 0x7f0e3012eb80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0e300a8f70 tx=0x7f0e24015000).stop 2020-11-20T12:11:39.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.006+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e301330a0 msgr2=0x7f0e3012eb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:39.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.006+0000 7f0e2e7fc700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e301330a0 0x7f0e3012eb80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.014+0000 7f0e3844a700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e30134560 0x7f0e300abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.014+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30061650 con 0x7f0e30134560 2020-11-20T12:11:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.014+0000 7f0e361e6700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e30134560 0x7f0e300abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.014+0000 7f0e361e6700 1 -- 172.21.15.44:0/3187512727 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0e300446d0 con 0x7f0e30134560 2020-11-20T12:11:39.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.014+0000 7f0e361e6700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e30134560 0x7f0e300abc30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f0e2400be30 tx=0x7f0e24021070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2402eb00 con 0x7f0e30134560 2020-11-20T12:11:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f0e24033070 con 0x7f0e30134560 2020-11-20T12:11:39.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e30134560 msgr2=0x7f0e300abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e2e7fc700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e30134560 0x7f0e300abc30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f0e2400be30 tx=0x7f0e24021070).stop 2020-11-20T12:11:39.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e2e7fc700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e30134560 msgr2=0x7f0e300abc30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:39.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e2e7fc700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0e30134560 0x7f0e300abc30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e08056d20 msgr2=0x7f0e080591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e08056d20 0x7f0e080591e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0e300adbc0 tx=0x7f0e20007be0).stop 2020-11-20T12:11:39.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e3013ac60 msgr2=0x7f0e3012e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 --2- 172.21.15.44:0/3187512727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e3013ac60 0x7f0e3012e690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0e2800ebd0 tx=0x7f0e28006df0).stop 2020-11-20T12:11:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e369e7700 1 -- 172.21.15.44:0/3187512727 reap_dead start 2020-11-20T12:11:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e369e7700 1 -- 172.21.15.44:0/3187512727 reap_dead start 2020-11-20T12:11:39.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 shutdown_connections 2020-11-20T12:11:39.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 >> 172.21.15.44:0/3187512727 conn(0x7f0e30062690 msgr2=0x7f0e300a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:39.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 shutdown_connections 2020-11-20T12:11:39.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.015+0000 7f0e3844a700 1 -- 172.21.15.44:0/3187512727 wait complete. 2020-11-20T12:11:39.040 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:39.040 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:39.040 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:39.041 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:11:39.041 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 94, 2020-11-20T12:11:39.041 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:39.042 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:39.042 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:11:39.042 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:39.042 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:39.043 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:11:39.043 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:39.043 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:39.044 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:39.044 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:39.044 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:39.045 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:39.045 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:39.045 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:39.046 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:39.046 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:39.046 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:39.046 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:39.047 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:39.047 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:39.047 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:39.048 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:39.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:39.048 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:39.049 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:39.049 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:39.049 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:39.050 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:39.050 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:39.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.050 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:39.051 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:39.051 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:39.051 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:39.052 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:39.052 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:39.052 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:39.053 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:39.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:39.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:39.053 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:39.054 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:39.054 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:39.054 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:39.055 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:39.055 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:39.055 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:39.056 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:39.056 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:39.056 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:39.057 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:39.057 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:39.057 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:39.057 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:39.058 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:39.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.058 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:39.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.059 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:39.059 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:39.060 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:39.060 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:39.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:39.061 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:39.061 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:39.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.062 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:39.062 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:39.063 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:39.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.063 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:39.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.064 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:39.064 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:39.064 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:39.065 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:39.065 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:39.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.066 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:39.066 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:39.067 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:39.067 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:39.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:39.068 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:39.068 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:39.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.069 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:39.069 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:39.070 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:39.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.070 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:39.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.071 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:39.071 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:39.071 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:39.072 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:39.072 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:39.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.073 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:39.073 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:39.074 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:39.074 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:39.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:39.075 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:39.075 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:39.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.076 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:39.076 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:39.077 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:39.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.077 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:39.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.078 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:39.078 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:39.078 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:39.079 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:39.079 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:39.079 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.080 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:39.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.080 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:39.080 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:39.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.081 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:39.081 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:39.082 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:39.082 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.082 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:39.083 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:11:39.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.083 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:39.084 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:39.084 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:39.084 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.084 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:39.085 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:11:39.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.085 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:39.086 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:39.086 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:11:39.086 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.087 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:39.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:39.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.088 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:39.088 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:39.088 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:11:39.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.089 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:39.089 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:11:39.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:39.090 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:39.090 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:39.090 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:39.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:39.091 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:39.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:39.092 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:39.092 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:39.094 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:11:39.094 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:11:39.094 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:11:39.095 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:11:39.095 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:11:39.095 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:11:39.109 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:11:39.109 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:11:39.110 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:39.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.246+0000 7ffa9359e700 1 Processor -- start 2020-11-20T12:11:39.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.246+0000 7ffa9359e700 1 -- start start 2020-11-20T12:11:39.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9359e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ffa940ad790 0x7ffa940a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9359e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa940a41e0 con 0x7ffa940a6df0 2020-11-20T12:11:39.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9359e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa940a4480 con 0x7ffa940ad3b0 2020-11-20T12:11:39.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9359e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa940a4720 con 0x7ffa940ad790 2020-11-20T12:11:39.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9259c700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7ffa940ad3b0 legacy=0x7ffa9413ab60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:39.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa91d9b700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7ffa940ad790 msgr2=0x7ffa940a5620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:39.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa91d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ffa940ad790 0x7ffa940a5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:39.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa92d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ffa940a6df0 0x7ffa940674c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43802/0 (socket says 172.21.15.44:43802) 2020-11-20T12:11:39.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa92d9d700 1 -- 172.21.15.44:0/3353498024 learned_addr learned my addr 172.21.15.44:0/3353498024 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:11:39.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073239728 0 0) 0x7ffa940a41e0 con 0x7ffa940a6df0 2020-11-20T12:11:39.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa78003240 con 0x7ffa940a6df0 2020-11-20T12:11:39.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1850949520 0 0) 0x7ffa78003240 con 0x7ffa940a6df0 2020-11-20T12:11:39.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 >> v2:172.21.15.79:3300/0 conn(0x7ffa940ad790 msgr2=0x7ffa940a5620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:39.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 --2- 172.21.15.44:0/3353498024 >> v2:172.21.15.79:3300/0 conn(0x7ffa940ad790 0x7ffa940a5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:39.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 >> v1:172.21.15.79:6789/0 conn(0x7ffa940ad3b0 legacy=0x7ffa9413ab60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:39.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9413b270 con 0x7ffa940a6df0 2020-11-20T12:11:39.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7ffa8c001860 con 0x7ffa940a6df0 2020-11-20T12:11:39.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 >> v1:172.21.15.44:6789/0 conn(0x7ffa940a6df0 legacy=0x7ffa940674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa78003730 0x7ffa78003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.247+0000 7ffa9159a700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa780044e0 0x7ffa78004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9159a700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa7800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940a4720 con 0x7ffa78004e50 2020-11-20T12:11:39.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780041f0 con 0x7ffa78003730 2020-11-20T12:11:39.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7800bc20 con 0x7ffa780044e0 2020-11-20T12:11:39.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9259c700 1 -- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa78003730 msgr2=0x7ffa78003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:39.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9259c700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa78003730 0x7ffa78003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:39.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa7800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa91d9b700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa780044e0 0x7ffa78004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 -- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa780044e0 msgr2=0x7ffa78004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:39.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa780044e0 0x7ffa78004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 -- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa78003730 msgr2=0x7ffa78003b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:39.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa78003730 0x7ffa78003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:39.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 -- 172.21.15.44:0/3353498024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78003240 con 0x7ffa78004e50 2020-11-20T12:11:39.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 -- 172.21.15.44:0/3353498024 reap_dead start 2020-11-20T12:11:39.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa92d9d700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa7800b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7ffa940a72a0 tx=0x7ffa8c007b50).ready entity=mon.0 client_cookie=a07a7bf110d6d877 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c005b80 con 0x7ffa78004e50 2020-11-20T12:11:39.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c005ce0 con 0x7ffa78004e50 2020-11-20T12:11:39.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.248+0000 7ffa9159a700 1 -- 172.21.15.44:0/3353498024 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0072e0 con 0x7ffa78004e50 2020-11-20T12:11:39.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.249+0000 7ffa9359e700 1 -- 172.21.15.44:0/3353498024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 msgr2=0x7ffa7800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.249+0000 7ffa9359e700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa7800b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7ffa940a72a0 tx=0x7ffa8c007b50).stop 2020-11-20T12:11:39.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.249+0000 7ffa9359e700 1 -- 172.21.15.44:0/3353498024 shutdown_connections 2020-11-20T12:11:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.249+0000 7ffa9359e700 1 --2- 172.21.15.44:0/3353498024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa7800b730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.249+0000 7ffa9359e700 1 -- 172.21.15.44:0/3353498024 >> 172.21.15.44:0/3353498024 conn(0x7ffa94061b60 msgr2=0x7ffa94061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.249+0000 7ffa9359e700 1 -- 172.21.15.44:0/3353498024 shutdown_connections 2020-11-20T12:11:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.249+0000 7ffa9359e700 1 -- 172.21.15.44:0/3353498024 wait complete. 2020-11-20T12:11:39.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 Processor -- start 2020-11-20T12:11:39.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 -- start start 2020-11-20T12:11:39.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa94066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa94066610 0x7ffa94065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa940aba70 0x7ffa9414a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa941304f0 con 0x7ffa78004e50 2020-11-20T12:11:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94138760 con 0x7ffa940aba70 2020-11-20T12:11:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9359e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94067080 con 0x7ffa94066610 2020-11-20T12:11:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa94066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa94066a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40062/0 (socket says 172.21.15.44:40062) 2020-11-20T12:11:39.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa92d9d700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa940aba70 msgr2=0x7ffa9414a900 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:39.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa9259c700 1 -- 172.21.15.44:0/2944834323 learned_addr learned my addr 172.21.15.44:0/2944834323 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:39.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.250+0000 7ffa92d9d700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa940aba70 0x7ffa9414a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:39.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa91d9b700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa94066610 0x7ffa94065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9259c700 1 -- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa94066610 msgr2=0x7ffa94065cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9259c700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa94066610 0x7ffa94065cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9259c700 1 -- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa940aba70 msgr2=0x7ffa9414a900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:39.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9259c700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa940aba70 0x7ffa9414a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:39.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9259c700 1 -- 172.21.15.44:0/2944834323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9413c9c0 con 0x7ffa78004e50 2020-11-20T12:11:39.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9259c700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa94066a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ffa8800dc40 tx=0x7ffa88006f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa827fc700 1 -- 172.21.15.44:0/2944834323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8800ecb0 con 0x7ffa78004e50 2020-11-20T12:11:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa827fc700 1 -- 172.21.15.44:0/2944834323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8800ee30 con 0x7ffa78004e50 2020-11-20T12:11:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9413c6b0 con 0x7ffa78004e50 2020-11-20T12:11:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa827fc700 1 -- 172.21.15.44:0/2944834323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88006170 con 0x7ffa78004e50 2020-11-20T12:11:39.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.251+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa940abef0 con 0x7ffa78004e50 2020-11-20T12:11:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.252+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa940a36f0 con 0x7ffa78004e50 2020-11-20T12:11:39.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.252+0000 7ffa827fc700 1 -- 172.21.15.44:0/2944834323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ffa88003710 con 0x7ffa78004e50 2020-11-20T12:11:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.252+0000 7ffa827fc700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffa68056cd0 0x7ffa68059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.253+0000 7ffa827fc700 1 -- 172.21.15.44:0/2944834323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ffa88013020 con 0x7ffa78004e50 2020-11-20T12:11:39.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.253+0000 7ffa91d9b700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffa68056cd0 0x7ffa68059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:39.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.253+0000 7ffa91d9b700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffa68056cd0 0x7ffa68059190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7ffa94065410 tx=0x7ffa8400b040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:39.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.256+0000 7ffa827fc700 1 -- 172.21.15.44:0/2944834323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffa880474e0 con 0x7ffa78004e50 2020-11-20T12:11:39.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.498+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffa940a5f50 con 0x7ffa78004e50 2020-11-20T12:11:39.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.498+0000 7ffa827fc700 1 -- 172.21.15.44:0/2944834323 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7ffa8804ab00 con 0x7ffa78004e50 2020-11-20T12:11:39.502 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.499+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffa68056cd0 msgr2=0x7ffa68059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.499+0000 7ffa9359e700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffa68056cd0 0x7ffa68059190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7ffa94065410 tx=0x7ffa8400b040).stop 2020-11-20T12:11:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.499+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 msgr2=0x7ffa94066a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.499+0000 7ffa9359e700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa94066a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ffa8800dc40 tx=0x7ffa88006f30).stop 2020-11-20T12:11:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 shutdown_connections 2020-11-20T12:11:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffa68056cd0 0x7ffa68059190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffa940aba70 0x7ffa9414a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa78004e50 0x7ffa94066a50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 --2- 172.21.15.44:0/2944834323 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffa94066610 0x7ffa94065cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 >> 172.21.15.44:0/2944834323 conn(0x7ffa94061b60 msgr2=0x7ffa94061f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:39.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 shutdown_connections 2020-11-20T12:11:39.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:39.500+0000 7ffa9359e700 1 -- 172.21.15.44:0/2944834323 wait complete. 2020-11-20T12:11:42.516 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:43.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.075+0000 7f3a6d683700 1 Processor -- start 2020-11-20T12:11:43.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a6d683700 1 -- start start 2020-11-20T12:11:43.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a6d683700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa020 0x7f3a680a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:43.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a6d683700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a680aa400 0x7f3a680a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:43.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a6d683700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a680a9770 con 0x7f3a6813ac60 2020-11-20T12:11:43.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a6d683700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680a6da0 con 0x7f3a680aa400 2020-11-20T12:11:43.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a6d683700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680a7040 con 0x7f3a680aa020 2020-11-20T12:11:43.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a677fe700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f3a6813ac60 legacy=0x7f3a680a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:43.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a667fc700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f3a680aa400 msgr2=0x7f3a680a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:43.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.076+0000 7f3a667fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a680aa400 0x7f3a680a4fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:43.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa020 0x7f3a680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:43.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa020 0x7f3a680a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:51518/0 (socket says 172.21.15.44:51518) 2020-11-20T12:11:43.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 -- 172.21.15.44:0/650310595 learned_addr learned my addr 172.21.15.44:0/650310595 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:43.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 -- 172.21.15.44:0/650310595 >> v2:172.21.15.79:3300/0 conn(0x7f3a680aa400 msgr2=0x7f3a680a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:43.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 --2- 172.21.15.44:0/650310595 >> v2:172.21.15.79:3300/0 conn(0x7f3a680aa400 0x7f3a680a4fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:43.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 -- 172.21.15.44:0/650310595 >> v1:172.21.15.79:6789/0 conn(0x7f3a6813ac60 legacy=0x7f3a680a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:43.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 -- 172.21.15.44:0/650310595 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680a6a90 con 0x7f3a680aa020 2020-11-20T12:11:43.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a66ffd700 1 --2- 172.21.15.44:0/650310595 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa020 0x7f3a680a9280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f3a58006fd0 tx=0x7f3a58006eb0).ready entity=mon.2 client_cookie=122affae4144883b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:43.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a65ffb700 1 -- 172.21.15.44:0/650310595 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58009890 con 0x7f3a680aa020 2020-11-20T12:11:43.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a65ffb700 1 -- 172.21.15.44:0/650310595 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a580099f0 con 0x7f3a680aa020 2020-11-20T12:11:43.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.077+0000 7f3a65ffb700 1 -- 172.21.15.44:0/650310595 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58009d70 con 0x7f3a680aa020 2020-11-20T12:11:43.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 -- 172.21.15.44:0/650310595 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa020 msgr2=0x7f3a680a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:43.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 --2- 172.21.15.44:0/650310595 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa020 0x7f3a680a9280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f3a58006fd0 tx=0x7f3a58006eb0).stop 2020-11-20T12:11:43.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 -- 172.21.15.44:0/650310595 shutdown_connections 2020-11-20T12:11:43.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 --2- 172.21.15.44:0/650310595 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa020 0x7f3a680a9280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:43.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 --2- 172.21.15.44:0/650310595 >> v2:172.21.15.79:3300/0 conn(0x7f3a680aa400 0x7f3a680a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:43.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 -- 172.21.15.44:0/650310595 >> 172.21.15.44:0/650310595 conn(0x7f3a68062690 msgr2=0x7f3a680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:43.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 -- 172.21.15.44:0/650310595 shutdown_connections 2020-11-20T12:11:43.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.078+0000 7f3a6d683700 1 -- 172.21.15.44:0/650310595 wait complete. 2020-11-20T12:11:43.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 Processor -- start 2020-11-20T12:11:43.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 -- start start 2020-11-20T12:11:43.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a680aa020 0x7f3a6812b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa400 0x7f3a680ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a6813ac60 0x7f3a68067140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680ab980 con 0x7f3a6813ac60 2020-11-20T12:11:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680abc20 con 0x7f3a680aa020 2020-11-20T12:11:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.079+0000 7f3a6d683700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a68065150 con 0x7f3a680aa400 2020-11-20T12:11:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a66ffd700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a680aa020 msgr2=0x7f3a6812b650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a66ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a680aa020 0x7f3a6812b650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a6813ac60 0x7f3a68067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a6813ac60 0x7f3a68067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40088/0 (socket says 172.21.15.44:40088) 2020-11-20T12:11:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 -- 172.21.15.44:0/2470912123 learned_addr learned my addr 172.21.15.44:0/2470912123 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a667fc700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa400 0x7f3a680ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 -- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa400 msgr2=0x7f3a680ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa400 0x7f3a680ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:43.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 -- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a680aa020 msgr2=0x7f3a6812b650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a680aa020 0x7f3a6812b650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 -- 172.21.15.44:0/2470912123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68067630 con 0x7f3a6813ac60 2020-11-20T12:11:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a677fe700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a6813ac60 0x7f3a68067140 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f3a6000bae0 tx=0x7f3a60006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a4effd700 1 -- 172.21.15.44:0/2470912123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60003c10 con 0x7f3a6813ac60 2020-11-20T12:11:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a4effd700 1 -- 172.21.15.44:0/2470912123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60003d70 con 0x7f3a6813ac60 2020-11-20T12:11:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a680ab590 con 0x7f3a6813ac60 2020-11-20T12:11:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a4effd700 1 -- 172.21.15.44:0/2470912123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60006170 con 0x7f3a6813ac60 2020-11-20T12:11:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.080+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a681290a0 con 0x7f3a6813ac60 2020-11-20T12:11:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.081+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a680abd90 con 0x7f3a6813ac60 2020-11-20T12:11:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.082+0000 7f3a4effd700 1 -- 172.21.15.44:0/2470912123 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3a600065a0 con 0x7f3a6813ac60 2020-11-20T12:11:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.082+0000 7f3a4effd700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a44056cd0 0x7f3a44059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.082+0000 7f3a4effd700 1 -- 172.21.15.44:0/2470912123 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3a60011020 con 0x7f3a6813ac60 2020-11-20T12:11:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.082+0000 7f3a66ffd700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a44056cd0 0x7f3a44059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.082+0000 7f3a66ffd700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a44056cd0 0x7f3a44059190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f3a58006ef0 tx=0x7f3a58015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.086+0000 7f3a4effd700 1 -- 172.21.15.44:0/2470912123 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3a60044cf0 con 0x7f3a6813ac60 2020-11-20T12:11:43.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.326+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3a68061650 con 0x7f3a6813ac60 2020-11-20T12:11:43.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.326+0000 7f3a4effd700 1 -- 172.21.15.44:0/2470912123 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f3a60009270 con 0x7f3a6813ac60 2020-11-20T12:11:43.330 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:43.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a44056cd0 msgr2=0x7f3a44059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:43.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a44056cd0 0x7f3a44059190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f3a58006ef0 tx=0x7f3a58015000).stop 2020-11-20T12:11:43.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a6813ac60 msgr2=0x7f3a68067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:43.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a6813ac60 0x7f3a68067140 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f3a6000bae0 tx=0x7f3a60006f90).stop 2020-11-20T12:11:43.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 shutdown_connections 2020-11-20T12:11:43.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3a44056cd0 0x7f3a44059190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:43.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a6813ac60 0x7f3a68067140 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:43.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a680aa020 0x7f3a6812b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:43.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 --2- 172.21.15.44:0/2470912123 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3a680aa400 0x7f3a680ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:43.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 >> 172.21.15.44:0/2470912123 conn(0x7f3a68062690 msgr2=0x7f3a68135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:43.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 shutdown_connections 2020-11-20T12:11:43.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:43.328+0000 7f3a6d683700 1 -- 172.21.15.44:0/2470912123 wait complete. 2020-11-20T12:11:46.344 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:46.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.478+0000 7fa581822700 1 Processor -- start 2020-11-20T12:11:46.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.478+0000 7fa581822700 1 -- start start 2020-11-20T12:11:46.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa581822700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa57c0a7810 0x7fa57c13c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:46.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa581822700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c066250 0x7fa57c13c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:46.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa581822700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c0a8bf0 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa581822700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57c0a86b0 con 0x7fa57c0a7810 2020-11-20T12:11:46.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa581822700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0a8950 con 0x7fa57c066250 2020-11-20T12:11:46.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa57a7fc700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7fa57c0a7810 msgr2=0x7fa57c13c170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:46.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa57affd700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fa57c0aa1e0 legacy=0x7fa57c065b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:46.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa57a7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa57c0a7810 0x7fa57c13c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:46.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c066250 0x7fa57c13c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:46.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c066250 0x7fa57c13c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:51534/0 (socket says 172.21.15.44:51534) 2020-11-20T12:11:46.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.479+0000 7fa57b7fe700 1 -- 172.21.15.44:0/1602673915 learned_addr learned my addr 172.21.15.44:0/1602673915 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:46.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa57b7fe700 1 -- 172.21.15.44:0/1602673915 >> v2:172.21.15.79:3300/0 conn(0x7fa57c0a7810 msgr2=0x7fa57c13c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:46.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa57b7fe700 1 --2- 172.21.15.44:0/1602673915 >> v2:172.21.15.79:3300/0 conn(0x7fa57c0a7810 0x7fa57c13c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:46.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa57b7fe700 1 -- 172.21.15.44:0/1602673915 >> v1:172.21.15.79:6789/0 conn(0x7fa57c0aa1e0 legacy=0x7fa57c065b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:46.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa57b7fe700 1 -- 172.21.15.44:0/1602673915 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c0aa5c0 con 0x7fa57c066250 2020-11-20T12:11:46.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa57b7fe700 1 --2- 172.21.15.44:0/1602673915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c066250 0x7fa57c13c660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa574006fd0 tx=0x7fa574006eb0).ready entity=mon.2 client_cookie=c0866283eb451a7b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:46.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa579ffb700 1 -- 172.21.15.44:0/1602673915 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574009890 con 0x7fa57c066250 2020-11-20T12:11:46.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa579ffb700 1 -- 172.21.15.44:0/1602673915 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5740099f0 con 0x7fa57c066250 2020-11-20T12:11:46.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.480+0000 7fa579ffb700 1 -- 172.21.15.44:0/1602673915 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574009d70 con 0x7fa57c066250 2020-11-20T12:11:46.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 -- 172.21.15.44:0/1602673915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c066250 msgr2=0x7fa57c13c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:46.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 --2- 172.21.15.44:0/1602673915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c066250 0x7fa57c13c660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa574006fd0 tx=0x7fa574006eb0).stop 2020-11-20T12:11:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 -- 172.21.15.44:0/1602673915 shutdown_connections 2020-11-20T12:11:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 --2- 172.21.15.44:0/1602673915 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c066250 0x7fa57c13c660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 --2- 172.21.15.44:0/1602673915 >> v2:172.21.15.79:3300/0 conn(0x7fa57c0a7810 0x7fa57c13c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 -- 172.21.15.44:0/1602673915 >> 172.21.15.44:0/1602673915 conn(0x7fa57c062690 msgr2=0x7fa57c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:46.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 -- 172.21.15.44:0/1602673915 shutdown_connections 2020-11-20T12:11:46.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.481+0000 7fa581822700 1 -- 172.21.15.44:0/1602673915 wait complete. 2020-11-20T12:11:46.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.482+0000 7fa581822700 1 Processor -- start 2020-11-20T12:11:46.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.482+0000 7fa581822700 1 -- start start 2020-11-20T12:11:46.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa581822700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa57c066250 0x7fa57c0a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:46.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa581822700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c0a7810 0x7fa57c1392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:46.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa581822700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa57c0aa1e0 0x7fa57c1397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:46.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa581822700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0a0dd0 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa581822700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0a1b60 con 0x7fa57c066250 2020-11-20T12:11:46.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa581822700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0a1e00 con 0x7fa57c0a7810 2020-11-20T12:11:46.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57affd700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa57c066250 msgr2=0x7fa57c0a2c00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:46.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa57c0aa1e0 0x7fa57c1397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:46.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa57c066250 0x7fa57c0a2c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:46.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa57c0aa1e0 0x7fa57c1397c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40104/0 (socket says 172.21.15.44:40104) 2020-11-20T12:11:46.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 -- 172.21.15.44:0/775450074 learned_addr learned my addr 172.21.15.44:0/775450074 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:46.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57a7fc700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c0a7810 0x7fa57c1392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:46.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 -- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c0a7810 msgr2=0x7fa57c1392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:46.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c0a7810 0x7fa57c1392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:46.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 -- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa57c066250 msgr2=0x7fa57c0a2c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:46.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa57c066250 0x7fa57c0a2c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:46.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 -- 172.21.15.44:0/775450074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c0a20a0 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa57b7fe700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa57c0aa1e0 0x7fa57c1397c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa574006ef0 tx=0x7fa574015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa572ffd700 1 -- 172.21.15.44:0/775450074 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57401d5d0 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa572ffd700 1 -- 172.21.15.44:0/775450074 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57401d730 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c0a30f0 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.483+0000 7fa572ffd700 1 -- 172.21.15.44:0/775450074 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57401daf0 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.484+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c0a1730 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.485+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57c0a2230 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.485+0000 7fa572ffd700 1 -- 172.21.15.44:0/775450074 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa574025030 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.485+0000 7fa572ffd700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa558056cd0 0x7fa558059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:46.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.485+0000 7fa572ffd700 1 -- 172.21.15.44:0/775450074 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa574071d60 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.485+0000 7fa57affd700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa558056cd0 0x7fa558059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:46.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.486+0000 7fa57affd700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa558056cd0 0x7fa558059190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa564006fd0 tx=0x7fa564006a90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:46.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.489+0000 7fa572ffd700 1 -- 172.21.15.44:0/775450074 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa5740470b0 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.733+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa57c0a1950 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.734+0000 7fa572ffd700 1 -- 172.21.15.44:0/775450074 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fa574023070 con 0x7fa57c0aa1e0 2020-11-20T12:11:46.739 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":94,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:46.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa558056cd0 msgr2=0x7fa558059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:46.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa558056cd0 0x7fa558059190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fa564006fd0 tx=0x7fa564006a90).stop 2020-11-20T12:11:46.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa57c0aa1e0 msgr2=0x7fa57c1397c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:46.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa57c0aa1e0 0x7fa57c1397c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa574006ef0 tx=0x7fa574015000).stop 2020-11-20T12:11:46.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 shutdown_connections 2020-11-20T12:11:46.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa558056cd0 0x7fa558059190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:46.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa57c0aa1e0 0x7fa57c1397c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:46.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa57c066250 0x7fa57c0a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:46.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 --2- 172.21.15.44:0/775450074 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa57c0a7810 0x7fa57c1392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 >> 172.21.15.44:0/775450074 conn(0x7fa57c062690 msgr2=0x7fa57c065a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 shutdown_connections 2020-11-20T12:11:46.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:46.735+0000 7fa581822700 1 -- 172.21.15.44:0/775450074 wait complete. 2020-11-20T12:11:49.751 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:11:49.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.887+0000 7f852514e700 1 Processor -- start 2020-11-20T12:11:49.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.887+0000 7f852514e700 1 -- start start 2020-11-20T12:11:49.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f852514e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:49.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f852514e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:49.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f852514e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520067350 con 0x7f85200ab620 2020-11-20T12:11:49.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f852514e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85200a9b40 con 0x7f85200abc30 2020-11-20T12:11:49.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f852514e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85200ab040 con 0x7f8520066bc0 2020-11-20T12:11:49.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851ed9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:49.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851e59c700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f85200ab620 legacy=0x7f8520137bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:49.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851ed9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40108/0 (socket says 172.21.15.44:40108) 2020-11-20T12:11:49.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 learned_addr learned my addr 172.21.15.44:0/1371564292 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:49.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:49.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:49.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:49.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:49.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:49.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:11:49.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:49.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:49.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:49.888+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851e59c700 1 -- 172.21.15.44:0/1371564292 >> v1:172.21.15.79:6789/0 conn(0x7f85200ab620 legacy=0x7f8520137bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:50.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:50.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:50.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:50.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:50.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:50.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:50.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:11:50.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:50.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:50.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.089+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:50.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851e59c700 1 -- 172.21.15.44:0/1371564292 >> v1:172.21.15.79:6789/0 conn(0x7f85200ab620 legacy=0x7f8520137bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:50.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:50.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:50.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:50.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:50.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:50.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:50.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:50.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:50.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:50.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:50.490+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:51.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851e59c700 1 -- 172.21.15.44:0/1371564292 >> v1:172.21.15.79:6789/0 conn(0x7f85200ab620 legacy=0x7f8520137bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:51.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:51.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:11:51.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:51.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:51.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:11:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:11:51.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:11:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:11:51.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:51.291+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:11:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 msgr2=0x7f85201382d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8520066bc0 0x7f85201382d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 msgr2=0x7f85200666d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:52.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.44:3300/0 conn(0x7f85200abc30 0x7f85200666d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:52.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 -- 172.21.15.44:0/1371564292 >> v1:172.21.15.79:6789/0 conn(0x7f85200ab620 legacy=0x7f8520137bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:52.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 0x7f8500005810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:52.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 -- 172.21.15.44:0/1371564292 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520067880 con 0x7f8500005e10 2020-11-20T12:11:52.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851cd99700 1 -- 172.21.15.44:0/1371564292 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85200a9b40 con 0x7f85000033b0 2020-11-20T12:11:52.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 msgr2=0x7f8500005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:52.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:52.888+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 0x7f8500005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:53.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.089+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 msgr2=0x7f8500005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:53.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.089+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 0x7f8500005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:11:53.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.490+0000 7f851ed9d700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 msgr2=0x7f8500005810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:54.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.490+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 0x7f8500005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:11:54.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.547+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279602866 0 0) 0x7f8520067880 con 0x7f8500005e10 2020-11-20T12:11:54.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.547+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f850c003240 con 0x7f8500005e10 2020-11-20T12:11:54.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.547+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1235916199 0 0) 0x7f850c003240 con 0x7f8500005e10 2020-11-20T12:11:54.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.547+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 msgr2=0x7f8500005810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:54.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.547+0000 7f851dd9b700 1 --2- 172.21.15.44:0/1371564292 >> v2:172.21.15.79:3300/0 conn(0x7f85000033b0 0x7f8500005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.547+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8520067350 con 0x7f8500005e10 2020-11-20T12:11:54.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.547+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f8510004180 con 0x7f8500005e10 2020-11-20T12:11:54.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 >> v1:172.21.15.44:6789/0 conn(0x7f8500005e10 legacy=0x7f85000082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 reap_dead start 2020-11-20T12:11:54.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851dd9b700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f850c003730 0x7f850c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851dd9b700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004040 0x7f850c004520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851dd9b700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c004bc0 0x7f850c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200a9b40 con 0x7f850c004040 2020-11-20T12:11:54.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c005260 con 0x7f850c004bc0 2020-11-20T12:11:54.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c00bc20 con 0x7f850c003730 2020-11-20T12:11:54.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851f59e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c004bc0 msgr2=0x7f850c00b730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851f59e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c004bc0 0x7f850c00b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851e59c700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004040 0x7f850c004520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f850c003730 0x7f850c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851e59c700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f850c003730 msgr2=0x7f850c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:54.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851e59c700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f850c003730 0x7f850c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851e59c700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c004bc0 msgr2=0x7f850c00b730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:54.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851e59c700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c004bc0 0x7f850c00b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.548+0000 7f851e59c700 1 -- 172.21.15.44:0/1371564292 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c003240 con 0x7f850c004040 2020-11-20T12:11:54.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.549+0000 7f851ed9d700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f850c003730 0x7f850c003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:11:54.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.549+0000 7f851e59c700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004040 0x7f850c004520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f85200ab330 tx=0x7f8510007be0).ready entity=mon.0 client_cookie=a57e1145594a03e3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.549+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100062e0 con 0x7f850c004040 2020-11-20T12:11:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.549+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510006440 con 0x7f850c004040 2020-11-20T12:11:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.549+0000 7f851dd9b700 1 -- 172.21.15.44:0/1371564292 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004810 con 0x7f850c004040 2020-11-20T12:11:54.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 -- 172.21.15.44:0/1371564292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004040 msgr2=0x7f850c004520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004040 0x7f850c004520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f85200ab330 tx=0x7f8510007be0).stop 2020-11-20T12:11:54.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 -- 172.21.15.44:0/1371564292 shutdown_connections 2020-11-20T12:11:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c004bc0 0x7f850c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004040 0x7f850c004520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 --2- 172.21.15.44:0/1371564292 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f850c003730 0x7f850c003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 -- 172.21.15.44:0/1371564292 >> 172.21.15.44:0/1371564292 conn(0x7f8520065290 msgr2=0x7f8520060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:54.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 -- 172.21.15.44:0/1371564292 shutdown_connections 2020-11-20T12:11:54.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.550+0000 7f852514e700 1 -- 172.21.15.44:0/1371564292 wait complete. 2020-11-20T12:11:54.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.552+0000 7f852514e700 1 Processor -- start 2020-11-20T12:11:54.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.552+0000 7f852514e700 1 -- start start 2020-11-20T12:11:54.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f852514e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c003730 0x7f85200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f852514e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004bc0 0x7f852014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f852514e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f852014aca0 0x7f852014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f852514e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852014a350 con 0x7f850c004bc0 2020-11-20T12:11:54.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f852514e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852014a560 con 0x7f850c003730 2020-11-20T12:11:54.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f852514e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852014a800 con 0x7f852014aca0 2020-11-20T12:11:54.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851ed9d700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c003730 msgr2=0x7f85200a5460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:54.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851ed9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c003730 0x7f85200a5460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004bc0 0x7f852014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004bc0 0x7f852014d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40150/0 (socket says 172.21.15.44:40150) 2020-11-20T12:11:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851e59c700 1 -- 172.21.15.44:0/454857262 learned_addr learned my addr 172.21.15.44:0/454857262 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:54.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851f59e700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f852014aca0 0x7f852014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851e59c700 1 -- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f852014aca0 msgr2=0x7f852014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851e59c700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f852014aca0 0x7f852014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.553+0000 7f851e59c700 1 -- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c003730 msgr2=0x7f85200a5460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.554+0000 7f851e59c700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c003730 0x7f85200a5460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.554+0000 7f851e59c700 1 -- 172.21.15.44:0/454857262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8520149fb0 con 0x7f850c004bc0 2020-11-20T12:11:54.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.554+0000 7f851e59c700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004bc0 0x7f852014d580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f850c005230 tx=0x7f8510015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.554+0000 7f8506ffd700 1 -- 172.21.15.44:0/454857262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004bc0 con 0x7f850c004bc0 2020-11-20T12:11:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.554+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852013a6a0 con 0x7f850c004bc0 2020-11-20T12:11:54.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.554+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85200a5950 con 0x7f850c004bc0 2020-11-20T12:11:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.555+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852014a9c0 con 0x7f850c004bc0 2020-11-20T12:11:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.556+0000 7f8506ffd700 1 -- 172.21.15.44:0/454857262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510003e90 con 0x7f850c004bc0 2020-11-20T12:11:54.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.556+0000 7f8506ffd700 1 -- 172.21.15.44:0/454857262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510007030 con 0x7f850c004bc0 2020-11-20T12:11:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.558+0000 7f8506ffd700 1 -- 172.21.15.44:0/454857262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8510027070 con 0x7f850c004bc0 2020-11-20T12:11:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.559+0000 7f8506ffd700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.559+0000 7f8506ffd700 1 -- 172.21.15.44:0/454857262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f85100711f0 con 0x7f850c004bc0 2020-11-20T12:11:54.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.559+0000 7f851ed9d700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.559+0000 7f851ed9d700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f8514006cc0 tx=0x7f8514008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.562+0000 7f8506ffd700 1 -- 172.21.15.44:0/454857262 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f85100464c0 con 0x7f850c004bc0 2020-11-20T12:11:54.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.805+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f852005c2a0 con 0x7f850c004bc0 2020-11-20T12:11:54.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.805+0000 7f8506ffd700 1 -- 172.21.15.44:0/454857262 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1463 (secure 0 0 0) 0x7f8510025070 con 0x7f850c004bc0 2020-11-20T12:11:54.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 msgr2=0x7f84f4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f8514006cc0 tx=0x7f8514008040).stop 2020-11-20T12:11:54.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004bc0 msgr2=0x7f852014d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004bc0 0x7f852014d580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f850c005230 tx=0x7f8510015000).stop 2020-11-20T12:11:54.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 shutdown_connections 2020-11-20T12:11:54.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f852014aca0 0x7f852014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f850c003730 0x7f85200a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 --2- 172.21.15.44:0/454857262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c004bc0 0x7f852014d580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 >> 172.21.15.44:0/454857262 conn(0x7f8520065290 msgr2=0x7f85200a1880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:54.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 shutdown_connections 2020-11-20T12:11:54.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:53.807+0000 7f852514e700 1 -- 172.21.15.44:0/454857262 wait complete. 2020-11-20T12:11:54.243 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":98,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:11:54.244 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:11:54.244 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:11:54.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.376+0000 7f8eb03dd700 1 Processor -- start 2020-11-20T12:11:54.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.376+0000 7f8eb03dd700 1 -- start start 2020-11-20T12:11:54.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8eb03dd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8ea80aa400 0x7f8ea80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8eb03dd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea813ac60 0x7f8ea80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8eb03dd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea80a9770 con 0x7f8ea80aa020 2020-11-20T12:11:54.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8eb03dd700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea80a56c0 con 0x7f8ea80aa400 2020-11-20T12:11:54.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8eb03dd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea80a6da0 con 0x7f8ea813ac60 2020-11-20T12:11:54.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8ea80aa400 0x7f8ea80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8eae179700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f8ea80aa020 legacy=0x7f8ea80a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:54.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8ea80aa400 0x7f8ea80a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40156/0 (socket says 172.21.15.44:40156) 2020-11-20T12:11:54.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 -- 172.21.15.44:0/1754696448 learned_addr learned my addr 172.21.15.44:0/1754696448 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:54.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8eae97a700 1 --2- 172.21.15.44:0/1754696448 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea813ac60 0x7f8ea80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 -- 172.21.15.44:0/1754696448 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea813ac60 msgr2=0x7f8ea80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:11:54.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 --2- 172.21.15.44:0/1754696448 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea813ac60 0x7f8ea80a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 -- 172.21.15.44:0/1754696448 >> v1:172.21.15.79:6789/0 conn(0x7f8ea80aa020 legacy=0x7f8ea80a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:54.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 -- 172.21.15.44:0/1754696448 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea80a7040 con 0x7f8ea80aa400 2020-11-20T12:11:54.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.377+0000 7f8ead978700 1 --2- 172.21.15.44:0/1754696448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea80aa400 0x7f8ea80a9280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8ea4006fd0 tx=0x7f8ea4006eb0).ready entity=mon.0 client_cookie=dd6666f0664a469c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8ead177700 1 -- 172.21.15.44:0/1754696448 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4009890 con 0x7f8ea80aa400 2020-11-20T12:11:54.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8ead177700 1 -- 172.21.15.44:0/1754696448 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea40099f0 con 0x7f8ea80aa400 2020-11-20T12:11:54.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8ead177700 1 -- 172.21.15.44:0/1754696448 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4009d70 con 0x7f8ea80aa400 2020-11-20T12:11:54.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 -- 172.21.15.44:0/1754696448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea80aa400 msgr2=0x7f8ea80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 --2- 172.21.15.44:0/1754696448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea80aa400 0x7f8ea80a9280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8ea4006fd0 tx=0x7f8ea4006eb0).stop 2020-11-20T12:11:54.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 -- 172.21.15.44:0/1754696448 shutdown_connections 2020-11-20T12:11:54.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 --2- 172.21.15.44:0/1754696448 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea813ac60 0x7f8ea80a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 --2- 172.21.15.44:0/1754696448 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea80aa400 0x7f8ea80a9280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 -- 172.21.15.44:0/1754696448 >> 172.21.15.44:0/1754696448 conn(0x7f8ea8062690 msgr2=0x7f8ea80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:54.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 -- 172.21.15.44:0/1754696448 shutdown_connections 2020-11-20T12:11:54.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.378+0000 7f8eb03dd700 1 -- 172.21.15.44:0/1754696448 wait complete. 2020-11-20T12:11:54.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.379+0000 7f8eb03dd700 1 Processor -- start 2020-11-20T12:11:54.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.379+0000 7f8eb03dd700 1 -- start start 2020-11-20T12:11:54.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ea80aa020 0x7f8ea81285b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea80aa400 0x7f8ea812a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea813ac60 0x7f8ea8125bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea80ad950 con 0x7f8ea813ac60 2020-11-20T12:11:54.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8126f70 con 0x7f8ea80aa020 2020-11-20T12:11:54.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81271d0 con 0x7f8ea80aa400 2020-11-20T12:11:54.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea813ac60 0x7f8ea8125bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea813ac60 0x7f8ea8125bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40166/0 (socket says 172.21.15.44:40166) 2020-11-20T12:11:54.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 -- 172.21.15.44:0/836374340 learned_addr learned my addr 172.21.15.44:0/836374340 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:54.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae179700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ea80aa020 msgr2=0x7f8ea81285b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:54.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae179700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ea80aa020 0x7f8ea81285b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:54.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8ead978700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea80aa400 0x7f8ea812a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea80aa400 msgr2=0x7f8ea812a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8ea80aa400 0x7f8ea812a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ea80aa020 msgr2=0x7f8ea81285b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:54.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ea80aa020 0x7f8ea81285b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 -- 172.21.15.44:0/836374340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8127470 con 0x7f8ea813ac60 2020-11-20T12:11:54.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eae97a700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea813ac60 0x7f8ea8125bb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8ea000eb70 tx=0x7f8ea0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000c7a0 con 0x7f8ea813ac60 2020-11-20T12:11:54.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea000c900 con 0x7f8ea813ac60 2020-11-20T12:11:54.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea8127710 con 0x7f8ea813ac60 2020-11-20T12:11:54.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000cc80 con 0x7f8ea813ac60 2020-11-20T12:11:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.380+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea8067f90 con 0x7f8ea813ac60 2020-11-20T12:11:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.381+0000 7f8eb03dd700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea8127ad0 0x7f8ea8129df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.381+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8127eb0 con 0x7f8ea8127ad0 2020-11-20T12:11:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.381+0000 7f8eae179700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea8127ad0 0x7f8ea8129df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8ea001d460 con 0x7f8ea813ac60 2020-11-20T12:11:54.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8eae179700 1 -- 172.21.15.44:0/836374340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ea8061650 con 0x7f8ea8127ad0 2020-11-20T12:11:54.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8eae179700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea8127ad0 0x7f8ea8129df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8e98009640 tx=0x7f8e98006e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8e80056d00 0x7f8e800591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8ea0070140 con 0x7f8ea813ac60 2020-11-20T12:11:54.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8ead978700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8e80056d00 0x7f8e800591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e980050d0 con 0x7f8ea8127ad0 2020-11-20T12:11:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f8e98005230 con 0x7f8ea8127ad0 2020-11-20T12:11:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea8127ad0 msgr2=0x7f8ea8129df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea8127ad0 0x7f8ea8129df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f8e98009640 tx=0x7f8e98006e60).stop 2020-11-20T12:11:54.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea8127ad0 msgr2=0x7f8ea8129df0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.382+0000 7f8e967fc700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea8127ad0 0x7f8ea8129df0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0x7f8e98009640 tx=0x7f8e98006e60).stop 2020-11-20T12:11:54.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.383+0000 7f8ead978700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8e80056d00 0x7f8e800591c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f8ea813ac30 tx=0x7f8ea4015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.390+0000 7f8eb03dd700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea814b360 0x7f8ea814b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.390+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea812b9f0 con 0x7f8ea814b360 2020-11-20T12:11:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.390+0000 7f8eae179700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea814b360 0x7f8ea814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eae179700 1 -- 172.21.15.44:0/836374340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8ea80446d0 con 0x7f8ea814b360 2020-11-20T12:11:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eae179700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea814b360 0x7f8ea814b770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8e98009640 tx=0x7f8e98013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98015c60 con 0x7f8ea814b360 2020-11-20T12:11:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4426 (secure 0 0 0) 0x7f8e98003720 con 0x7f8ea814b360 2020-11-20T12:11:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea814b360 msgr2=0x7f8ea814b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8e967fc700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea814b360 0x7f8ea814b770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f8e98009640 tx=0x7f8e98013000).stop 2020-11-20T12:11:54.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8e967fc700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea814b360 msgr2=0x7f8ea814b770 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8e967fc700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea814b360 0x7f8ea814b770 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0x7f8e98009640 tx=0x7f8e98013000).stop 2020-11-20T12:11:54.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8e80056d00 msgr2=0x7f8e800591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8e80056d00 0x7f8e800591c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f8ea813ac30 tx=0x7f8ea4015000).stop 2020-11-20T12:11:54.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea813ac60 msgr2=0x7f8ea8125bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 --2- 172.21.15.44:0/836374340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8ea813ac60 0x7f8ea8125bb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8ea000eb70 tx=0x7f8ea0008040).stop 2020-11-20T12:11:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eae97a700 1 -- 172.21.15.44:0/836374340 reap_dead start 2020-11-20T12:11:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eae97a700 1 -- 172.21.15.44:0/836374340 reap_dead start 2020-11-20T12:11:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 shutdown_connections 2020-11-20T12:11:54.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 >> 172.21.15.44:0/836374340 conn(0x7f8ea8062690 msgr2=0x7f8ea8138b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 shutdown_connections 2020-11-20T12:11:54.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.391+0000 7f8eb03dd700 1 -- 172.21.15.44:0/836374340 wait complete. 2020-11-20T12:11:54.414 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:54.414 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:54.415 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:54.415 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:11:54.415 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 98, 2020-11-20T12:11:54.415 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:54.415 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:54.416 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:54.416 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.416 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:11:54.416 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:54.416 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:54.417 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:54.417 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:54.417 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:54.417 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:54.417 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:54.418 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:54.418 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:54.418 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:54.418 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:54.418 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.419 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:54.419 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:54.419 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:54.419 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:54.419 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:54.420 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:54.420 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:54.420 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:54.420 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:54.420 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:54.421 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.421 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:54.421 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:54.421 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:54.422 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:54.422 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:54.422 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:54.422 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:54.422 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:54.423 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:54.423 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:54.423 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:54.423 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:54.423 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:54.424 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:54.424 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:54.424 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:54.424 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:54.424 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:54.425 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:54.425 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:54.425 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:54.425 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:54.425 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:54.426 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.426 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:54.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.426 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:54.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.427 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:54.427 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:54.427 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:54.427 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:54.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:54.428 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:54.428 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.429 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:54.429 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:54.429 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.429 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.430 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:54.430 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:54.430 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:54.430 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:54.431 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:54.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.431 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:54.431 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:54.432 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:54.432 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:54.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:54.432 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:54.433 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.433 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:54.433 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:54.434 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.434 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.434 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.434 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:54.435 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:54.435 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:54.435 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:54.435 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:54.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.436 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:54.436 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:54.436 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:54.436 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:54.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:54.437 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:54.437 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.438 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:54.438 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:54.438 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.439 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.439 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:54.439 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:54.439 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:54.440 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:54.440 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:54.440 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.440 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.441 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:54.441 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:54.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.441 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.441 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.442 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:54.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.442 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.442 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:11:54.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.443 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.443 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.443 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:54.443 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.443 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.444 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:54.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.444 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.444 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.444 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:54.445 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.445 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.445 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:54.446 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:54.447 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:11:54.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.583+0000 7f632efcf700 1 Processor -- start 2020-11-20T12:11:54.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.583+0000 7f632efcf700 1 -- start start 2020-11-20T12:11:54.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.583+0000 7f632efcf700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813a990 0x7f632813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.583+0000 7f632efcf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f632813f800 0x7f6328141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.583+0000 7f632efcf700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63280a6260 con 0x7f6328060330 2020-11-20T12:11:54.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.583+0000 7f632efcf700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63280a52e0 con 0x7f632813f800 2020-11-20T12:11:54.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.583+0000 7f632efcf700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280a6490 con 0x7f632813a990 2020-11-20T12:11:54.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f632cd6b700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f6328060330 legacy=0x7f63280a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:11:54.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f632d56c700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f632813f800 msgr2=0x7f6328141bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:54.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f632d56c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f632813f800 0x7f6328141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813a990 0x7f632813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813a990 0x7f632813adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:51612/0 (socket says 172.21.15.44:51612) 2020-11-20T12:11:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 -- 172.21.15.44:0/713493113 learned_addr learned my addr 172.21.15.44:0/713493113 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 -- 172.21.15.44:0/713493113 >> v2:172.21.15.79:3300/0 conn(0x7f632813f800 msgr2=0x7f6328141bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:54.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 --2- 172.21.15.44:0/713493113 >> v2:172.21.15.79:3300/0 conn(0x7f632813f800 0x7f6328141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 -- 172.21.15.44:0/713493113 >> v1:172.21.15.79:6789/0 conn(0x7f6328060330 legacy=0x7f63280a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:11:54.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 -- 172.21.15.44:0/713493113 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280a6730 con 0x7f632813a990 2020-11-20T12:11:54.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.584+0000 7f6327fff700 1 --2- 172.21.15.44:0/713493113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813a990 0x7f632813adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6314006fd0 tx=0x7f6314006eb0).ready entity=mon.2 client_cookie=bea10eca8522b6c3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f63277fe700 1 -- 172.21.15.44:0/713493113 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314009890 con 0x7f632813a990 2020-11-20T12:11:54.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f63277fe700 1 -- 172.21.15.44:0/713493113 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63140099f0 con 0x7f632813a990 2020-11-20T12:11:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f63277fe700 1 -- 172.21.15.44:0/713493113 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314009d70 con 0x7f632813a990 2020-11-20T12:11:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 -- 172.21.15.44:0/713493113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813a990 msgr2=0x7f632813adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 --2- 172.21.15.44:0/713493113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813a990 0x7f632813adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6314006fd0 tx=0x7f6314006eb0).stop 2020-11-20T12:11:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 -- 172.21.15.44:0/713493113 shutdown_connections 2020-11-20T12:11:54.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 --2- 172.21.15.44:0/713493113 >> v2:172.21.15.79:3300/0 conn(0x7f632813f800 0x7f6328141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 --2- 172.21.15.44:0/713493113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813a990 0x7f632813adf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 -- 172.21.15.44:0/713493113 >> 172.21.15.44:0/713493113 conn(0x7f6328062690 msgr2=0x7f63280a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:54.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 -- 172.21.15.44:0/713493113 shutdown_connections 2020-11-20T12:11:54.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.585+0000 7f632efcf700 1 -- 172.21.15.44:0/713493113 wait complete. 2020-11-20T12:11:54.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.586+0000 7f632efcf700 1 Processor -- start 2020-11-20T12:11:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.586+0000 7f632efcf700 1 -- start start 2020-11-20T12:11:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632efcf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6328060330 0x7f632812f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632efcf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f632813a990 0x7f63281461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632efcf700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813f800 0x7f63281466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632efcf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6328146b90 con 0x7f632813a990 2020-11-20T12:11:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632efcf700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6328145d80 con 0x7f6328060330 2020-11-20T12:11:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632efcf700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6328067ed0 con 0x7f632813f800 2020-11-20T12:11:54.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632cd6b700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6328060330 msgr2=0x7f632812f6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:11:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632cd6b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6328060330 0x7f632812f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:11:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f632813a990 0x7f63281461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f632813a990 0x7f63281461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40184/0 (socket says 172.21.15.44:40184) 2020-11-20T12:11:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f632d56c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813f800 0x7f63281466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 -- 172.21.15.44:0/45293888 learned_addr learned my addr 172.21.15.44:0/45293888 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:11:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813f800 msgr2=0x7f63281466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632813f800 0x7f63281466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:11:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6328060330 msgr2=0x7f632812f6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:11:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6328060330 0x7f632812f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:11:54.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 -- 172.21.15.44:0/45293888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63281391d0 con 0x7f632813a990 2020-11-20T12:11:54.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.587+0000 7f6327fff700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f632813a990 0x7f63281461b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f63280a65d0 tx=0x7f6314015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.588+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631401e460 con 0x7f632813a990 2020-11-20T12:11:54.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.588+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631401e600 con 0x7f632813a990 2020-11-20T12:11:54.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.588+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63280abad0 con 0x7f632813a990 2020-11-20T12:11:54.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.588+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631401e9c0 con 0x7f632813a990 2020-11-20T12:11:54.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.588+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6328067b80 con 0x7f632813a990 2020-11-20T12:11:54.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.588+0000 7f632efcf700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63280ab6e0 0x7f632812e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.588+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280abf30 con 0x7f63280ab6e0 2020-11-20T12:11:54.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f632cd6b700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63280ab6e0 0x7f632812e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f631401a040 con 0x7f632813a990 2020-11-20T12:11:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f632cd6b700 1 -- 172.21.15.44:0/45293888 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f632812fbb0 con 0x7f63280ab6e0 2020-11-20T12:11:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f6324ff9700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6300056d00 0x7f63000591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f6314071640 con 0x7f632813a990 2020-11-20T12:11:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f632cd6b700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63280ab6e0 0x7f632812e0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f631c009640 tx=0x7f631c006e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f632d56c700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6300056d00 0x7f63000591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.589+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0050d0 con 0x7f63280ab6e0 2020-11-20T12:11:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.590+0000 7f632d56c700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6300056d00 0x7f63000591c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f6320003fd0 tx=0x7f63200090f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.590+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f631c005230 con 0x7f63280ab6e0 2020-11-20T12:11:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.590+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63280ab6e0 msgr2=0x7f632812e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.590+0000 7f6324ff9700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63280ab6e0 0x7f632812e0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f631c009640 tx=0x7f631c006e60).stop 2020-11-20T12:11:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.590+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63280ab6e0 msgr2=0x7f632812e0d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.590+0000 7f6324ff9700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f63280ab6e0 0x7f632812e0d0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0x7f631c009640 tx=0x7f631c006e60).stop 2020-11-20T12:11:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.597+0000 7f632efcf700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632812e5c0 0x7f63281492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:11:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.597+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6328061650 con 0x7f632812e5c0 2020-11-20T12:11:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.598+0000 7f632cd6b700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632812e5c0 0x7f63281492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:11:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.598+0000 7f632cd6b700 1 -- 172.21.15.44:0/45293888 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f63280446d0 con 0x7f632812e5c0 2020-11-20T12:11:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.598+0000 7f632cd6b700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632812e5c0 0x7f63281492d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f631c006ea0 tx=0x7f631c013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:11:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.598+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c0150d0 con 0x7f632812e5c0 2020-11-20T12:11:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4764 (secure 0 0 0) 0x7f631c003720 con 0x7f632812e5c0 2020-11-20T12:11:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632812e5c0 msgr2=0x7f63281492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f6324ff9700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632812e5c0 0x7f63281492d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f631c006ea0 tx=0x7f631c013000).stop 2020-11-20T12:11:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f6324ff9700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632812e5c0 msgr2=0x7f63281492d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:11:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f6324ff9700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f632812e5c0 0x7f63281492d0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0x7f631c006ea0 tx=0x7f631c013000).stop 2020-11-20T12:11:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6300056d00 msgr2=0x7f63000591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6300056d00 0x7f63000591c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f6320003fd0 tx=0x7f63200090f0).stop 2020-11-20T12:11:54.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f632813a990 msgr2=0x7f63281461b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:11:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 --2- 172.21.15.44:0/45293888 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f632813a990 0x7f63281461b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f63280a65d0 tx=0x7f6314015000).stop 2020-11-20T12:11:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632d56c700 1 -- 172.21.15.44:0/45293888 reap_dead start 2020-11-20T12:11:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632d56c700 1 -- 172.21.15.44:0/45293888 reap_dead start 2020-11-20T12:11:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 shutdown_connections 2020-11-20T12:11:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 >> 172.21.15.44:0/45293888 conn(0x7f6328062690 msgr2=0x7f63280a8690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:11:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 shutdown_connections 2020-11-20T12:11:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:11:54.599+0000 7f632efcf700 1 -- 172.21.15.44:0/45293888 wait complete. 2020-11-20T12:11:54.612 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:11:54.612 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:54.613 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:54.613 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:11:54.613 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 98, 2020-11-20T12:11:54.613 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:11:54.613 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:11:54.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:11:54.614 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.614 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:11:54.614 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:54.614 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:11:54.615 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:11:54.615 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:54.615 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:54.615 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:54.615 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.617 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:11:54.617 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:11:54.617 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:54.617 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:54.617 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:54.618 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:54.618 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:54.618 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:54.618 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:54.618 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:54.619 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.619 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:11:54.619 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:11:54.619 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:11:54.620 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:11:54.620 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:11:54.620 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:11:54.620 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:54.620 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:11:54.621 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:11:54.621 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:11:54.621 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:11:54.621 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:11:54.621 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:11:54.622 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:11:54.622 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:11:54.622 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:11:54.622 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:11:54.622 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:11:54.623 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:11:54.623 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:11:54.623 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:11:54.623 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:11:54.623 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:11:54.624 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.624 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:11:54.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.624 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:11:54.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.625 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:11:54.625 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:11:54.625 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:54.625 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:54.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:54.626 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:11:54.626 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.627 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:54.627 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:11:54.627 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.627 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.627 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.628 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:11:54.628 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:11:54.628 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:54.628 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:54.629 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:54.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.629 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:11:54.629 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:11:54.630 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:54.630 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:54.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.630 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:54.630 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:11:54.631 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:54.631 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:11:54.632 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.632 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.632 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:11:54.633 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:11:54.633 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:54.633 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:54.633 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:54.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.634 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:11:54.634 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:11:54.634 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:11:54.634 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:11:54.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.635 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:11:54.635 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:11:54.635 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.636 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:11:54.636 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:11:54.636 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:11:54.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.636 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.637 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:11:54.637 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:11:54.637 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:11:54.637 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:11:54.638 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:11:54.638 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.638 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.638 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:11:54.639 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:11:54.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.639 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.639 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.639 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:11:54.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.640 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.640 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:11:54.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.640 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.641 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.641 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:54.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.641 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.641 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:11:54.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.642 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.642 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.642 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:11:54.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.643 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:11:54.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.643 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.643 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.644 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:11:54.644 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.644 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:11:54.644 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:11:54.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:11:54.645 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:11:54.645 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:11:54.645 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:11:54.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:11:54.645 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:11:54.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:11:54.646 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:11:54.646 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:11:54.647 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:12:07.471 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:12:07.474 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:12:14.648 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:12:14.648 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:12:14.648 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:12:14.649 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:12:14.651 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:12:14.652 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:12:14.652 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:12:14.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.788+0000 7f342a4fe700 1 Processor -- start 2020-11-20T12:12:14.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.788+0000 7f342a4fe700 1 -- start start 2020-11-20T12:12:14.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.789+0000 7f342a4fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 0x7f3424142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.789+0000 7f342a4fe700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240a6260 con 0x7f342413a990 2020-11-20T12:12:14.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.789+0000 7f342a4fe700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240a6490 con 0x7f3424060330 2020-11-20T12:12:14.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.789+0000 7f342a4fe700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240a52e0 con 0x7f3424140810 2020-11-20T12:12:14.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.789+0000 7f34237fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f342413a990 0x7f342413ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43944/0 (socket says 172.21.15.44:43944) 2020-11-20T12:12:14.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.789+0000 7f34237fe700 1 -- 172.21.15.44:0/3644904921 learned_addr learned my addr 172.21.15.44:0/3644904921 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:14.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.789+0000 7f3428a9b700 1 --2- 172.21.15.44:0/3644904921 >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 0x7f3424142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810593145 0 0) 0x7f34240a6260 con 0x7f342413a990 2020-11-20T12:12:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3428a9b700 1 -- 172.21.15.44:0/3644904921 >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 msgr2=0x7f3424142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:12:14.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3428a9b700 1 -- 172.21.15.44:0/3644904921 >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 msgr2=0x7f3424142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3410003240 con 0x7f342413a990 2020-11-20T12:12:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3428a9b700 1 --2- 172.21.15.44:0/3644904921 >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 0x7f3424142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3428a9b700 1 --2- 172.21.15.44:0/3644904921 >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 0x7f3424142c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:14.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1308103766 0 0) 0x7f3410003240 con 0x7f342413a990 2020-11-20T12:12:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 msgr2=0x7f3424142c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 --2- 172.21.15.44:0/3644904921 >> v2:172.21.15.79:3300/0 conn(0x7f3424140810 0x7f3424142c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 >> v1:172.21.15.79:6789/0 conn(0x7f3424060330 legacy=0x7f34240a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240a6730 con 0x7f342413a990 2020-11-20T12:12:14.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f34180014e0 con 0x7f342413a990 2020-11-20T12:12:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 >> v1:172.21.15.44:6789/0 conn(0x7f342413a990 legacy=0x7f342413ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410003730 0x7f3410003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34100044e0 0x7f3410004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3410004e50 0x7f341000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240a52e0 con 0x7f3410003730 2020-11-20T12:12:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410004260 con 0x7f34100044e0 2020-11-20T12:12:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341000bc20 con 0x7f3410004e50 2020-11-20T12:12:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3423fff700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410003730 0x7f3410003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f34237fe700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34100044e0 0x7f3410004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3428a9b700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3410004e50 0x7f341000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3423fff700 1 -- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3410004e50 msgr2=0x7f341000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3423fff700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3410004e50 0x7f341000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3423fff700 1 -- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34100044e0 msgr2=0x7f3410004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:14.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3423fff700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34100044e0 0x7f3410004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3423fff700 1 -- 172.21.15.44:0/3644904921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410003240 con 0x7f3410003730 2020-11-20T12:12:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f34237fe700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f34100044e0 0x7f3410004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:12:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.790+0000 7f3428a9b700 1 -- 172.21.15.44:0/3644904921 reap_dead start 2020-11-20T12:12:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f3423fff700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410003730 0x7f3410003b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f3414001950 tx=0x7f3414007b90).ready entity=mon.0 client_cookie=44cb3cf2e2f102c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:14.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414007020 con 0x7f3410003730 2020-11-20T12:12:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414007180 con 0x7f3410003730 2020-11-20T12:12:14.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f3422ffd700 1 -- 172.21.15.44:0/3644904921 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341400c830 con 0x7f3410003730 2020-11-20T12:12:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f342a4fe700 1 -- 172.21.15.44:0/3644904921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410003730 msgr2=0x7f3410003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f342a4fe700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410003730 0x7f3410003b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f3414001950 tx=0x7f3414007b90).stop 2020-11-20T12:12:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f342a4fe700 1 -- 172.21.15.44:0/3644904921 shutdown_connections 2020-11-20T12:12:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f342a4fe700 1 --2- 172.21.15.44:0/3644904921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3410003730 0x7f3410003b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:14.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f342a4fe700 1 -- 172.21.15.44:0/3644904921 >> 172.21.15.44:0/3644904921 conn(0x7f3424062690 msgr2=0x7f34240a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f342a4fe700 1 -- 172.21.15.44:0/3644904921 shutdown_connections 2020-11-20T12:12:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.791+0000 7f342a4fe700 1 -- 172.21.15.44:0/3644904921 wait complete. 2020-11-20T12:12:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.792+0000 7f342a4fe700 1 Processor -- start 2020-11-20T12:12:14.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.792+0000 7f342a4fe700 1 -- start start 2020-11-20T12:12:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f342a4fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 0x7f3424135f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f342a4fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3424137190 0x7f34240ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f342a4fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34241493a0 0x7f3424149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f342a4fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3424136a50 con 0x7f3424137190 2020-11-20T12:12:14.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f342a4fe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3424136cf0 con 0x7f3410003730 2020-11-20T12:12:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f342a4fe700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3424137610 con 0x7f34241493a0 2020-11-20T12:12:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f34237fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3424137190 0x7f34240ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3423fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 0x7f3424135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3423fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 0x7f3424135f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60488/0 (socket says 172.21.15.44:60488) 2020-11-20T12:12:14.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3423fff700 1 -- 172.21.15.44:0/1912996876 learned_addr learned my addr 172.21.15.44:0/1912996876 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f34237fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3424137190 0x7f34240ab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40208/0 (socket says 172.21.15.44:40208) 2020-11-20T12:12:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3428a9b700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34241493a0 0x7f3424149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3423fff700 1 -- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 msgr2=0x7f3424135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:12:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3423fff700 1 -- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 msgr2=0x7f3424135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:12:14.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3423fff700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 0x7f3424135f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f3423fff700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 0x7f3424135f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.793+0000 7f34237fe700 1 -- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34241493a0 msgr2=0x7f3424149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f34237fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34241493a0 0x7f3424149800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f34237fe700 1 -- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 msgr2=0x7f3424135f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:12:14.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f34237fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 0x7f3424135f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f34237fe700 1 -- 172.21.15.44:0/1912996876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424138120 con 0x7f3424137190 2020-11-20T12:12:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f34237fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3424137190 0x7f34240ab800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f342405f0e0 tx=0x7f34240a68c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f3403fff700 1 -- 172.21.15.44:0/1912996876 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341800f9f0 con 0x7f3424137190 2020-11-20T12:12:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f3403fff700 1 -- 172.21.15.44:0/1912996876 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341800fb50 con 0x7f3424137190 2020-11-20T12:12:14.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240abcf0 con 0x7f3424137190 2020-11-20T12:12:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f3403fff700 1 -- 172.21.15.44:0/1912996876 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418012350 con 0x7f3424137190 2020-11-20T12:12:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.794+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3424067ab0 con 0x7f3424137190 2020-11-20T12:12:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.795+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3424061650 con 0x7f3424137190 2020-11-20T12:12:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.795+0000 7f3403fff700 1 -- 172.21.15.44:0/1912996876 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f34180126c0 con 0x7f3424137190 2020-11-20T12:12:14.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.796+0000 7f3403fff700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f33fc056d50 0x7f33fc059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:14.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.796+0000 7f3403fff700 1 -- 172.21.15.44:0/1912996876 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f341806f950 con 0x7f3424137190 2020-11-20T12:12:14.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.796+0000 7f3423fff700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f33fc056d50 0x7f33fc059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:14.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.796+0000 7f3423fff700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f33fc056d50 0x7f33fc059210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f3414001010 tx=0x7f3414015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:14.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:14.799+0000 7f3403fff700 1 -- 172.21.15.44:0/1912996876 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3418044c20 con 0x7f3424137190 2020-11-20T12:12:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.071+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f34240446d0 con 0x7f3424137190 2020-11-20T12:12:15.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.071+0000 7f3403fff700 1 -- 172.21.15.44:0/1912996876 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f3418019030 con 0x7f3424137190 2020-11-20T12:12:15.075 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":98,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:12:15.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f33fc056d50 msgr2=0x7f33fc059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:15.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f33fc056d50 0x7f33fc059210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f3414001010 tx=0x7f3414015000).stop 2020-11-20T12:12:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3424137190 msgr2=0x7f34240ab800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3424137190 0x7f34240ab800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f342405f0e0 tx=0x7f34240a68c0).stop 2020-11-20T12:12:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 shutdown_connections 2020-11-20T12:12:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f33fc056d50 0x7f33fc059210 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:15.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f34241493a0 0x7f3424149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3410003730 0x7f3424135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 --2- 172.21.15.44:0/1912996876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3424137190 0x7f34240ab800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.073+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 >> 172.21.15.44:0/1912996876 conn(0x7f3424062690 msgr2=0x7f34240a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:15.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.074+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 shutdown_connections 2020-11-20T12:12:15.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:15.074+0000 7f342a4fe700 1 -- 172.21.15.44:0/1912996876 wait complete. 2020-11-20T12:12:18.089 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:12:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.227+0000 7f1648c7e700 1 Processor -- start 2020-11-20T12:12:18.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.227+0000 7f1648c7e700 1 -- start start 2020-11-20T12:12:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f1648c7e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f1648c7e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f1648c7e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440a6260 con 0x7f164413ae80 2020-11-20T12:12:18.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f1648c7e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440a52e0 con 0x7f16440a4c30 2020-11-20T12:12:18.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f1648c7e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16440a6490 con 0x7f1644060330 2020-11-20T12:12:18.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f163bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:18.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f163bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40218/0 (socket says 172.21.15.44:40218) 2020-11-20T12:12:18.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 learned_addr learned my addr 172.21.15.44:0/3827824916 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:18.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:18.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.228+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:18.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.229+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:18.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.229+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:18.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.229+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:18.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.429+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:18.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.429+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:18.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.429+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:18.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.429+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:18.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.429+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:12:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.430+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:18.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.430+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.430+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.430+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:18.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.430+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:12:18.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.830+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.830+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.830+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.830+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.830+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:12:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.831+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.831+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:18.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.831+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:18.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.831+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:18.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:18.831+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:12:19.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.631+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:19.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.631+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:19.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.631+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:19.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.631+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:19.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.631+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:12:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.632+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.632+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.632+0000 7f164259c700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.632+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:19.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:19.632+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:12:21.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f16440a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:21.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f16440a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:21.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 msgr2=0x7f164413a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:21.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.44:3300/0 conn(0x7f16440a4c30 0x7f164413a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:21.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 -- 172.21.15.44:0/3827824916 >> v1:172.21.15.79:6789/0 conn(0x7f164413ae80 legacy=0x7f1644136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:21.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:21.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 -- 172.21.15.44:0/3827824916 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440a6730 con 0x7f1628003390 2020-11-20T12:12:21.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f1640d99700 1 -- 172.21.15.44:0/3827824916 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440a52e0 con 0x7f1628007040 2020-11-20T12:12:21.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.228+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:21.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.229+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:21.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.229+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:21.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.229+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:21.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.229+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.430+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.430+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.430+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.430+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.430+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:12:21.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.831+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:21.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.831+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:21.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.831+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:21.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.831+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:21.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:21.831+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:12:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.632+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.632+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:23.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.632+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:23.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.632+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:23.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.632+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:12:23.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.783+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239875870 0 0) 0x7f16440a6730 con 0x7f1628003390 2020-11-20T12:12:23.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.783+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1624003240 con 0x7f1628003390 2020-11-20T12:12:23.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 486197451 0 0) 0x7f1624003240 con 0x7f1628003390 2020-11-20T12:12:23.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 msgr2=0x7f1628009500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:23.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 --2- 172.21.15.44:0/3827824916 >> v2:172.21.15.79:3300/0 conn(0x7f1628007040 0x7f1628009500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162800ac20 con 0x7f1628003390 2020-11-20T12:12:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f163400e2d0 con 0x7f1628003390 2020-11-20T12:12:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 >> v1:172.21.15.44:6789/0 conn(0x7f1628003390 legacy=0x7f1628005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f1624003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f16240044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16240049d0 0x7f1624004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440a52e0 con 0x7f1624003c70 2020-11-20T12:12:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1624004260 con 0x7f16240049d0 2020-11-20T12:12:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162400b730 con 0x7f1644060330 2020-11-20T12:12:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.803+0000 7f1642d9d700 1 -- 172.21.15.44:0/3827824916 reap_dead start 2020-11-20T12:12:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f16240044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f1642d9d700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16240049d0 0x7f1624004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f164259c700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f1624003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 msgr2=0x7f1624003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f1624003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16240049d0 msgr2=0x7f1624004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16240049d0 0x7f1624004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f163bfff700 1 -- 172.21.15.44:0/3827824916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624003240 con 0x7f1624003c70 2020-11-20T12:12:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.804+0000 7f163bfff700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f16240044e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f162c006cc0 tx=0x7f162c008040).ready entity=mon.0 client_cookie=f4d37b117d1f54d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.807+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c01e070 con 0x7f1624003c70 2020-11-20T12:12:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.807+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c017d00 con 0x7f1624003c70 2020-11-20T12:12:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.807+0000 7f1641d9b700 1 -- 172.21.15.44:0/3827824916 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c023630 con 0x7f1624003c70 2020-11-20T12:12:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 -- 172.21.15.44:0/3827824916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 msgr2=0x7f16240044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f16240044e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f162c006cc0 tx=0x7f162c008040).stop 2020-11-20T12:12:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 -- 172.21.15.44:0/3827824916 shutdown_connections 2020-11-20T12:12:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16240049d0 0x7f1624004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1644060330 0x7f1624003730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 --2- 172.21.15.44:0/3827824916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f16240044e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 -- 172.21.15.44:0/3827824916 >> 172.21.15.44:0/3827824916 conn(0x7f1644062690 msgr2=0x7f16440a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 -- 172.21.15.44:0/3827824916 shutdown_connections 2020-11-20T12:12:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.808+0000 7f1648c7e700 1 -- 172.21.15.44:0/3827824916 wait complete. 2020-11-20T12:12:23.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.809+0000 7f1648c7e700 1 Processor -- start 2020-11-20T12:12:23.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.809+0000 7f1648c7e700 1 -- start start 2020-11-20T12:12:23.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1648c7e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f1644134500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1648c7e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16240049d0 0x7f16440accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f1644134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f1644134500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40250/0 (socket says 172.21.15.44:40250) 2020-11-20T12:12:23.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f163bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16240049d0 0x7f16440accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1648c7e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 0x7f16440ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f163bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16240049d0 0x7f16440accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:51688/0 (socket says 172.21.15.44:51688) 2020-11-20T12:12:23.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1648c7e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440adc50 con 0x7f1624003c70 2020-11-20T12:12:23.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f163bfff700 1 -- 172.21.15.44:0/1344917830 learned_addr learned my addr 172.21.15.44:0/1344917830 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:23.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164409f880 con 0x7f1644060330 2020-11-20T12:12:23.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164409f340 con 0x7f16240049d0 2020-11-20T12:12:23.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1642d9d700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 0x7f16440ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1642d9d700 1 -- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 msgr2=0x7f16440ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:12:23.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1642d9d700 1 -- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 msgr2=0x7f16440ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:12:23.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 -- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16240049d0 msgr2=0x7f16440accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16240049d0 0x7f16440accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1642d9d700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 0x7f16440ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:23.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 -- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 msgr2=0x7f16440ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f1642d9d700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 0x7f16440ad1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:23.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 0x7f16440ad1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 -- 172.21.15.44:0/1344917830 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164409f5e0 con 0x7f1624003c70 2020-11-20T12:12:23.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f164259c700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f1644134500 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f1628006f50 tx=0x7f1634008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.810+0000 7f163a7fc700 1 -- 172.21.15.44:0/1344917830 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634019070 con 0x7f1624003c70 2020-11-20T12:12:23.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.811+0000 7f163a7fc700 1 -- 172.21.15.44:0/1344917830 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1634013de0 con 0x7f1624003c70 2020-11-20T12:12:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.811+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164409f030 con 0x7f1624003c70 2020-11-20T12:12:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.811+0000 7f163a7fc700 1 -- 172.21.15.44:0/1344917830 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163401e640 con 0x7f1624003c70 2020-11-20T12:12:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.811+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164409d920 con 0x7f1624003c70 2020-11-20T12:12:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.812+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16440ad6d0 con 0x7f1624003c70 2020-11-20T12:12:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.813+0000 7f163a7fc700 1 -- 172.21.15.44:0/1344917830 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1634025070 con 0x7f1624003c70 2020-11-20T12:12:23.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.813+0000 7f163a7fc700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1620056d20 0x7f16200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.813+0000 7f163a7fc700 1 -- 172.21.15.44:0/1344917830 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1634071490 con 0x7f1624003c70 2020-11-20T12:12:23.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.813+0000 7f163bfff700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1620056d20 0x7f16200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.813+0000 7f163bfff700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1620056d20 0x7f16200591e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f162c006e60 tx=0x7f162c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:22.817+0000 7f163a7fc700 1 -- 172.21.15.44:0/1344917830 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f16340467e0 con 0x7f1624003c70 2020-11-20T12:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.073+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f16440aca30 con 0x7f1624003c70 2020-11-20T12:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.074+0000 7f163a7fc700 1 -- 172.21.15.44:0/1344917830 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f1634049e00 con 0x7f1624003c70 2020-11-20T12:12:23.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1620056d20 msgr2=0x7f16200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1620056d20 0x7f16200591e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f162c006e60 tx=0x7f162c008040).stop 2020-11-20T12:12:23.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 msgr2=0x7f1644134500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f1644134500 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f1628006f50 tx=0x7f1634008040).stop 2020-11-20T12:12:23.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 shutdown_connections 2020-11-20T12:12:23.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1620056d20 0x7f16200591e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1644060330 0x7f16440ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1624003c70 0x7f1644134500 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 --2- 172.21.15.44:0/1344917830 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16240049d0 0x7f16440accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 >> 172.21.15.44:0/1344917830 conn(0x7f1644062690 msgr2=0x7f16441377d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 shutdown_connections 2020-11-20T12:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.076+0000 7f1648c7e700 1 -- 172.21.15.44:0/1344917830 wait complete. 2020-11-20T12:12:23.127 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:12:23.128 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:12:23.128 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:12:23.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.276+0000 7f307c7da700 1 Processor -- start 2020-11-20T12:12:23.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.276+0000 7f307c7da700 1 -- start start 2020-11-20T12:12:23.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307c7da700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f30740aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307c7da700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 0x7f3074137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307c7da700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3074067470 con 0x7f30740a1a40 2020-11-20T12:12:23.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307c7da700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30740ab240 con 0x7f3074137b30 2020-11-20T12:12:23.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307c7da700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3074067710 con 0x7f3074066de0 2020-11-20T12:12:23.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307a576700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f30740a1a40 0x7f30740666d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44006/0 (socket says 172.21.15.44:44006) 2020-11-20T12:12:23.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307a576700 1 -- 172.21.15.44:0/2838482828 learned_addr learned my addr 172.21.15.44:0/2838482828 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:23.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f3079d75700 1 --2- 172.21.15.44:0/2838482828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f30740aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.277+0000 7f307ad77700 1 --2- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 0x7f3074137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079574700 1 -- 172.21.15.44:0/2838482828 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386146871 0 0) 0x7f3074067470 con 0x7f30740a1a40 2020-11-20T12:12:23.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f307ad77700 1 -- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 msgr2=0x7f3074137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:12:23.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f307ad77700 1 -- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 msgr2=0x7f3074137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f307ad77700 1 --2- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 0x7f3074137fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079574700 1 -- 172.21.15.44:0/2838482828 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f305c003240 con 0x7f30740a1a40 2020-11-20T12:12:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f307ad77700 1 --2- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 0x7f3074137fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:23.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079d75700 1 -- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 msgr2=0x7f3074137fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079d75700 1 --2- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 0x7f3074137fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079d75700 1 -- 172.21.15.44:0/2838482828 >> v1:172.21.15.44:6789/0 conn(0x7f30740a1a40 legacy=0x7f30740666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079d75700 1 -- 172.21.15.44:0/2838482828 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740679b0 con 0x7f3074066de0 2020-11-20T12:12:23.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079d75700 1 --2- 172.21.15.44:0/2838482828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f30740aad50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f306400b9a0 tx=0x7f3064006df0).ready entity=mon.2 client_cookie=821242ac71679aae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079574700 1 -- 172.21.15.44:0/2838482828 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30640097b0 con 0x7f3074066de0 2020-11-20T12:12:23.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079574700 1 -- 172.21.15.44:0/2838482828 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064009910 con 0x7f3074066de0 2020-11-20T12:12:23.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.278+0000 7f3079574700 1 -- 172.21.15.44:0/2838482828 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064009c90 con 0x7f3074066de0 2020-11-20T12:12:23.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 -- 172.21.15.44:0/2838482828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 msgr2=0x7f30740aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 --2- 172.21.15.44:0/2838482828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f30740aad50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f306400b9a0 tx=0x7f3064006df0).stop 2020-11-20T12:12:23.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 -- 172.21.15.44:0/2838482828 shutdown_connections 2020-11-20T12:12:23.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 --2- 172.21.15.44:0/2838482828 >> v2:172.21.15.79:3300/0 conn(0x7f3074137b30 0x7f3074137fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 --2- 172.21.15.44:0/2838482828 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f30740aad50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 -- 172.21.15.44:0/2838482828 >> 172.21.15.44:0/2838482828 conn(0x7f30740af030 msgr2=0x7f30740af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 -- 172.21.15.44:0/2838482828 shutdown_connections 2020-11-20T12:12:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.279+0000 7f307c7da700 1 -- 172.21.15.44:0/2838482828 wait complete. 2020-11-20T12:12:23.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.280+0000 7f307c7da700 1 Processor -- start 2020-11-20T12:12:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.280+0000 7f307c7da700 1 -- start start 2020-11-20T12:12:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307c7da700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f3074062760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307c7da700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30740a1a40 0x7f30741363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307c7da700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 0x7f307413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307c7da700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741368d0 con 0x7f30740a1a40 2020-11-20T12:12:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307c7da700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741361b0 con 0x7f3074137b30 2020-11-20T12:12:23.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307c7da700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3074062300 con 0x7f3074066de0 2020-11-20T12:12:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f3079d75700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30740a1a40 0x7f30741363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f3079d75700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30740a1a40 0x7f30741363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40266/0 (socket says 172.21.15.44:40266) 2020-11-20T12:12:23.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f3079d75700 1 -- 172.21.15.44:0/3983586209 learned_addr learned my addr 172.21.15.44:0/3983586209 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307a576700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f3074062760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307ad77700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 0x7f307413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307ad77700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 msgr2=0x7f307413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:12:23.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307ad77700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 msgr2=0x7f307413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:12:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307ad77700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 0x7f307413f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f3079d75700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 msgr2=0x7f3074062760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f307ad77700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 0x7f307413f500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.281+0000 7f3079d75700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3074066de0 0x7f3074062760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f3079d75700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 msgr2=0x7f307413f500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:12:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f3079d75700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3074137b30 0x7f307413f500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f3079d75700 1 -- 172.21.15.44:0/3983586209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30741401b0 con 0x7f30740a1a40 2020-11-20T12:12:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f3079d75700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30740a1a40 0x7f30741363e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3064006e30 tx=0x7f3064015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011dd0 con 0x7f30740a1a40 2020-11-20T12:12:23.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306401e460 con 0x7f30740a1a40 2020-11-20T12:12:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306401e7e0 con 0x7f30740a1a40 2020-11-20T12:12:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30740636d0 con 0x7f30740a1a40 2020-11-20T12:12:23.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.282+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3074135f00 con 0x7f30740a1a40 2020-11-20T12:12:23.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.283+0000 7f307c7da700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413e380 0x7f307413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.283+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740624c0 con 0x7f307413e380 2020-11-20T12:12:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.283+0000 7f307a576700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413e380 0x7f307413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.283+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3064025030 con 0x7f30740a1a40 2020-11-20T12:12:23.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.283+0000 7f307a576700 1 -- 172.21.15.44:0/3983586209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f307405c2a0 con 0x7f307413e380 2020-11-20T12:12:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.283+0000 7f307a576700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413e380 0x7f307413e760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f30700055a0 tx=0x7f3070009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f304c056d20 0x7f304c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f30640717e0 con 0x7f30740a1a40 2020-11-20T12:12:23.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f307ad77700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f304c056d20 0x7f304c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700070d0 con 0x7f307413e380 2020-11-20T12:12:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f3070007230 con 0x7f307413e380 2020-11-20T12:12:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413e380 msgr2=0x7f307413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413e380 0x7f307413e760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f30700055a0 tx=0x7f3070009040).stop 2020-11-20T12:12:23.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413e380 msgr2=0x7f307413e760 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f30627fc700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413e380 0x7f307413e760 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0x7f30700055a0 tx=0x7f3070009040).stop 2020-11-20T12:12:23.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.284+0000 7f307ad77700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f304c056d20 0x7f304c0591e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f306c006fd0 tx=0x7f306c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.293+0000 7f307c7da700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413ec50 0x7f3074138800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074067230 con 0x7f307413ec50 2020-11-20T12:12:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f307a576700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413ec50 0x7f3074138800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f307a576700 1 -- 172.21.15.44:0/3983586209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f30740446d0 con 0x7f307413ec50 2020-11-20T12:12:23.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f307a576700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413ec50 0x7f3074138800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f30700055a0 tx=0x7f3070018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000d990 con 0x7f307413ec50 2020-11-20T12:12:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4438 (secure 0 0 0) 0x7f30700055d0 con 0x7f307413ec50 2020-11-20T12:12:23.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413ec50 msgr2=0x7f3074138800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f30627fc700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413ec50 0x7f3074138800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f30700055a0 tx=0x7f3070018000).stop 2020-11-20T12:12:23.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f30627fc700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413ec50 msgr2=0x7f3074138800 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.294+0000 7f30627fc700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f307413ec50 0x7f3074138800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f304c056d20 msgr2=0x7f304c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f304c056d20 0x7f304c0591e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f306c006fd0 tx=0x7f306c008040).stop 2020-11-20T12:12:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30740a1a40 msgr2=0x7f30741363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 --2- 172.21.15.44:0/3983586209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30740a1a40 0x7f30741363e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3064006e30 tx=0x7f3064015000).stop 2020-11-20T12:12:23.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307ad77700 1 -- 172.21.15.44:0/3983586209 reap_dead start 2020-11-20T12:12:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307ad77700 1 -- 172.21.15.44:0/3983586209 reap_dead start 2020-11-20T12:12:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 shutdown_connections 2020-11-20T12:12:23.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 >> 172.21.15.44:0/3983586209 conn(0x7f30740af030 msgr2=0x7f30740aba60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 shutdown_connections 2020-11-20T12:12:23.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.295+0000 7f307c7da700 1 -- 172.21.15.44:0/3983586209 wait complete. 2020-11-20T12:12:23.321 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:23.322 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:23.323 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:23.323 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:12:23.323 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 106, 2020-11-20T12:12:23.324 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:23.324 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:23.324 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:12:23.325 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:12:23.325 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.326 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:12:23.327 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:23.327 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:23.327 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:23.327 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.328 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.328 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.329 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.329 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.330 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.330 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.330 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.331 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:23.331 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:23.332 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.333 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.333 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.334 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.334 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.334 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.334 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.335 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.335 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:23.336 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:23.336 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:23.336 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:23.337 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:23.337 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:23.337 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:23.338 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:23.338 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:23.338 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:23.339 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:23.339 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:23.339 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:23.340 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:23.340 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:23.340 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.341 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.343 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:23.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:23.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.345 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:23.345 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:23.345 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.345 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.346 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:23.347 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.348 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.348 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:23.349 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.349 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.350 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:23.350 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:23.350 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.351 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.351 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:23.352 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:23.353 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.353 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.354 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.354 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:23.354 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.356 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:23.357 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.358 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.358 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:23.359 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:23.359 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.359 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.360 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:23.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:23.361 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.362 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.363 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:23.363 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.365 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:23.366 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.366 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.367 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:23.368 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:23.368 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.368 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.369 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.370 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.371 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:23.371 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:23.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.372 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.372 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.372 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:23.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.373 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.374 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:12:23.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.374 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.375 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.375 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:23.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.376 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.376 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:23.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.377 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.377 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.377 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:23.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.378 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:23.379 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:23.380 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:12:23.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.523+0000 7f29a6048700 1 Processor -- start 2020-11-20T12:12:23.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.523+0000 7f29a6048700 1 -- start start 2020-11-20T12:12:23.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a6048700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a9880 0x7f29a00a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a6048700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 0x7f29a00a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a6048700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a00a6b80 con 0x7f29a00a7630 2020-11-20T12:12:23.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a6048700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a00a71b0 con 0x7f29a00a6600 2020-11-20T12:12:23.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a6048700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00a6e20 con 0x7f29a00a9880 2020-11-20T12:12:23.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a4845700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 0x7f29a00a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a5046700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a9880 0x7f29a00a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a5847700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f29a00a7630 0x7f29a00a7ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:46904/0 (socket says 172.21.15.44:46904) 2020-11-20T12:12:23.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a4845700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 0x7f29a00a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60560/0 (socket says 172.21.15.44:60560) 2020-11-20T12:12:23.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a5847700 1 -- 172.21.15.44:0/4105592092 learned_addr learned my addr 172.21.15.44:0/4105592092 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:23.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a4845700 1 -- 172.21.15.44:0/4105592092 >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 msgr2=0x7f29a00a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:23.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a4845700 1 -- 172.21.15.44:0/4105592092 >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 msgr2=0x7f29a00a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:23.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.524+0000 7f29a4845700 1 --2- 172.21.15.44:0/4105592092 >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 0x7f29a00a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:23.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f29a4845700 1 --2- 172.21.15.44:0/4105592092 >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 0x7f29a00a8af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:23.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f29a5046700 1 -- 172.21.15.44:0/4105592092 >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 msgr2=0x7f29a00a8af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:23.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f29a5046700 1 --2- 172.21.15.44:0/4105592092 >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 0x7f29a00a8af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f29a5046700 1 -- 172.21.15.44:0/4105592092 >> v1:172.21.15.79:6789/0 conn(0x7f29a00a7630 legacy=0x7f29a00a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f29a5046700 1 -- 172.21.15.44:0/4105592092 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00a4a50 con 0x7f29a00a9880 2020-11-20T12:12:23.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f29a5046700 1 --2- 172.21.15.44:0/4105592092 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a9880 0x7f29a00a4ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f299c00b9a0 tx=0x7f299c006df0).ready entity=mon.2 client_cookie=cd2889320befa162 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f2993fff700 1 -- 172.21.15.44:0/4105592092 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c009840 con 0x7f29a00a9880 2020-11-20T12:12:23.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f2993fff700 1 -- 172.21.15.44:0/4105592092 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c0099a0 con 0x7f29a00a9880 2020-11-20T12:12:23.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.525+0000 7f2993fff700 1 -- 172.21.15.44:0/4105592092 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c009d20 con 0x7f29a00a9880 2020-11-20T12:12:23.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 -- 172.21.15.44:0/4105592092 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a9880 msgr2=0x7f29a00a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 --2- 172.21.15.44:0/4105592092 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a9880 0x7f29a00a4ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f299c00b9a0 tx=0x7f299c006df0).stop 2020-11-20T12:12:23.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 -- 172.21.15.44:0/4105592092 shutdown_connections 2020-11-20T12:12:23.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 --2- 172.21.15.44:0/4105592092 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a9880 0x7f29a00a4ef0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 --2- 172.21.15.44:0/4105592092 >> v2:172.21.15.79:3300/0 conn(0x7f29a00a6600 0x7f29a00a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 -- 172.21.15.44:0/4105592092 >> 172.21.15.44:0/4105592092 conn(0x7f29a0061b60 msgr2=0x7f29a0061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 -- 172.21.15.44:0/4105592092 shutdown_connections 2020-11-20T12:12:23.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.526+0000 7f29a6048700 1 -- 172.21.15.44:0/4105592092 wait complete. 2020-11-20T12:12:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.527+0000 7f29a6048700 1 Processor -- start 2020-11-20T12:12:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.527+0000 7f29a6048700 1 -- start start 2020-11-20T12:12:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a6048700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a6600 0x7f29a0132740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a6048700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a00a7630 0x7f29a0066670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a6048700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a00a9880 0x7f29a0066b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a6048700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0132db0 con 0x7f29a00a7630 2020-11-20T12:12:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a6048700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0067050 con 0x7f29a00a9880 2020-11-20T12:12:23.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a6048700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0130a20 con 0x7f29a00a6600 2020-11-20T12:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a4845700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a00a7630 0x7f29a0066670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a4845700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a00a7630 0x7f29a0066670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40284/0 (socket says 172.21.15.44:40284) 2020-11-20T12:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a5046700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a6600 0x7f29a0132740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a4845700 1 -- 172.21.15.44:0/2550788056 learned_addr learned my addr 172.21.15.44:0/2550788056 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.528+0000 7f29a5847700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a00a9880 0x7f29a0066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a4845700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a6600 msgr2=0x7f29a0132740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a4845700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f29a00a6600 0x7f29a0132740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a4845700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a00a9880 msgr2=0x7f29a0066b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a4845700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a00a9880 0x7f29a0066b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a4845700 1 -- 172.21.15.44:0/2550788056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0130cc0 con 0x7f29a00a7630 2020-11-20T12:12:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a4845700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a00a7630 0x7f29a0066670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f299400db60 tx=0x7f2994008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299400de70 con 0x7f29a00a7630 2020-11-20T12:12:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2994009790 con 0x7f29a00a7630 2020-11-20T12:12:23.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994009b30 con 0x7f29a00a7630 2020-11-20T12:12:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a0130f60 con 0x7f29a00a7630 2020-11-20T12:12:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.529+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a014aa30 con 0x7f29a00a7630 2020-11-20T12:12:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.530+0000 7f29a6048700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014ad10 0x7f29a014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.530+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0061180 con 0x7f29a014ad10 2020-11-20T12:12:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.530+0000 7f29a5046700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014ad10 0x7f29a014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29a5046700 1 -- 172.21.15.44:0/2550788056 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29a00ac9e0 con 0x7f29a014ad10 2020-11-20T12:12:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29a5046700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014ad10 0x7f29a014b0f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f299c00b9a0 tx=0x7f299c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2994013480 con 0x7f29a00a7630 2020-11-20T12:12:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2978056d20 0x7f29780591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2994070380 con 0x7f29a00a7630 2020-11-20T12:12:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c01f440 con 0x7f29a014ad10 2020-11-20T12:12:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29a5847700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2978056d20 0x7f29780591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f299c01e420 con 0x7f29a014ad10 2020-11-20T12:12:23.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014ad10 msgr2=0x7f29a014b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014ad10 0x7f29a014b0f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f299c00b9a0 tx=0x7f299c015000).stop 2020-11-20T12:12:23.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014ad10 msgr2=0x7f29a014b0f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:23.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.531+0000 7f29917fa700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014ad10 0x7f29a014b0f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.532+0000 7f29a5847700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2978056d20 0x7f29780591e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f29a0132430 tx=0x7f2998007f20).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.539+0000 7f29a6048700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014b5e0 0x7f29a01428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.539+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0134cf0 con 0x7f29a014b5e0 2020-11-20T12:12:23.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.539+0000 7f29a5046700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014b5e0 0x7f29a01428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29a5046700 1 -- 172.21.15.44:0/2550788056 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f29a0130830 con 0x7f29a014b5e0 2020-11-20T12:12:23.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29a5046700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014b5e0 0x7f29a01428b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f299c006e30 tx=0x7f299c021070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c020cd0 con 0x7f29a014b5e0 2020-11-20T12:12:23.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f299c011cd0 con 0x7f29a014b5e0 2020-11-20T12:12:23.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014b5e0 msgr2=0x7f29a01428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29917fa700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014b5e0 0x7f29a01428b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f299c006e30 tx=0x7f299c021070).stop 2020-11-20T12:12:23.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29917fa700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014b5e0 msgr2=0x7f29a01428b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:23.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.540+0000 7f29917fa700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f29a014b5e0 0x7f29a01428b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2978056d20 msgr2=0x7f29780591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2978056d20 0x7f29780591e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f29a0132430 tx=0x7f2998007f20).stop 2020-11-20T12:12:23.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a00a7630 msgr2=0x7f29a0066670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 --2- 172.21.15.44:0/2550788056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29a00a7630 0x7f29a0066670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f299400db60 tx=0x7f2994008040).stop 2020-11-20T12:12:23.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a5847700 1 -- 172.21.15.44:0/2550788056 reap_dead start 2020-11-20T12:12:23.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a5847700 1 -- 172.21.15.44:0/2550788056 reap_dead start 2020-11-20T12:12:23.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 shutdown_connections 2020-11-20T12:12:23.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 >> 172.21.15.44:0/2550788056 conn(0x7f29a0061b60 msgr2=0x7f29a00ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 shutdown_connections 2020-11-20T12:12:23.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.541+0000 7f29a6048700 1 -- 172.21.15.44:0/2550788056 wait complete. 2020-11-20T12:12:23.555 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:12:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 106, 2020-11-20T12:12:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:12:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:12:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:12:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:23.587 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:23.588 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:12:23.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.735+0000 7f1c3222b700 1 Processor -- start 2020-11-20T12:12:23.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.735+0000 7f1c3222b700 1 -- start start 2020-11-20T12:12:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.736+0000 7f1c3222b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1c2c0a8250 0x7f1c2c138030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.736+0000 7f1c3222b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c0a3620 con 0x7f1c2c0a3f90 2020-11-20T12:12:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.736+0000 7f1c3222b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c0a4400 con 0x7f1c2c068f60 2020-11-20T12:12:23.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.736+0000 7f1c3222b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c0a3380 con 0x7f1c2c0a8250 2020-11-20T12:12:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.736+0000 7f1c31a2a700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1c2c0a3f90 0x7f1c2c145250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44046/0 (socket says 172.21.15.44:44046) 2020-11-20T12:12:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.736+0000 7f1c31a2a700 1 -- 172.21.15.44:0/3607731608 learned_addr learned my addr 172.21.15.44:0/3607731608 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.736+0000 7f1c30a28700 1 --2- 172.21.15.44:0/3607731608 >> v2:172.21.15.79:3300/0 conn(0x7f1c2c0a8250 0x7f1c2c138030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c1bfff700 1 -- 172.21.15.44:0/3607731608 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259497345 0 0) 0x7f1c2c0a3620 con 0x7f1c2c0a3f90 2020-11-20T12:12:23.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c1bfff700 1 -- 172.21.15.44:0/3607731608 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c14003240 con 0x7f1c2c0a3f90 2020-11-20T12:12:23.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c30a28700 1 -- 172.21.15.44:0/3607731608 >> v1:172.21.15.79:6789/0 conn(0x7f1c2c068f60 legacy=0x7f1c2c137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c30a28700 1 -- 172.21.15.44:0/3607731608 >> v1:172.21.15.44:6789/0 conn(0x7f1c2c0a3f90 legacy=0x7f1c2c145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c30a28700 1 -- 172.21.15.44:0/3607731608 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0a9150 con 0x7f1c2c0a8250 2020-11-20T12:12:23.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c30a28700 1 --2- 172.21.15.44:0/3607731608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c138030 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1c1c00b9a0 tx=0x7f1c1c006df0).ready entity=mon.1 client_cookie=f26cc281bafc8a62 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c1bfff700 1 -- 172.21.15.44:0/3607731608 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c009840 con 0x7f1c2c0a8250 2020-11-20T12:12:23.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c1bfff700 1 -- 172.21.15.44:0/3607731608 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0099a0 con 0x7f1c2c0a8250 2020-11-20T12:12:23.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.737+0000 7f1c1bfff700 1 -- 172.21.15.44:0/3607731608 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c009d20 con 0x7f1c2c0a8250 2020-11-20T12:12:23.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.738+0000 7f1c3222b700 1 -- 172.21.15.44:0/3607731608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c0a8250 msgr2=0x7f1c2c138030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.738+0000 7f1c3222b700 1 --2- 172.21.15.44:0/3607731608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c138030 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1c1c00b9a0 tx=0x7f1c1c006df0).stop 2020-11-20T12:12:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.738+0000 7f1c3222b700 1 -- 172.21.15.44:0/3607731608 shutdown_connections 2020-11-20T12:12:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.738+0000 7f1c3222b700 1 --2- 172.21.15.44:0/3607731608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c138030 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.738+0000 7f1c3222b700 1 -- 172.21.15.44:0/3607731608 >> 172.21.15.44:0/3607731608 conn(0x7f1c2c0ad520 msgr2=0x7f1c2c0ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.738+0000 7f1c3222b700 1 -- 172.21.15.44:0/3607731608 shutdown_connections 2020-11-20T12:12:23.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.738+0000 7f1c3222b700 1 -- 172.21.15.44:0/3607731608 wait complete. 2020-11-20T12:12:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 Processor -- start 2020-11-20T12:12:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 -- start start 2020-11-20T12:12:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c068f60 0x7f1c2c13a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c0a3f90 0x7f1c2c14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c13dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c13bce0 con 0x7f1c2c0a8250 2020-11-20T12:12:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c13bf80 con 0x7f1c2c068f60 2020-11-20T12:12:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.739+0000 7f1c3222b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c13c220 con 0x7f1c2c0a3f90 2020-11-20T12:12:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c13dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c13dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40304/0 (socket says 172.21.15.44:40304) 2020-11-20T12:12:23.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 -- 172.21.15.44:0/2123392005 learned_addr learned my addr 172.21.15.44:0/2123392005 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:23.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c30a28700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c0a3f90 0x7f1c2c14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31229700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c068f60 0x7f1c2c13a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c0a3f90 msgr2=0x7f1c2c14d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c0a3f90 0x7f1c2c14d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c068f60 msgr2=0x7f1c2c13a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1c2c068f60 0x7f1c2c13a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 -- 172.21.15.44:0/2123392005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c14d970 con 0x7f1c2c0a8250 2020-11-20T12:12:23.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c31a2a700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c13dce0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1c24001420 tx=0x7f1c24007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24005b80 con 0x7f1c2c0a8250 2020-11-20T12:12:23.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c24005ce0 con 0x7f1c2c0a8250 2020-11-20T12:12:23.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24007320 con 0x7f1c2c0a8250 2020-11-20T12:12:23.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c0a5c60 con 0x7f1c2c0a8250 2020-11-20T12:12:23.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.740+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c13c880 con 0x7f1c2c0a8250 2020-11-20T12:12:23.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.741+0000 7f1c3222b700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c13e1d0 0x7f1c2c1358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.741+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c05cf40 con 0x7f1c2c13e1d0 2020-11-20T12:12:23.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.742+0000 7f1c31229700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c13e1d0 0x7f1c2c1358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.742+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1c24010030 con 0x7f1c2c0a8250 2020-11-20T12:12:23.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.742+0000 7f1c31229700 1 -- 172.21.15.44:0/2123392005 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c2c13c3e0 con 0x7f1c2c13e1d0 2020-11-20T12:12:23.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.742+0000 7f1c31229700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c13e1d0 0x7f1c2c1358a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f1c2800f820 tx=0x7f1c28007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.742+0000 7f1c197fa700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c04056d20 0x7f1c040591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.742+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1c2406fff0 con 0x7f1c2c0a8250 2020-11-20T12:12:23.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.742+0000 7f1c30a28700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c04056d20 0x7f1c040591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.743+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28011650 con 0x7f1c2c13e1d0 2020-11-20T12:12:23.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.743+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f1c280117b0 con 0x7f1c2c13e1d0 2020-11-20T12:12:23.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.743+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c13e1d0 msgr2=0x7f1c2c1358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.743+0000 7f1c197fa700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c13e1d0 0x7f1c2c1358a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f1c2800f820 tx=0x7f1c28007b90).stop 2020-11-20T12:12:23.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.743+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c13e1d0 msgr2=0x7f1c2c1358a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.743+0000 7f1c197fa700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c13e1d0 0x7f1c2c1358a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.743+0000 7f1c30a28700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c04056d20 0x7f1c040591e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f1c1c006e30 tx=0x7f1c1c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.751+0000 7f1c3222b700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c135d90 0x7f1c2c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.751+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c13c580 con 0x7f1c2c135d90 2020-11-20T12:12:23.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.752+0000 7f1c31229700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c135d90 0x7f1c2c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.752+0000 7f1c31229700 1 -- 172.21.15.44:0/2123392005 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1c2c057d80 con 0x7f1c2c135d90 2020-11-20T12:12:23.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.752+0000 7f1c31229700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c135d90 0x7f1c2c0a4890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f1c2800f820 tx=0x7f1c28018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.752+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28014e30 con 0x7f1c2c135d90 2020-11-20T12:12:23.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f1c28011440 con 0x7f1c2c135d90 2020-11-20T12:12:23.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c135d90 msgr2=0x7f1c2c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c197fa700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c135d90 0x7f1c2c0a4890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f1c2800f820 tx=0x7f1c28018000).stop 2020-11-20T12:12:23.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c197fa700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c135d90 msgr2=0x7f1c2c0a4890 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:23.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c197fa700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1c2c135d90 0x7f1c2c0a4890 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0x7f1c2800f820 tx=0x7f1c28018000).stop 2020-11-20T12:12:23.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c04056d20 msgr2=0x7f1c040591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c04056d20 0x7f1c040591e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f1c1c006e30 tx=0x7f1c1c015000).stop 2020-11-20T12:12:23.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c2c0a8250 msgr2=0x7f1c2c13dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 --2- 172.21.15.44:0/2123392005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c2c0a8250 0x7f1c2c13dce0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1c24001420 tx=0x7f1c24007e20).stop 2020-11-20T12:12:23.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c31a2a700 1 -- 172.21.15.44:0/2123392005 reap_dead start 2020-11-20T12:12:23.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c31a2a700 1 -- 172.21.15.44:0/2123392005 reap_dead start 2020-11-20T12:12:23.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 shutdown_connections 2020-11-20T12:12:23.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 >> 172.21.15.44:0/2123392005 conn(0x7f1c2c0ad520 msgr2=0x7f1c2c0ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 shutdown_connections 2020-11-20T12:12:23.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.753+0000 7f1c3222b700 1 -- 172.21.15.44:0/2123392005 wait complete. 2020-11-20T12:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 106, 2020-11-20T12:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:23.797 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:23.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.797 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.797 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.797 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:23.798 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.798 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.798 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:12:23.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.798 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.799 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.799 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:23.799 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.799 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.799 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:12:23.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.800 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.800 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.800 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:12:23.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.801 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.801 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:23.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.801 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.802 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.802 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:12:23.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.802 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:23.802 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:12:23.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:23.803 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:23.803 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:23.803 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:23.803 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:23.804 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:23.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:23.804 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:23.804 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:23.805 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:12:23.806 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:12:23.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.943+0000 7f01504fd700 1 Processor -- start 2020-11-20T12:12:23.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.943+0000 7f01504fd700 1 -- start start 2020-11-20T12:12:23.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.944+0000 7f01504fd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f014813ebf0 0x7f0148060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.944+0000 7f01504fd700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014813e080 con 0x7f01480ae050 2020-11-20T12:12:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.944+0000 7f01504fd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0148060d60 con 0x7f01480669d0 2020-11-20T12:12:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.944+0000 7f01504fd700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0148060b40 con 0x7f014813ebf0 2020-11-20T12:12:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.944+0000 7f014da98700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f014813ebf0 0x7f0148060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.944+0000 7f014e299700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f01480ae050 0x7f014813e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44060/0 (socket says 172.21.15.44:44060) 2020-11-20T12:12:23.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.944+0000 7f014e299700 1 -- 172.21.15.44:0/3914516532 learned_addr learned my addr 172.21.15.44:0/3914516532 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014da98700 1 -- 172.21.15.44:0/3914516532 >> v1:172.21.15.79:6789/0 conn(0x7f01480669d0 legacy=0x7f0148137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014da98700 1 -- 172.21.15.44:0/3914516532 >> v1:172.21.15.44:6789/0 conn(0x7f01480ae050 legacy=0x7f014813e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014d297700 1 -- 172.21.15.44:0/3914516532 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081057405 0 0) 0x7f014813e080 con 0x7f01480ae050 2020-11-20T12:12:23.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014da98700 1 -- 172.21.15.44:0/3914516532 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01480ae430 con 0x7f014813ebf0 2020-11-20T12:12:23.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014da98700 1 --2- 172.21.15.44:0/3914516532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f014813ebf0 0x7f0148060650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f013800b9a0 tx=0x7f0138006df0).ready entity=mon.0 client_cookie=fc44814679f846c7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014d297700 1 -- 172.21.15.44:0/3914516532 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380097b0 con 0x7f014813ebf0 2020-11-20T12:12:23.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014d297700 1 -- 172.21.15.44:0/3914516532 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138009910 con 0x7f014813ebf0 2020-11-20T12:12:23.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f014d297700 1 -- 172.21.15.44:0/3914516532 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138009c90 con 0x7f014813ebf0 2020-11-20T12:12:23.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.945+0000 7f01504fd700 1 -- 172.21.15.44:0/3914516532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f014813ebf0 msgr2=0x7f0148060650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.946+0000 7f01504fd700 1 --2- 172.21.15.44:0/3914516532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f014813ebf0 0x7f0148060650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f013800b9a0 tx=0x7f0138006df0).stop 2020-11-20T12:12:23.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.946+0000 7f01504fd700 1 -- 172.21.15.44:0/3914516532 shutdown_connections 2020-11-20T12:12:23.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.946+0000 7f01504fd700 1 --2- 172.21.15.44:0/3914516532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f014813ebf0 0x7f0148060650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.946+0000 7f01504fd700 1 -- 172.21.15.44:0/3914516532 >> 172.21.15.44:0/3914516532 conn(0x7f0148062bc0 msgr2=0x7f0148062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:23.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.946+0000 7f01504fd700 1 -- 172.21.15.44:0/3914516532 shutdown_connections 2020-11-20T12:12:23.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.946+0000 7f01504fd700 1 -- 172.21.15.44:0/3914516532 wait complete. 2020-11-20T12:12:23.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 Processor -- start 2020-11-20T12:12:23.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 -- start start 2020-11-20T12:12:23.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01480669d0 0x7f014813be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01480ae050 0x7f0148138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f014813ebf0 0x7f01481392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014814d430 con 0x7f01480ae050 2020-11-20T12:12:23.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014814d660 con 0x7f01480669d0 2020-11-20T12:12:23.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f01504fd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014814d900 con 0x7f014813ebf0 2020-11-20T12:12:23.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.947+0000 7f014da98700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01480ae050 0x7f0148138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01480ae050 0x7f0148138dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40320/0 (socket says 172.21.15.44:40320) 2020-11-20T12:12:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 -- 172.21.15.44:0/285246237 learned_addr learned my addr 172.21.15.44:0/285246237 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014e299700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01480669d0 0x7f014813be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014ea9a700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f014813ebf0 0x7f01481392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 -- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f014813ebf0 msgr2=0x7f01481392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f014813ebf0 0x7f01481392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 -- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01480669d0 msgr2=0x7f014813be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01480669d0 0x7f014813be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 -- 172.21.15.44:0/285246237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014814dba0 con 0x7f01480ae050 2020-11-20T12:12:23.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f014da98700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01480ae050 0x7f0148138dc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0138010000 tx=0x7f0138015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f01367fc700 1 -- 172.21.15.44:0/285246237 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013801e460 con 0x7f01480ae050 2020-11-20T12:12:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f01367fc700 1 -- 172.21.15.44:0/285246237 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013801e5c0 con 0x7f01480ae050 2020-11-20T12:12:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014813c300 con 0x7f01480ae050 2020-11-20T12:12:23.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f01367fc700 1 -- 172.21.15.44:0/285246237 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013801e940 con 0x7f01480ae050 2020-11-20T12:12:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.948+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014814e280 con 0x7f01480ae050 2020-11-20T12:12:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.949+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0148066700 con 0x7f01480ae050 2020-11-20T12:12:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.949+0000 7f01367fc700 1 -- 172.21.15.44:0/285246237 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0138025030 con 0x7f01480ae050 2020-11-20T12:12:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.950+0000 7f01367fc700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0120056d20 0x7f01200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:23.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.950+0000 7f01367fc700 1 -- 172.21.15.44:0/285246237 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0138071960 con 0x7f01480ae050 2020-11-20T12:12:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.950+0000 7f014e299700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0120056d20 0x7f01200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.951+0000 7f014e299700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0120056d20 0x7f01200591e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f01440011a0 tx=0x7f0144009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:23.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:23.954+0000 7f01367fc700 1 -- 172.21.15.44:0/285246237 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0138046ce0 con 0x7f01480ae050 2020-11-20T12:12:24.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.216+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f01480a50c0 con 0x7f01480ae050 2020-11-20T12:12:24.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.217+0000 7f01367fc700 1 -- 172.21.15.44:0/285246237 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f0138023070 con 0x7f01480ae050 2020-11-20T12:12:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0120056d20 msgr2=0x7f01200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0120056d20 0x7f01200591e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f01440011a0 tx=0x7f0144009040).stop 2020-11-20T12:12:24.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01480ae050 msgr2=0x7f0148138dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01480ae050 0x7f0148138dc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0138010000 tx=0x7f0138015000).stop 2020-11-20T12:12:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 shutdown_connections 2020-11-20T12:12:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0120056d20 0x7f01200591e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f014813ebf0 0x7f01481392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:24.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01480669d0 0x7f014813be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 --2- 172.21.15.44:0/285246237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01480ae050 0x7f0148138dc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.219+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 >> 172.21.15.44:0/285246237 conn(0x7f0148062bc0 msgr2=0x7f01480aa980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.220+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 shutdown_connections 2020-11-20T12:12:24.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:24.220+0000 7f01504fd700 1 -- 172.21.15.44:0/285246237 wait complete. 2020-11-20T12:12:24.235 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:12:34.236 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:12:34.237 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:12:34.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.373+0000 7fbc8ae05700 1 Processor -- start 2020-11-20T12:12:34.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.373+0000 7fbc8ae05700 1 -- start start 2020-11-20T12:12:34.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.373+0000 7fbc8ae05700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbc84061340 0x7fbc84068130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.373+0000 7fbc8ae05700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 0x7fbc8413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.373+0000 7fbc8ae05700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840a6d90 con 0x7fbc840a74d0 2020-11-20T12:12:34.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.373+0000 7fbc8ae05700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc840a5900 con 0x7fbc84061340 2020-11-20T12:12:34.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.373+0000 7fbc8ae05700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840a6ba0 con 0x7fbc840a4e10 2020-11-20T12:12:34.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc893a2700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbc840a74d0 0x7fbc84137a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44078/0 (socket says 172.21.15.44:44078) 2020-11-20T12:12:34.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc893a2700 1 -- 172.21.15.44:0/2544716611 learned_addr learned my addr 172.21.15.44:0/2544716611 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:34.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc83fff700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 0x7fbc8413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881209416 0 0) 0x7fbc840a6d90 con 0x7fbc840a74d0 2020-11-20T12:12:34.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc68003240 con 0x7fbc840a74d0 2020-11-20T12:12:34.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/2544716611 >> v2:172.21.15.79:3300/0 conn(0x7fbc84061340 0x7fbc84068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3556222192 0 0) 0x7fbc68003240 con 0x7fbc840a74d0 2020-11-20T12:12:34.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 msgr2=0x7fbc8413ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:34.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 0x7fbc8413ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 >> v2:172.21.15.79:3300/0 conn(0x7fbc84061340 msgr2=0x7fbc84068130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:34.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 --2- 172.21.15.44:0/2544716611 >> v2:172.21.15.79:3300/0 conn(0x7fbc84061340 0x7fbc84068130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840a7030 con 0x7fbc840a74d0 2020-11-20T12:12:34.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fbc7c001860 con 0x7fbc840a74d0 2020-11-20T12:12:34.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 >> v1:172.21.15.44:6789/0 conn(0x7fbc840a74d0 legacy=0x7fbc84137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc68003730 0x7fbc68003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc680044e0 0x7fbc68004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 0x7fbc68004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc68005360 con 0x7fbc68003730 2020-11-20T12:12:34.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc68004270 con 0x7fbc680044e0 2020-11-20T12:12:34.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6800b730 con 0x7fbc840a4e10 2020-11-20T12:12:34.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.374+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc68003730 0x7fbc68003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc893a2700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 0x7fbc68004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc83fff700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc680044e0 0x7fbc68004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc88ba1700 1 -- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 msgr2=0x7fbc68004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a4e10 0x7fbc68004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc88ba1700 1 -- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc680044e0 msgr2=0x7fbc68004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc680044e0 0x7fbc68004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc88ba1700 1 -- 172.21.15.44:0/2544716611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc68003240 con 0x7fbc68003730 2020-11-20T12:12:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc68003730 0x7fbc68003b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbc7800bc00 tx=0x7fbc78006df0).ready entity=mon.0 client_cookie=7cb0a6027006965 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800b310 con 0x7fbc68003730 2020-11-20T12:12:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7800e9d0 con 0x7fbc68003730 2020-11-20T12:12:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.375+0000 7fbc837fe700 1 -- 172.21.15.44:0/2544716611 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800ed50 con 0x7fbc68003730 2020-11-20T12:12:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.376+0000 7fbc8ae05700 1 -- 172.21.15.44:0/2544716611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc68003730 msgr2=0x7fbc68003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.376+0000 7fbc8ae05700 1 --2- 172.21.15.44:0/2544716611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc68003730 0x7fbc68003b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbc7800bc00 tx=0x7fbc78006df0).stop 2020-11-20T12:12:34.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.376+0000 7fbc893a2700 1 -- 172.21.15.44:0/2544716611 reap_dead start 2020-11-20T12:12:34.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.376+0000 7fbc8ae05700 1 -- 172.21.15.44:0/2544716611 shutdown_connections 2020-11-20T12:12:34.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.376+0000 7fbc8ae05700 1 -- 172.21.15.44:0/2544716611 >> 172.21.15.44:0/2544716611 conn(0x7fbc840ad6e0 msgr2=0x7fbc840adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:34.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.376+0000 7fbc8ae05700 1 -- 172.21.15.44:0/2544716611 shutdown_connections 2020-11-20T12:12:34.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.376+0000 7fbc8ae05700 1 -- 172.21.15.44:0/2544716611 wait complete. 2020-11-20T12:12:34.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 Processor -- start 2020-11-20T12:12:34.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 -- start start 2020-11-20T12:12:34.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc68003730 0x7fbc840ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc840a98d0 0x7fbc840a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a9d50 0x7fbc840a2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840a40a0 con 0x7fbc840a98d0 2020-11-20T12:12:34.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840a2d10 con 0x7fbc68003730 2020-11-20T12:12:34.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc8ae05700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8413bdb0 con 0x7fbc840a9d50 2020-11-20T12:12:34.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.377+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc840a98d0 0x7fbc840a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc840a98d0 0x7fbc840a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40340/0 (socket says 172.21.15.44:40340) 2020-11-20T12:12:34.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 -- 172.21.15.44:0/415870557 learned_addr learned my addr 172.21.15.44:0/415870557 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:34.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc893a2700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a9d50 0x7fbc840a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc68003730 0x7fbc840ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 -- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a9d50 msgr2=0x7fbc840a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a9d50 0x7fbc840a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 -- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc68003730 msgr2=0x7fbc840ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc68003730 0x7fbc840ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 -- 172.21.15.44:0/415870557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840a4290 con 0x7fbc840a98d0 2020-11-20T12:12:34.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc893a2700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a9d50 0x7fbc840a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:12:34.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc83fff700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc840a98d0 0x7fbc840a2330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fbc70011cf0 tx=0x7fbc70006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc80ff9700 1 -- 172.21.15.44:0/415870557 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000fb00 con 0x7fbc840a98d0 2020-11-20T12:12:34.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc80ff9700 1 -- 172.21.15.44:0/415870557 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7000fc60 con 0x7fbc840a98d0 2020-11-20T12:12:34.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc840a4530 con 0x7fbc840a98d0 2020-11-20T12:12:34.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc80ff9700 1 -- 172.21.15.44:0/415870557 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70014440 con 0x7fbc840a98d0 2020-11-20T12:12:34.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.378+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8414d060 con 0x7fbc840a98d0 2020-11-20T12:12:34.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.380+0000 7fbc80ff9700 1 -- 172.21.15.44:0/415870557 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbc70014830 con 0x7fbc840a98d0 2020-11-20T12:12:34.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.380+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc840446d0 con 0x7fbc840a98d0 2020-11-20T12:12:34.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.380+0000 7fbc80ff9700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbc5c056d20 0x7fbc5c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.380+0000 7fbc80ff9700 1 -- 172.21.15.44:0/415870557 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbc70070ca0 con 0x7fbc840a98d0 2020-11-20T12:12:34.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.380+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbc5c056d20 0x7fbc5c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.381+0000 7fbc88ba1700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbc5c056d20 0x7fbc5c0591e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fbc78006e30 tx=0x7fbc78019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.385+0000 7fbc80ff9700 1 -- 172.21.15.44:0/415870557 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbc70049e60 con 0x7fbc840a98d0 2020-11-20T12:12:34.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.674+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbc840697f0 con 0x7fbc840a98d0 2020-11-20T12:12:34.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.674+0000 7fbc80ff9700 1 -- 172.21.15.44:0/415870557 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fbc70075070 con 0x7fbc840a98d0 2020-11-20T12:12:34.678 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:12:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbc5c056d20 msgr2=0x7fbc5c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbc5c056d20 0x7fbc5c0591e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fbc78006e30 tx=0x7fbc78019000).stop 2020-11-20T12:12:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc840a98d0 msgr2=0x7fbc840a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc840a98d0 0x7fbc840a2330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fbc70011cf0 tx=0x7fbc70006df0).stop 2020-11-20T12:12:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 shutdown_connections 2020-11-20T12:12:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbc5c056d20 0x7fbc5c0591e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbc840a9d50 0x7fbc840a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc68003730 0x7fbc840ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 --2- 172.21.15.44:0/415870557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc840a98d0 0x7fbc840a2330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 >> 172.21.15.44:0/415870557 conn(0x7fbc840ad6e0 msgr2=0x7fbc84137850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:34.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 shutdown_connections 2020-11-20T12:12:34.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.676+0000 7fbc8ae05700 1 -- 172.21.15.44:0/415870557 wait complete. 2020-11-20T12:12:34.691 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:12:34.692 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:12:34.692 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:12:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.828+0000 7f8133cb2700 1 Processor -- start 2020-11-20T12:12:34.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8133cb2700 1 -- start start 2020-11-20T12:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8133cb2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f812c0aad00 0x7f812c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8133cb2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0ab0e0 0x7f812c13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8133cb2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f812c0a86d0 0x7f812c0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8133cb2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c1385f0 con 0x7f812c0aad00 2020-11-20T12:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8133cb2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c13e250 con 0x7f812c0a86d0 2020-11-20T12:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8133cb2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c13e4f0 con 0x7f812c0ab0e0 2020-11-20T12:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.829+0000 7f8131a4e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f812c0aad00 0x7f812c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f812c0aad00 0x7f812c0a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40346/0 (socket says 172.21.15.44:40346) 2020-11-20T12:12:34.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 -- 172.21.15.44:0/401945042 learned_addr learned my addr 172.21.15.44:0/401945042 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:34.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f813124d700 1 --2- 172.21.15.44:0/401945042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0ab0e0 0x7f812c13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f813224f700 1 --2- 172.21.15.44:0/401945042 >> v2:172.21.15.79:3300/0 conn(0x7f812c0a86d0 0x7f812c0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 -- 172.21.15.44:0/401945042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0ab0e0 msgr2=0x7f812c13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:34.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 --2- 172.21.15.44:0/401945042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0ab0e0 0x7f812c13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 -- 172.21.15.44:0/401945042 >> v2:172.21.15.79:3300/0 conn(0x7f812c0a86d0 msgr2=0x7f812c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 --2- 172.21.15.44:0/401945042 >> v2:172.21.15.79:3300/0 conn(0x7f812c0a86d0 0x7f812c0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 -- 172.21.15.44:0/401945042 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0ac120 con 0x7f812c0aad00 2020-11-20T12:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f813224f700 1 --2- 172.21.15.44:0/401945042 >> v2:172.21.15.79:3300/0 conn(0x7f812c0a86d0 0x7f812c0abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8131a4e700 1 --2- 172.21.15.44:0/401945042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a94d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f812c0a99c0 tx=0x7f8128006eb0).ready entity=mon.0 client_cookie=9cec0f482810f0f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8130a4c700 1 -- 172.21.15.44:0/401945042 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280098a0 con 0x7f812c0aad00 2020-11-20T12:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8130a4c700 1 -- 172.21.15.44:0/401945042 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8128009a00 con 0x7f812c0aad00 2020-11-20T12:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.830+0000 7f8130a4c700 1 -- 172.21.15.44:0/401945042 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128009d80 con 0x7f812c0aad00 2020-11-20T12:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 -- 172.21.15.44:0/401945042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 msgr2=0x7f812c0a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 --2- 172.21.15.44:0/401945042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a94d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f812c0a99c0 tx=0x7f8128006eb0).stop 2020-11-20T12:12:34.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 -- 172.21.15.44:0/401945042 shutdown_connections 2020-11-20T12:12:34.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 --2- 172.21.15.44:0/401945042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a94d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 --2- 172.21.15.44:0/401945042 >> v2:172.21.15.79:3300/0 conn(0x7f812c0a86d0 0x7f812c0abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 --2- 172.21.15.44:0/401945042 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0ab0e0 0x7f812c13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 -- 172.21.15.44:0/401945042 >> 172.21.15.44:0/401945042 conn(0x7f812c060170 msgr2=0x7f812c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:34.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 -- 172.21.15.44:0/401945042 shutdown_connections 2020-11-20T12:12:34.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.831+0000 7f8133cb2700 1 -- 172.21.15.44:0/401945042 wait complete. 2020-11-20T12:12:34.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.832+0000 7f8133cb2700 1 Processor -- start 2020-11-20T12:12:34.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.832+0000 7f8133cb2700 1 -- start start 2020-11-20T12:12:34.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8133cb2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0a86d0 0x7f812c09e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8133cb2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8133cb2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f812c0ab0e0 0x7f812c139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8133cb2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c13b390 con 0x7f812c0aad00 2020-11-20T12:12:34.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8133cb2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c13a9b0 con 0x7f812c0ab0e0 2020-11-20T12:12:34.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8133cb2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c13ac50 con 0x7f812c0a86d0 2020-11-20T12:12:34.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40354/0 (socket says 172.21.15.44:40354) 2020-11-20T12:12:34.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 -- 172.21.15.44:0/3052121667 learned_addr learned my addr 172.21.15.44:0/3052121667 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:34.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8131a4e700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0a86d0 0x7f812c09e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813224f700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f812c0ab0e0 0x7f812c139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0a86d0 msgr2=0x7f812c09e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f812c0a86d0 0x7f812c09e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f812c0ab0e0 msgr2=0x7f812c139710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f812c0ab0e0 0x7f812c139710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 -- 172.21.15.44:0/3052121667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c13aef0 con 0x7f812c0aad00 2020-11-20T12:12:34.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f813124d700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a43c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f811c00ebd0 tx=0x7f811c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00b8a0 con 0x7f812c0aad00 2020-11-20T12:12:34.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c00ba00 con 0x7f812c0aad00 2020-11-20T12:12:34.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c0a48b0 con 0x7f812c0aad00 2020-11-20T12:12:34.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.833+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00bd80 con 0x7f812c0aad00 2020-11-20T12:12:34.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.834+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c139c00 con 0x7f812c0aad00 2020-11-20T12:12:34.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8133cb2700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c133c40 0x7f812c14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c138d30 con 0x7f812c133c40 2020-11-20T12:12:34.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f811c012600 con 0x7f812c0aad00 2020-11-20T12:12:34.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8131a4e700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c133c40 0x7f812c14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8119ffb700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8104056d20 0x7f81040591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8131a4e700 1 -- 172.21.15.44:0/3052121667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f812c13b080 con 0x7f812c133c40 2020-11-20T12:12:34.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f811c070430 con 0x7f812c0aad00 2020-11-20T12:12:34.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f813224f700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8104056d20 0x7f81040591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8131a4e700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c133c40 0x7f812c14d430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f812c0a99c0 tx=0x7f8128015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.835+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128011c80 con 0x7f812c133c40 2020-11-20T12:12:34.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.836+0000 7f813224f700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8104056d20 0x7f81040591e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f812c09dfa0 tx=0x7f8124008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.836+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f8128011de0 con 0x7f812c133c40 2020-11-20T12:12:34.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.836+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c133c40 msgr2=0x7f812c14d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.836+0000 7f8119ffb700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c133c40 0x7f812c14d430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f812c0a99c0 tx=0x7f8128015000).stop 2020-11-20T12:12:34.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.836+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c133c40 msgr2=0x7f812c14d430 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:34.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.836+0000 7f8119ffb700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c133c40 0x7f812c14d430 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.844+0000 7f8133cb2700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c14d920 0x7f812c14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:34.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.844+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c05c5f0 con 0x7f812c14d920 2020-11-20T12:12:34.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.844+0000 7f8131a4e700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c14d920 0x7f812c14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.844+0000 7f8131a4e700 1 -- 172.21.15.44:0/3052121667 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f812c069680 con 0x7f812c14d920 2020-11-20T12:12:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.844+0000 7f8131a4e700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c14d920 0x7f812c14dd30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f812800b1b0 tx=0x7f8128021070).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.844+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128029a40 con 0x7f812c14d920 2020-11-20T12:12:34.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4439 (secure 0 0 0) 0x7f8128011a70 con 0x7f812c14d920 2020-11-20T12:12:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c14d920 msgr2=0x7f812c14dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8119ffb700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c14d920 0x7f812c14dd30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f812800b1b0 tx=0x7f8128021070).stop 2020-11-20T12:12:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8119ffb700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c14d920 msgr2=0x7f812c14dd30 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8119ffb700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c14d920 0x7f812c14dd30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:34.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8104056d20 msgr2=0x7f81040591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8104056d20 0x7f81040591e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f812c09dfa0 tx=0x7f8124008040).stop 2020-11-20T12:12:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 msgr2=0x7f812c0a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 --2- 172.21.15.44:0/3052121667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f812c0aad00 0x7f812c0a43c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f811c00ebd0 tx=0x7f811c006df0).stop 2020-11-20T12:12:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f813224f700 1 -- 172.21.15.44:0/3052121667 reap_dead start 2020-11-20T12:12:34.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f813224f700 1 -- 172.21.15.44:0/3052121667 reap_dead start 2020-11-20T12:12:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 shutdown_connections 2020-11-20T12:12:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 >> 172.21.15.44:0/3052121667 conn(0x7f812c060170 msgr2=0x7f812c0a8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:34.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 shutdown_connections 2020-11-20T12:12:34.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:34.845+0000 7f8133cb2700 1 -- 172.21.15.44:0/3052121667 wait complete. 2020-11-20T12:12:34.862 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:12:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 106, 2020-11-20T12:12:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:12:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:12:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:12:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.880 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:34.880 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:34.880 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:34.880 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:34.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:34.881 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:34.881 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:34.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:34.882 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:34.882 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:34.882 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:34.883 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:34.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.883 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:34.883 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:34.883 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:34.884 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:34.884 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:34.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.884 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:34.885 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:34.885 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:34.885 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:34.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.885 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:34.886 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:34.886 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:34.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:34.887 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:34.887 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:34.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:34.887 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:34.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.888 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:34.888 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:34.888 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:34.888 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:34.888 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:34.888 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:34.889 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:34.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.889 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:34.889 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:34.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.890 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:34.890 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:34.890 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:34.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:34.890 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:34.891 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:12:34.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.891 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:34.891 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:34.891 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:34.892 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:34.892 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:34.892 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:34.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:34.892 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:34.893 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:34.893 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:34.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:34.893 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:34.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:34.894 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:34.894 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:34.895 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:12:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.047+0000 7ff628deb700 1 Processor -- start 2020-11-20T12:12:35.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff628deb700 1 -- start start 2020-11-20T12:12:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff628deb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff624068f60 0x7ff6240a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff628deb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff62413e040 0x7ff624060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff628deb700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6240a8870 con 0x7ff624060660 2020-11-20T12:12:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff628deb700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6240a9f00 con 0x7ff62413e040 2020-11-20T12:12:35.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff628deb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6240a8aa0 con 0x7ff624068f60 2020-11-20T12:12:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff61bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff62413e040 0x7ff624060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff622d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff624060660 0x7ff6241376f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44116/0 (socket says 172.21.15.44:44116) 2020-11-20T12:12:35.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff622d9d700 1 -- 172.21.15.44:0/3022950345 learned_addr learned my addr 172.21.15.44:0/3022950345 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:35.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.048+0000 7ff62259c700 1 --2- 172.21.15.44:0/3022950345 >> v2:172.21.15.79:3300/0 conn(0x7ff624068f60 0x7ff6240a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff621d9b700 1 -- 172.21.15.44:0/3022950345 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122822383 0 0) 0x7ff6240a8870 con 0x7ff624060660 2020-11-20T12:12:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff621d9b700 1 -- 172.21.15.44:0/3022950345 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff604003240 con 0x7ff624060660 2020-11-20T12:12:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff61bfff700 1 -- 172.21.15.44:0/3022950345 >> v2:172.21.15.79:3300/0 conn(0x7ff624068f60 msgr2=0x7ff6240a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:35.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff61bfff700 1 --2- 172.21.15.44:0/3022950345 >> v2:172.21.15.79:3300/0 conn(0x7ff624068f60 0x7ff6240a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff61bfff700 1 -- 172.21.15.44:0/3022950345 >> v1:172.21.15.44:6789/0 conn(0x7ff624060660 legacy=0x7ff6241376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff61bfff700 1 -- 172.21.15.44:0/3022950345 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6240a8d40 con 0x7ff62413e040 2020-11-20T12:12:35.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff61bfff700 1 --2- 172.21.15.44:0/3022950345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff62413e040 0x7ff624060170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff6240ab1e0 tx=0x7ff60c006df0).ready entity=mon.0 client_cookie=69a6971e497dbd49 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff621d9b700 1 -- 172.21.15.44:0/3022950345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0096d0 con 0x7ff62413e040 2020-11-20T12:12:35.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff621d9b700 1 -- 172.21.15.44:0/3022950345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c009830 con 0x7ff62413e040 2020-11-20T12:12:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.049+0000 7ff621d9b700 1 -- 172.21.15.44:0/3022950345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c009bb0 con 0x7ff62413e040 2020-11-20T12:12:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 -- 172.21.15.44:0/3022950345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff62413e040 msgr2=0x7ff624060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 --2- 172.21.15.44:0/3022950345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff62413e040 0x7ff624060170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff6240ab1e0 tx=0x7ff60c006df0).stop 2020-11-20T12:12:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 -- 172.21.15.44:0/3022950345 shutdown_connections 2020-11-20T12:12:35.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 --2- 172.21.15.44:0/3022950345 >> v2:172.21.15.79:3300/0 conn(0x7ff624068f60 0x7ff6240a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 --2- 172.21.15.44:0/3022950345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff62413e040 0x7ff624060170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 -- 172.21.15.44:0/3022950345 >> 172.21.15.44:0/3022950345 conn(0x7ff6240ad6f0 msgr2=0x7ff6240adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:35.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 -- 172.21.15.44:0/3022950345 shutdown_connections 2020-11-20T12:12:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.050+0000 7ff628deb700 1 -- 172.21.15.44:0/3022950345 wait complete. 2020-11-20T12:12:35.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 Processor -- start 2020-11-20T12:12:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 -- start start 2020-11-20T12:12:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff624060660 0x7ff6240aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff624068f60 0x7ff6240a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff62413e040 0x7ff6240a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62413af20 con 0x7ff624060660 2020-11-20T12:12:35.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62413b180 con 0x7ff62413e040 2020-11-20T12:12:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff628deb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62413b420 con 0x7ff624068f60 2020-11-20T12:12:35.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff62259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff624060660 0x7ff6240aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff62259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff624060660 0x7ff6240aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40370/0 (socket says 172.21.15.44:40370) 2020-11-20T12:12:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff62259c700 1 -- 172.21.15.44:0/318483198 learned_addr learned my addr 172.21.15.44:0/318483198 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:35.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff61bfff700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff624068f60 0x7ff6240a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.051+0000 7ff622d9d700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff62413e040 0x7ff6240a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff62259c700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff624068f60 msgr2=0x7ff6240a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff62259c700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff624068f60 0x7ff6240a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff62259c700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff62413e040 msgr2=0x7ff6240a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff62259c700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff62413e040 0x7ff6240a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff62259c700 1 -- 172.21.15.44:0/318483198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62414d2f0 con 0x7ff624060660 2020-11-20T12:12:35.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff62259c700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff624060660 0x7ff6240aceb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff61400eba0 tx=0x7ff614006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400c8e0 con 0x7ff624060660 2020-11-20T12:12:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61400ca40 con 0x7ff624060660 2020-11-20T12:12:35.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62414d540 con 0x7ff624060660 2020-11-20T12:12:35.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400cdc0 con 0x7ff624060660 2020-11-20T12:12:35.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.052+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62414dad0 con 0x7ff624060660 2020-11-20T12:12:35.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.053+0000 7ff628deb700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624131940 0x7ff624131d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.053+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62414dcf0 con 0x7ff624131940 2020-11-20T12:12:35.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.053+0000 7ff61bfff700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624131940 0x7ff624131d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.053+0000 7ff61bfff700 1 -- 172.21.15.44:0/318483198 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff62413ada0 con 0x7ff624131940 2020-11-20T12:12:35.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61bfff700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624131940 0x7ff624131d20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff6240ab1e0 tx=0x7ff60c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff614012600 con 0x7ff624060660 2020-11-20T12:12:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff600056d20 0x7ff6000591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c023030 con 0x7ff624131940 2020-11-20T12:12:35.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff614070420 con 0x7ff624060660 2020-11-20T12:12:35.086 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff622d9d700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff600056d20 0x7ff6000591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7ff60c01f440 con 0x7ff624131940 2020-11-20T12:12:35.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624131940 msgr2=0x7ff624131d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624131940 0x7ff624131d20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff6240ab1e0 tx=0x7ff60c015000).stop 2020-11-20T12:12:35.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624131940 msgr2=0x7ff624131d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:35.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.054+0000 7ff61a7fc700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff624131940 0x7ff624131d20 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0x7ff6240ab1e0 tx=0x7ff60c015000).stop 2020-11-20T12:12:35.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.055+0000 7ff622d9d700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff600056d20 0x7ff6000591e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7ff61c0013b0 tx=0x7ff61c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.067+0000 7ff628deb700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241348a0 0x7ff624134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.067+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624063f00 con 0x7ff6241348a0 2020-11-20T12:12:35.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.067+0000 7ff61bfff700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241348a0 0x7ff624134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61bfff700 1 -- 172.21.15.44:0/318483198 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff6240aae90 con 0x7ff6241348a0 2020-11-20T12:12:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61bfff700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241348a0 0x7ff624134cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff60c0093d0 tx=0x7ff60c021070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c01eb80 con 0x7ff6241348a0 2020-11-20T12:12:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7ff60c033070 con 0x7ff6241348a0 2020-11-20T12:12:35.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241348a0 msgr2=0x7ff624134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61a7fc700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241348a0 0x7ff624134cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff60c0093d0 tx=0x7ff60c021070).stop 2020-11-20T12:12:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61a7fc700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241348a0 msgr2=0x7ff624134cb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:35.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.068+0000 7ff61a7fc700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6241348a0 0x7ff624134cb0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0x7ff60c0093d0 tx=0x7ff60c021070).stop 2020-11-20T12:12:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff600056d20 msgr2=0x7ff6000591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff600056d20 0x7ff6000591e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7ff61c0013b0 tx=0x7ff61c009040).stop 2020-11-20T12:12:35.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff624060660 msgr2=0x7ff6240aceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 --2- 172.21.15.44:0/318483198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff624060660 0x7ff6240aceb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff61400eba0 tx=0x7ff614006df0).stop 2020-11-20T12:12:35.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff622d9d700 1 -- 172.21.15.44:0/318483198 reap_dead start 2020-11-20T12:12:35.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff622d9d700 1 -- 172.21.15.44:0/318483198 reap_dead start 2020-11-20T12:12:35.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 shutdown_connections 2020-11-20T12:12:35.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 >> 172.21.15.44:0/318483198 conn(0x7ff6240ad6f0 msgr2=0x7ff62413e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:35.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 shutdown_connections 2020-11-20T12:12:35.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.069+0000 7ff628deb700 1 -- 172.21.15.44:0/318483198 wait complete. 2020-11-20T12:12:35.102 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:35.102 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:35.103 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:35.103 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:12:35.103 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 106, 2020-11-20T12:12:35.104 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:35.104 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:35.104 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:12:35.105 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:12:35.105 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.105 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:12:35.105 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:35.106 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:35.106 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:35.106 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:35.106 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:35.106 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:35.107 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:35.108 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:35.109 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:35.109 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:35.109 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:35.109 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:35.109 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:35.110 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:35.110 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.110 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:35.110 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:35.111 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:35.111 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:35.111 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:35.111 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:35.111 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:35.112 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:35.113 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:35.113 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:35.113 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:35.113 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:35.113 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:35.114 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:35.114 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:35.114 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:35.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:35.114 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:35.115 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.115 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:35.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.115 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:35.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.116 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:35.116 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:35.116 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:35.116 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:35.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:35.117 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:35.117 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:35.118 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:35.118 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.118 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.118 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.119 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:35.119 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:35.119 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:35.119 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:35.120 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:35.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.120 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:35.120 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:35.121 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:35.121 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:35.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:35.121 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:35.121 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:35.122 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:35.122 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.123 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.123 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:35.123 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:35.124 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:35.124 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:35.124 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:35.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.125 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:35.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:35.125 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:35.125 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:35.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:35.126 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:35.126 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:35.127 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:35.127 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.127 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.127 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:35.133 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:12:35.269 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.266+0000 7f282cdec700 1 Processor -- start 2020-11-20T12:12:35.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.266+0000 7f282cdec700 1 -- start start 2020-11-20T12:12:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f282cdec700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f282813b780 0x7f28280aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f282cdec700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28280a4af0 0x7f2828137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f282cdec700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28280a4f70 con 0x7f28280a9460 2020-11-20T12:12:35.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f282cdec700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282813ac80 con 0x7f282813b780 2020-11-20T12:12:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f282cdec700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28280ab310 con 0x7f28280a4af0 2020-11-20T12:12:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f2825d9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f282813b780 0x7f28280aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f282659c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f28280a9460 0x7f282813b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44130/0 (socket says 172.21.15.44:44130) 2020-11-20T12:12:35.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f282659c700 1 -- 172.21.15.44:0/1221083912 learned_addr learned my addr 172.21.15.44:0/1221083912 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f2826d9d700 1 --2- 172.21.15.44:0/1221083912 >> v2:172.21.15.79:3300/0 conn(0x7f28280a4af0 0x7f2828137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f2825d9b700 1 -- 172.21.15.44:0/1221083912 >> v2:172.21.15.79:3300/0 conn(0x7f28280a4af0 msgr2=0x7f2828137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f2825d9b700 1 --2- 172.21.15.44:0/1221083912 >> v2:172.21.15.79:3300/0 conn(0x7f28280a4af0 0x7f2828137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f2825d9b700 1 -- 172.21.15.44:0/1221083912 >> v1:172.21.15.44:6789/0 conn(0x7f28280a9460 legacy=0x7f282813b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.267+0000 7f2825d9b700 1 -- 172.21.15.44:0/1221083912 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280ab5b0 con 0x7f282813b780 2020-11-20T12:12:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f2825d9b700 1 --2- 172.21.15.44:0/1221083912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280aad20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f28280af220 tx=0x7f281c006df0).ready entity=mon.0 client_cookie=8c9ae55e68280496 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282559a700 1 -- 172.21.15.44:0/1221083912 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c0097b0 con 0x7f282813b780 2020-11-20T12:12:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282559a700 1 -- 172.21.15.44:0/1221083912 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c009910 con 0x7f282813b780 2020-11-20T12:12:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282559a700 1 -- 172.21.15.44:0/1221083912 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c009c90 con 0x7f282813b780 2020-11-20T12:12:35.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 -- 172.21.15.44:0/1221083912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 msgr2=0x7f28280aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 --2- 172.21.15.44:0/1221083912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280aad20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f28280af220 tx=0x7f281c006df0).stop 2020-11-20T12:12:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 -- 172.21.15.44:0/1221083912 shutdown_connections 2020-11-20T12:12:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 --2- 172.21.15.44:0/1221083912 >> v2:172.21.15.79:3300/0 conn(0x7f28280a4af0 0x7f2828137c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 --2- 172.21.15.44:0/1221083912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280aad20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 -- 172.21.15.44:0/1221083912 >> 172.21.15.44:0/1221083912 conn(0x7f28280621c0 msgr2=0x7f2828060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:35.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 -- 172.21.15.44:0/1221083912 shutdown_connections 2020-11-20T12:12:35.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.268+0000 7f282cdec700 1 -- 172.21.15.44:0/1221083912 wait complete. 2020-11-20T12:12:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 Processor -- start 2020-11-20T12:12:35.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 -- start start 2020-11-20T12:12:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28280a4af0 0x7f2828136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28280a9460 0x7f28280a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282813e8e0 con 0x7f282813b780 2020-11-20T12:12:35.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2828137670 con 0x7f28280a4af0 2020-11-20T12:12:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282cdec700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282813eb40 con 0x7f28280a9460 2020-11-20T12:12:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f2826d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f2826d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280a6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40392/0 (socket says 172.21.15.44:40392) 2020-11-20T12:12:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f2826d9d700 1 -- 172.21.15.44:0/4195074308 learned_addr learned my addr 172.21.15.44:0/4195074308 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:35.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f2825d9b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28280a9460 0x7f28280a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.270+0000 7f282659c700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28280a4af0 0x7f2828136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f2826d9d700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28280a9460 msgr2=0x7f28280a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f2826d9d700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f28280a9460 0x7f28280a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f2826d9d700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28280a4af0 msgr2=0x7f2828136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f2826d9d700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28280a4af0 0x7f2828136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f2826d9d700 1 -- 172.21.15.44:0/4195074308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282813ede0 con 0x7f282813b780 2020-11-20T12:12:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f2826d9d700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280a6e90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f282000ef50 tx=0x7f28200091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282000cbd0 con 0x7f282813b780 2020-11-20T12:12:35.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282000cd30 con 0x7f282813b780 2020-11-20T12:12:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820012620 con 0x7f282813b780 2020-11-20T12:12:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282813da80 con 0x7f282813b780 2020-11-20T12:12:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.271+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282813def0 con 0x7f282813b780 2020-11-20T12:12:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.272+0000 7f282cdec700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2828131da0 0x7f282814da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.272+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2828136e10 con 0x7f2828131da0 2020-11-20T12:12:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.272+0000 7f282659c700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2828131da0 0x7f282814da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.272+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2820022070 con 0x7f282813b780 2020-11-20T12:12:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f282659c700 1 -- 172.21.15.44:0/4195074308 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28280ac640 con 0x7f2828131da0 2020-11-20T12:12:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f282659c700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2828131da0 0x7f282814da10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2818000bc0 tx=0x7f2818009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27fc056d20 0x7f27fc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f28200709f0 con 0x7f282813b780 2020-11-20T12:12:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f2825d9b700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27fc056d20 0x7f27fc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180070d0 con 0x7f2828131da0 2020-11-20T12:12:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f2818007230 con 0x7f2828131da0 2020-11-20T12:12:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2828131da0 msgr2=0x7f282814da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2828131da0 0x7f282814da10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2818000bc0 tx=0x7f2818009040).stop 2020-11-20T12:12:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2828131da0 msgr2=0x7f282814da10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f28167fc700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2828131da0 0x7f282814da10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.273+0000 7f2825d9b700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27fc056d20 0x7f27fc0591e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f282813bba0 tx=0x7f281c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.285+0000 7f282cdec700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f282813e140 0x7f282814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.285+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282813ef70 con 0x7f282813e140 2020-11-20T12:12:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.285+0000 7f282659c700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f282813e140 0x7f282814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f282659c700 1 -- 172.21.15.44:0/4195074308 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2828060880 con 0x7f282813e140 2020-11-20T12:12:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f282659c700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f282813e140 0x7f282814df00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f2818015000 tx=0x7f2818018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281800d9c0 con 0x7f282813e140 2020-11-20T12:12:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f28180055d0 con 0x7f282813e140 2020-11-20T12:12:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f282813e140 msgr2=0x7f282814df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f28167fc700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f282813e140 0x7f282814df00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f2818015000 tx=0x7f2818018000).stop 2020-11-20T12:12:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f28167fc700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f282813e140 msgr2=0x7f282814df00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f28167fc700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f282813e140 0x7f282814df00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27fc056d20 msgr2=0x7f27fc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f282cdec700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27fc056d20 0x7f27fc0591e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f282813bba0 tx=0x7f281c015000).stop 2020-11-20T12:12:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 msgr2=0x7f28280a6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f282cdec700 1 --2- 172.21.15.44:0/4195074308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f282813b780 0x7f28280a6e90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f282000ef50 tx=0x7f28200091d0).stop 2020-11-20T12:12:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f2826d9d700 1 -- 172.21.15.44:0/4195074308 reap_dead start 2020-11-20T12:12:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.286+0000 7f2826d9d700 1 -- 172.21.15.44:0/4195074308 reap_dead start 2020-11-20T12:12:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.287+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 shutdown_connections 2020-11-20T12:12:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.287+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 >> 172.21.15.44:0/4195074308 conn(0x7f28280621c0 msgr2=0x7f2828132a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.287+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 shutdown_connections 2020-11-20T12:12:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.287+0000 7f282cdec700 1 -- 172.21.15.44:0/4195074308 wait complete. 2020-11-20T12:12:35.295 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:35.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:35.295 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:35.295 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:12:35.296 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 106, 2020-11-20T12:12:35.296 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:35.296 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:35.296 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:12:35.296 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:12:35.297 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.297 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:12:35.297 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:35.297 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:35.298 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:35.298 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:35.298 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:35.298 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:35.298 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:35.299 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:35.299 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:35.299 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:35.299 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:35.299 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.300 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:35.300 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:35.300 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:35.300 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:35.300 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:35.301 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:35.301 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:35.301 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:35.301 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:35.301 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:35.302 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.302 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:35.302 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:35.302 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:35.303 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:35.303 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:35.303 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:35.303 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:35.303 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:35.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:35.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:35.304 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:35.304 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:35.304 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:35.305 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:35.305 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:35.305 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:35.305 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:35.306 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:35.306 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:35.306 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:35.306 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:35.306 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:35.307 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:35.307 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.307 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:35.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.307 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:35.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.308 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:35.308 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:35.308 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:35.308 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:35.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:35.309 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:35.309 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:35.310 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:35.310 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.311 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.311 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:35.311 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:35.311 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:35.312 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:35.312 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:35.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.312 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:35.313 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:35.313 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:35.313 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:35.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:35.314 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:35.314 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.314 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:35.315 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:35.315 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.315 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.315 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.316 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:35.316 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:35.316 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:35.316 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:35.316 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:35.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.317 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:35.317 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:35.317 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:35.318 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:35.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.318 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:35.318 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:35.318 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:35.319 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:35.319 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:35.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.320 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.320 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:35.320 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:35.321 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:35.321 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:35.321 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:35.321 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.321 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.322 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:35.322 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:35.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.322 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:35.323 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:35.323 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:35.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.323 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.323 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:12:35.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.324 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:35.324 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:35.324 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:35.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.325 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:12:35.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.325 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:35.325 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:35.326 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:12:35.326 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.326 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.326 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:35.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.327 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:35.327 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:35.327 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:12:35.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.328 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:35.328 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:12:35.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:35.328 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:35.328 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:35.329 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:35.329 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:35.329 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:35.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:35.329 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:35.330 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:35.331 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2020-11-20T12:12:35.331 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:12:35.331 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2020-11-20T12:12:35.332 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2020-11-20T12:12:35.332 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2020-11-20T12:12:35.332 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:12:35.348 INFO:tasks.ceph.mon.c:Stopped 2020-11-20T12:12:35.348 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:12:35.349 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:12:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.492+0000 7f01e8e3b700 1 Processor -- start 2020-11-20T12:12:35.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.492+0000 7f01e8e3b700 1 -- start start 2020-11-20T12:12:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e8e3b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f01e40ae050 0x7f01e4060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e8e3b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e413ebf0 0x7f01e4137b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e8e3b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e413e080 con 0x7f01e40669d0 2020-11-20T12:12:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e8e3b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e4060b40 con 0x7f01e40ae050 2020-11-20T12:12:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e8e3b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4060d60 con 0x7f01e413ebf0 2020-11-20T12:12:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e259c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f01e40ae050 0x7f01e4060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e2d9d700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e413ebf0 msgr2=0x7f01e4137b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e259c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f01e40ae050 0x7f01e4060650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40436/0 (socket says 172.21.15.44:40436) 2020-11-20T12:12:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e259c700 1 -- 172.21.15.44:0/4293983337 learned_addr learned my addr 172.21.15.44:0/4293983337 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.493+0000 7f01e2d9d700 1 --2- 172.21.15.44:0/4293983337 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e413ebf0 0x7f01e4137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e259c700 1 -- 172.21.15.44:0/4293983337 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e413ebf0 msgr2=0x7f01e4137b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e259c700 1 --2- 172.21.15.44:0/4293983337 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e413ebf0 0x7f01e4137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e259c700 1 -- 172.21.15.44:0/4293983337 >> v1:172.21.15.79:6789/0 conn(0x7f01e40669d0 legacy=0x7f01e413e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e259c700 1 -- 172.21.15.44:0/4293983337 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40ae430 con 0x7f01e40ae050 2020-11-20T12:12:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e259c700 1 --2- 172.21.15.44:0/4293983337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40ae050 0x7f01e4060650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f01d4006fd0 tx=0x7f01d4006eb0).ready entity=mon.0 client_cookie=e744f470a307f961 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e159a700 1 -- 172.21.15.44:0/4293983337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40098d0 con 0x7f01e40ae050 2020-11-20T12:12:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e159a700 1 -- 172.21.15.44:0/4293983337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d4009a30 con 0x7f01e40ae050 2020-11-20T12:12:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e159a700 1 -- 172.21.15.44:0/4293983337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d4009db0 con 0x7f01e40ae050 2020-11-20T12:12:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 -- 172.21.15.44:0/4293983337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40ae050 msgr2=0x7f01e4060650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/4293983337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40ae050 0x7f01e4060650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f01d4006fd0 tx=0x7f01d4006eb0).stop 2020-11-20T12:12:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 -- 172.21.15.44:0/4293983337 shutdown_connections 2020-11-20T12:12:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/4293983337 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e413ebf0 0x7f01e4137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/4293983337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40ae050 0x7f01e4060650 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 -- 172.21.15.44:0/4293983337 >> 172.21.15.44:0/4293983337 conn(0x7f01e4062bc0 msgr2=0x7f01e4062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 -- 172.21.15.44:0/4293983337 shutdown_connections 2020-11-20T12:12:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.494+0000 7f01e8e3b700 1 -- 172.21.15.44:0/4293983337 wait complete. 2020-11-20T12:12:35.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.495+0000 7f01e8e3b700 1 Processor -- start 2020-11-20T12:12:35.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.495+0000 7f01e8e3b700 1 -- start start 2020-11-20T12:12:35.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e8e3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40669d0 0x7f01e4140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e8e3b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e40ae050 0x7f01e4139050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e8e3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01e413ebf0 0x7f01e414ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e8e3b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40adca0 con 0x7f01e40669d0 2020-11-20T12:12:35.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e8e3b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4139840 con 0x7f01e413ebf0 2020-11-20T12:12:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e8e3b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e414d6b0 con 0x7f01e40ae050 2020-11-20T12:12:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40669d0 0x7f01e4140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40669d0 0x7f01e4140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40442/0 (socket says 172.21.15.44:40442) 2020-11-20T12:12:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e259c700 1 -- 172.21.15.44:0/1641329893 learned_addr learned my addr 172.21.15.44:0/1641329893 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:35.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e1d9b700 1 -- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e40ae050 msgr2=0x7f01e4139050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e1d9b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e40ae050 0x7f01e4139050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e2d9d700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01e413ebf0 0x7f01e414ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e259c700 1 -- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e40ae050 msgr2=0x7f01e4139050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:12:35.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.496+0000 7f01e259c700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e40ae050 0x7f01e4139050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:35.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01e259c700 1 -- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01e413ebf0 msgr2=0x7f01e414ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01e259c700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01e413ebf0 0x7f01e414ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01e259c700 1 -- 172.21.15.44:0/1641329893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e414d950 con 0x7f01e40669d0 2020-11-20T12:12:35.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01e259c700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40669d0 0x7f01e4140230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f01d400b1b0 tx=0x7f01d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01d27fc700 1 -- 172.21.15.44:0/1641329893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d401e460 con 0x7f01e40669d0 2020-11-20T12:12:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01d27fc700 1 -- 172.21.15.44:0/1641329893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d401e5e0 con 0x7f01e40669d0 2020-11-20T12:12:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e4139540 con 0x7f01e40669d0 2020-11-20T12:12:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01d27fc700 1 -- 172.21.15.44:0/1641329893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d401e9a0 con 0x7f01e40669d0 2020-11-20T12:12:35.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.497+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e4140da0 con 0x7f01e40669d0 2020-11-20T12:12:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.498+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e4140f60 con 0x7f01e40669d0 2020-11-20T12:12:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.498+0000 7f01d27fc700 1 -- 172.21.15.44:0/1641329893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f01d4025030 con 0x7f01e40669d0 2020-11-20T12:12:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.498+0000 7f01d27fc700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f01b8056cd0 0x7f01b8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.499+0000 7f01d27fc700 1 -- 172.21.15.44:0/1641329893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f01d4071a50 con 0x7f01e40669d0 2020-11-20T12:12:35.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.499+0000 7f01e1d9b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f01b8056cd0 0x7f01b8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.499+0000 7f01e1d9b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f01b8056cd0 0x7f01b8059190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f01d8001010 tx=0x7f01d80077f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:35.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.502+0000 7f01d27fc700 1 -- 172.21.15.44:0/1641329893 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f01d4046cc0 con 0x7f01e40669d0 2020-11-20T12:12:35.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.760+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f01e40a50c0 con 0x7f01e40669d0 2020-11-20T12:12:35.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.761+0000 7f01d27fc700 1 -- 172.21.15.44:0/1641329893 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f01d4023020 con 0x7f01e40669d0 2020-11-20T12:12:35.765 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:12:35.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f01b8056cd0 msgr2=0x7f01b8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f01b8056cd0 0x7f01b8059190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f01d8001010 tx=0x7f01d80077f0).stop 2020-11-20T12:12:35.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40669d0 msgr2=0x7f01e4140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:35.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40669d0 0x7f01e4140230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f01d400b1b0 tx=0x7f01d4015000).stop 2020-11-20T12:12:35.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 shutdown_connections 2020-11-20T12:12:35.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f01b8056cd0 0x7f01b8059190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01e413ebf0 0x7f01e414ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01e40669d0 0x7f01e4140230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:35.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 --2- 172.21.15.44:0/1641329893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01e40ae050 0x7f01e4139050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:35.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 >> 172.21.15.44:0/1641329893 conn(0x7f01e4062bc0 msgr2=0x7f01e4134190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:35.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 shutdown_connections 2020-11-20T12:12:35.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:35.763+0000 7f01e8e3b700 1 -- 172.21.15.44:0/1641329893 wait complete. 2020-11-20T12:12:37.547 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:12:37.555 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:12:38.780 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:12:38.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.961+0000 7f4b7766b700 1 Processor -- start 2020-11-20T12:12:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b7766b700 1 -- start start 2020-11-20T12:12:40.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b7766b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4b70138220 0x7f4b70060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b7766b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7013e080 con 0x7f4b700ae050 2020-11-20T12:12:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b7766b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b70060d60 con 0x7f4b7013ebf0 2020-11-20T12:12:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b7766b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b70060b40 con 0x7f4b70138220 2020-11-20T12:12:40.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b75407700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f4b700ae050 0x7f4b7013e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44208/0 (socket says 172.21.15.44:44208) 2020-11-20T12:12:40.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b75407700 1 -- 172.21.15.44:0/3781734218 learned_addr learned my addr 172.21.15.44:0/3781734218 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:40.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.962+0000 7f4b75c08700 1 --2- 172.21.15.44:0/3781734218 >> v2:172.21.15.44:3300/0 conn(0x7f4b70138220 0x7f4b70060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:40.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b75c08700 1 -- 172.21.15.44:0/3781734218 >> v1:172.21.15.79:6789/0 conn(0x7f4b7013ebf0 legacy=0x7f4b70137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:40.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b75c08700 1 -- 172.21.15.44:0/3781734218 >> v1:172.21.15.44:6789/0 conn(0x7f4b700ae050 legacy=0x7f4b7013e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b5ffff700 1 -- 172.21.15.44:0/3781734218 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614602004 0 0) 0x7f4b7013e080 con 0x7f4b700ae050 2020-11-20T12:12:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b75c08700 1 -- 172.21.15.44:0/3781734218 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700ae430 con 0x7f4b70138220 2020-11-20T12:12:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b75c08700 1 --2- 172.21.15.44:0/3781734218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b70138220 0x7f4b70060650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f4b68009c90 tx=0x7f4b680091d0).ready entity=mon.0 client_cookie=c4405791a76c1e5a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b5ffff700 1 -- 172.21.15.44:0/3781734218 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800bbc0 con 0x7f4b70138220 2020-11-20T12:12:40.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b5ffff700 1 -- 172.21.15.44:0/3781734218 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6800bd20 con 0x7f4b70138220 2020-11-20T12:12:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.963+0000 7f4b5ffff700 1 -- 172.21.15.44:0/3781734218 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68011610 con 0x7f4b70138220 2020-11-20T12:12:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.964+0000 7f4b7766b700 1 -- 172.21.15.44:0/3781734218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b70138220 msgr2=0x7f4b70060650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.964+0000 7f4b7766b700 1 --2- 172.21.15.44:0/3781734218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b70138220 0x7f4b70060650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f4b68009c90 tx=0x7f4b680091d0).stop 2020-11-20T12:12:40.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.964+0000 7f4b7766b700 1 -- 172.21.15.44:0/3781734218 shutdown_connections 2020-11-20T12:12:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.964+0000 7f4b7766b700 1 --2- 172.21.15.44:0/3781734218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b70138220 0x7f4b70060650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.964+0000 7f4b7766b700 1 -- 172.21.15.44:0/3781734218 >> 172.21.15.44:0/3781734218 conn(0x7f4b70062bc0 msgr2=0x7f4b70062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.964+0000 7f4b7766b700 1 -- 172.21.15.44:0/3781734218 shutdown_connections 2020-11-20T12:12:40.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.964+0000 7f4b7766b700 1 -- 172.21.15.44:0/3781734218 wait complete. 2020-11-20T12:12:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.965+0000 7f4b7766b700 1 Processor -- start 2020-11-20T12:12:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.965+0000 7f4b7766b700 1 -- start start 2020-11-20T12:12:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b700ae050 0x7f4b700a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4b70138220 0x7f4b700a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b7013ebf0 0x7f4b7014a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700a42f0 con 0x7f4b7013ebf0 2020-11-20T12:12:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700a1d70 con 0x7f4b700ae050 2020-11-20T12:12:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700a44d0 con 0x7f4b70138220 2020-11-20T12:12:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b7013ebf0 0x7f4b7014a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b74c06700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4b70138220 msgr2=0x7f4b700a1880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b7013ebf0 0x7f4b7014a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40470/0 (socket says 172.21.15.44:40470) 2020-11-20T12:12:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b74c06700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4b70138220 0x7f4b700a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 -- 172.21.15.44:0/2535336721 learned_addr learned my addr 172.21.15.44:0/2535336721 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75407700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b700ae050 0x7f4b700a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 -- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4b70138220 msgr2=0x7f4b700a1880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:12:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4b70138220 0x7f4b700a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 -- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b700ae050 msgr2=0x7f4b700a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:40.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b700ae050 0x7f4b700a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 -- 172.21.15.44:0/2535336721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700a4770 con 0x7f4b7013ebf0 2020-11-20T12:12:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b75c08700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b7013ebf0 0x7f4b7014a1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4b68009210 tx=0x7f4b68016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b5d7fa700 1 -- 172.21.15.44:0/2535336721 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68011e30 con 0x7f4b7013ebf0 2020-11-20T12:12:40.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b5d7fa700 1 -- 172.21.15.44:0/2535336721 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68010b50 con 0x7f4b7013ebf0 2020-11-20T12:12:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b70149cb0 con 0x7f4b7013ebf0 2020-11-20T12:12:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b5d7fa700 1 -- 172.21.15.44:0/2535336721 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6801d480 con 0x7f4b7013ebf0 2020-11-20T12:12:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.966+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7013c670 con 0x7f4b7013ebf0 2020-11-20T12:12:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.967+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b700a50c0 con 0x7f4b7013ebf0 2020-11-20T12:12:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.968+0000 7f4b5d7fa700 1 -- 172.21.15.44:0/2535336721 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4b68017070 con 0x7f4b7013ebf0 2020-11-20T12:12:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.969+0000 7f4b5d7fa700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4b48056cd0 0x7f4b48059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.969+0000 7f4b5d7fa700 1 -- 172.21.15.44:0/2535336721 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4b680731a0 con 0x7f4b7013ebf0 2020-11-20T12:12:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.969+0000 7f4b75407700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4b48056cd0 0x7f4b48059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.969+0000 7f4b75407700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4b48056cd0 0x7f4b48059190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f4b7013c990 tx=0x7f4b6c0079e0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:38.972+0000 7f4b5d7fa700 1 -- 172.21.15.44:0/2535336721 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4b6804bf20 con 0x7f4b7013ebf0 2020-11-20T12:12:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.242+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4b70149ea0 con 0x7f4b7013ebf0 2020-11-20T12:12:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.242+0000 7f4b5d7fa700 1 -- 172.21.15.44:0/2535336721 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f4b68015070 con 0x7f4b7013ebf0 2020-11-20T12:12:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4b48056cd0 msgr2=0x7f4b48059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4b48056cd0 0x7f4b48059190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f4b7013c990 tx=0x7f4b6c0079e0).stop 2020-11-20T12:12:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b7013ebf0 msgr2=0x7f4b7014a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b7013ebf0 0x7f4b7014a1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4b68009210 tx=0x7f4b68016000).stop 2020-11-20T12:12:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 shutdown_connections 2020-11-20T12:12:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4b48056cd0 0x7f4b48059190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b7013ebf0 0x7f4b7014a1c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b700ae050 0x7f4b700a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 --2- 172.21.15.44:0/2535336721 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4b70138220 0x7f4b700a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 >> 172.21.15.44:0/2535336721 conn(0x7f4b70062bc0 msgr2=0x7f4b7005ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 shutdown_connections 2020-11-20T12:12:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:39.244+0000 7f4b7766b700 1 -- 172.21.15.44:0/2535336721 wait complete. 2020-11-20T12:12:40.456 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":106,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:12:43.459 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:12:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.594+0000 7fa156141700 1 Processor -- start 2020-11-20T12:12:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.594+0000 7fa156141700 1 -- start start 2020-11-20T12:12:43.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa156141700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 0x7fa1500a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:43.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa156141700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1501385f0 con 0x7fa1500ab0e0 2020-11-20T12:12:43.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa156141700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1500aca50 con 0x7fa1500aad00 2020-11-20T12:12:43.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa156141700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15013e250 con 0x7fa1500a86d0 2020-11-20T12:12:43.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa147fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa1500ab0e0 0x7fa1500ac340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44226/0 (socket says 172.21.15.44:44226) 2020-11-20T12:12:43.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa14ffff700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 msgr2=0x7fa1500a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:43.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 learned_addr learned my addr 172.21.15.44:0/2845451678 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:43.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.595+0000 7fa14ffff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 0x7fa1500a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:43.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.796+0000 7fa14ffff700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 msgr2=0x7fa1500a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:43.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:43.796+0000 7fa14ffff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 0x7fa1500a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:12:44.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:44.197+0000 7fa14ffff700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 msgr2=0x7fa1500a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:44.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:44.197+0000 7fa14ffff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 0x7fa1500a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:12:45.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:44.998+0000 7fa14ffff700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 msgr2=0x7fa1500a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:45.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:44.998+0000 7fa14ffff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 0x7fa1500a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:12:46.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 msgr2=0x7fa1500a94d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1500a86d0 0x7fa1500a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 >> v1:172.21.15.79:6789/0 conn(0x7fa1500aad00 legacy=0x7fa1500abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 >> v1:172.21.15.44:6789/0 conn(0x7fa1500ab0e0 legacy=0x7fa1500ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15013e250 con 0x7fa134003400 2020-11-20T12:12:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1340089f0 con 0x7fa134005f10 2020-11-20T12:12:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:46.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:46.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:46.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:46.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:46.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:46.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:46.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:46.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.595+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:12:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:46.796+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:12:47.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:47.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:47.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:47.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:47.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:47.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:12:47.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:47.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:47.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:47.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.197+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:12:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:12:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:48.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:47.998+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:12:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14ffff700 1 -- 172.21.15.44:0/2845451678 reap_dead start 2020-11-20T12:12:49.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa13400b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15013e250 con 0x7fa134003400 2020-11-20T12:12:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1340089f0 con 0x7fa134005f10 2020-11-20T12:12:49.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14dffb700 1 -- 172.21.15.44:0/2845451678 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13400b890 con 0x7fa134008e60 2020-11-20T12:12:49.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:49.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14ffff700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 msgr2=0x7fa13400b250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:49.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14ffff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa13400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:49.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:12:49.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 msgr2=0x7fa134005820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:49.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:49.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:49.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:49.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:12:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14f7fe700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:12:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.595+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:12:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.596+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:49.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.44:3300/0 conn(0x7fa134003400 0x7fa134005820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:49.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa14ffff700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 msgr2=0x7fa13400b250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:49.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa14ffff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa13400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:12:49.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 msgr2=0x7fa13400b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:49.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa13400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:49.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 msgr2=0x7fa134008370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:49.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:49.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa147fff700 1 -- 172.21.15.44:0/2845451678 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15013e4f0 con 0x7fa134003400 2020-11-20T12:12:49.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa147fff700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa134005820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fa1500ab6a0 tx=0x7fa138009040).ready entity=mon.0 client_cookie=74b89d1cb5c5015b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:49.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa14effd700 1 -- 172.21.15.44:0/2845451678 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13801d070 con 0x7fa134003400 2020-11-20T12:12:49.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa14effd700 1 -- 172.21.15.44:0/2845451678 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1380187e0 con 0x7fa134003400 2020-11-20T12:12:49.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.796+0000 7fa14effd700 1 -- 172.21.15.44:0/2845451678 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138018b60 con 0x7fa134003400 2020-11-20T12:12:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 -- 172.21.15.44:0/2845451678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 msgr2=0x7fa134005820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa134005820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fa1500ab6a0 tx=0x7fa138009040).stop 2020-11-20T12:12:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 -- 172.21.15.44:0/2845451678 shutdown_connections 2020-11-20T12:12:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa134005820 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 --2- 172.21.15.44:0/2845451678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa13400b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 --2- 172.21.15.44:0/2845451678 >> v2:172.21.15.79:3300/0 conn(0x7fa134005f10 0x7fa134008370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 -- 172.21.15.44:0/2845451678 >> 172.21.15.44:0/2845451678 conn(0x7fa150060170 msgr2=0x7fa150060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 -- 172.21.15.44:0/2845451678 shutdown_connections 2020-11-20T12:12:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.797+0000 7fa156141700 1 -- 172.21.15.44:0/2845451678 wait complete. 2020-11-20T12:12:49.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.798+0000 7fa156141700 1 Processor -- start 2020-11-20T12:12:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa156141700 1 -- start start 2020-11-20T12:12:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa156141700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa15009deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa156141700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa134005f10 0x7fa150139a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa156141700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa1501350a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:49.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa156141700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15013b700 con 0x7fa134003400 2020-11-20T12:12:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa156141700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150139350 con 0x7fa134005f10 2020-11-20T12:12:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa156141700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15014b470 con 0x7fa134008e60 2020-11-20T12:12:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa14f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa15009deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa14f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa15009deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40516/0 (socket says 172.21.15.44:40516) 2020-11-20T12:12:49.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa14f7fe700 1 -- 172.21.15.44:0/4033670235 learned_addr learned my addr 172.21.15.44:0/4033670235 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa14ffff700 1 -- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 msgr2=0x7fa1501350a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa14ffff700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa1501350a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.799+0000 7fa147fff700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa134005f10 0x7fa150139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:49.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa14f7fe700 1 -- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 msgr2=0x7fa1501350a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:12:49.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa1501350a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:49.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa14f7fe700 1 -- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa134005f10 msgr2=0x7fa150139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:49.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa134005f10 0x7fa150139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:49.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa14f7fe700 1 -- 172.21.15.44:0/4033670235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15013ac40 con 0x7fa134003400 2020-11-20T12:12:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa14f7fe700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa15009deb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fa140003660 tx=0x7fa140009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa1477fe700 1 -- 172.21.15.44:0/4033670235 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140014850 con 0x7fa134003400 2020-11-20T12:12:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa1477fe700 1 -- 172.21.15.44:0/4033670235 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1400149b0 con 0x7fa134003400 2020-11-20T12:12:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15013aee0 con 0x7fa134003400 2020-11-20T12:12:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa1477fe700 1 -- 172.21.15.44:0/4033670235 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140014d30 con 0x7fa134003400 2020-11-20T12:12:49.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.800+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1501395e0 con 0x7fa134003400 2020-11-20T12:12:49.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.801+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa150139780 con 0x7fa134003400 2020-11-20T12:12:49.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.801+0000 7fa1477fe700 1 -- 172.21.15.44:0/4033670235 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa140028070 con 0x7fa134003400 2020-11-20T12:12:49.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.801+0000 7fa1477fe700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa12c056d00 0x7fa12c0591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:49.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.802+0000 7fa1477fe700 1 -- 172.21.15.44:0/4033670235 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa140070be0 con 0x7fa134003400 2020-11-20T12:12:49.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.802+0000 7fa147fff700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa12c056d00 0x7fa12c0591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:49.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.802+0000 7fa147fff700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa12c056d00 0x7fa12c0591c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa13801c000 tx=0x7fa13801f000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:49.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:49.805+0000 7fa1477fe700 1 -- 172.21.15.44:0/4033670235 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa150139780 con 0x7fa134003400 2020-11-20T12:12:50.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.092+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa1500a9160 con 0x7fa134003400 2020-11-20T12:12:51.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.092+0000 7fa1477fe700 1 -- 172.21.15.44:0/4033670235 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fa140045eb0 con 0x7fa134003400 2020-11-20T12:12:51.269 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":110,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:12:51.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa12c056d00 msgr2=0x7fa12c0591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa12c056d00 0x7fa12c0591c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa13801c000 tx=0x7fa13801f000).stop 2020-11-20T12:12:51.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 msgr2=0x7fa15009deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa15009deb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fa140003660 tx=0x7fa140009040).stop 2020-11-20T12:12:51.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 shutdown_connections 2020-11-20T12:12:51.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa12c056d00 0x7fa12c0591c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa134008e60 0x7fa1501350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:51.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa134003400 0x7fa15009deb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 --2- 172.21.15.44:0/4033670235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa134005f10 0x7fa150139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 >> 172.21.15.44:0/4033670235 conn(0x7fa150060170 msgr2=0x7fa150060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:51.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 shutdown_connections 2020-11-20T12:12:51.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:50.094+0000 7fa156141700 1 -- 172.21.15.44:0/4033670235 wait complete. 2020-11-20T12:12:51.273 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:12:51.274 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:12:51.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.444+0000 7f1506c98700 1 Processor -- start 2020-11-20T12:12:51.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.444+0000 7f1506c98700 1 -- start start 2020-11-20T12:12:51.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f1506c98700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f150013ebf0 0x7f1500060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f1506c98700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 0x7f1500137b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f1506c98700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150013e080 con 0x7f15000ae050 2020-11-20T12:12:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f1506c98700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1500060b40 con 0x7f150013ebf0 2020-11-20T12:12:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f1506c98700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1500060d60 con 0x7f15000669d0 2020-11-20T12:12:51.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f1505235700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 msgr2=0x7f1500137b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f14fffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f150013ebf0 0x7f1500060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f1505235700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 0x7f1500137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f14fffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f150013ebf0 0x7f1500060650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40528/0 (socket says 172.21.15.44:40528) 2020-11-20T12:12:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.445+0000 7f14fffff700 1 -- 172.21.15.44:0/2595066020 learned_addr learned my addr 172.21.15.44:0/2595066020 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:51.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14fffff700 1 -- 172.21.15.44:0/2595066020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 msgr2=0x7f1500137b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:51.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14fffff700 1 --2- 172.21.15.44:0/2595066020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 0x7f1500137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:51.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14fffff700 1 -- 172.21.15.44:0/2595066020 >> v1:172.21.15.79:6789/0 conn(0x7f15000ae050 legacy=0x7f150013e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14fffff700 1 -- 172.21.15.44:0/2595066020 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000ae430 con 0x7f150013ebf0 2020-11-20T12:12:51.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14fffff700 1 --2- 172.21.15.44:0/2595066020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013ebf0 0x7f1500060650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f14ec006fd0 tx=0x7f14ec006eb0).ready entity=mon.0 client_cookie=c122cf53242abd97 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14ff7fe700 1 -- 172.21.15.44:0/2595066020 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0098d0 con 0x7f150013ebf0 2020-11-20T12:12:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14ff7fe700 1 -- 172.21.15.44:0/2595066020 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec009a30 con 0x7f150013ebf0 2020-11-20T12:12:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f14ff7fe700 1 -- 172.21.15.44:0/2595066020 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec009db0 con 0x7f150013ebf0 2020-11-20T12:12:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f1506c98700 1 -- 172.21.15.44:0/2595066020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013ebf0 msgr2=0x7f1500060650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f1506c98700 1 --2- 172.21.15.44:0/2595066020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013ebf0 0x7f1500060650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f14ec006fd0 tx=0x7f14ec006eb0).stop 2020-11-20T12:12:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f1506c98700 1 -- 172.21.15.44:0/2595066020 shutdown_connections 2020-11-20T12:12:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f1506c98700 1 --2- 172.21.15.44:0/2595066020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 0x7f1500137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.446+0000 7f1506c98700 1 --2- 172.21.15.44:0/2595066020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013ebf0 0x7f1500060650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.447+0000 7f1506c98700 1 -- 172.21.15.44:0/2595066020 >> 172.21.15.44:0/2595066020 conn(0x7f1500062bc0 msgr2=0x7f1500062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:51.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.447+0000 7f1506c98700 1 -- 172.21.15.44:0/2595066020 shutdown_connections 2020-11-20T12:12:51.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.447+0000 7f1506c98700 1 -- 172.21.15.44:0/2595066020 wait complete. 2020-11-20T12:12:51.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.448+0000 7f1506c98700 1 Processor -- start 2020-11-20T12:12:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.448+0000 7f1506c98700 1 -- start start 2020-11-20T12:12:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1506c98700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 0x7f15000ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1506c98700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15000ae050 0x7f150013c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1506c98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f150013ebf0 0x7f150013c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1506c98700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150013ca50 con 0x7f15000ae050 2020-11-20T12:12:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1506c98700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500138990 con 0x7f150013ebf0 2020-11-20T12:12:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1506c98700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150014a860 con 0x7f15000669d0 2020-11-20T12:12:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15000ae050 0x7f150013c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1504a34700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 msgr2=0x7f15000ad340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15000ae050 0x7f150013c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40532/0 (socket says 172.21.15.44:40532) 2020-11-20T12:12:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 -- 172.21.15.44:0/388672333 learned_addr learned my addr 172.21.15.44:0/388672333 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1504a34700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 0x7f15000ad340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:51.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f1505235700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f150013ebf0 0x7f150013c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 msgr2=0x7f15000ad340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:12:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f15000669d0 0x7f15000ad340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f150013ebf0 msgr2=0x7f150013c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f150013ebf0 0x7f150013c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 -- 172.21.15.44:0/388672333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150014ab00 con 0x7f15000ae050 2020-11-20T12:12:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fffff700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15000ae050 0x7f150013c070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f14ec006ef0 tx=0x7f14ec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec01e460 con 0x7f15000ae050 2020-11-20T12:12:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec01e5e0 con 0x7f15000ae050 2020-11-20T12:12:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.449+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec01e9a0 con 0x7f15000ae050 2020-11-20T12:12:51.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.450+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150014ada0 con 0x7f15000ae050 2020-11-20T12:12:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.450+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150013d780 con 0x7f15000ae050 2020-11-20T12:12:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.451+0000 7f1506c98700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150014a430 0x7f1500139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.451+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000a50c0 con 0x7f150014a430 2020-11-20T12:12:51.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.451+0000 7f1504a34700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150014a430 0x7f1500139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.451+0000 7f1504a34700 1 -- 172.21.15.44:0/388672333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15000446d0 con 0x7f150014a430 2020-11-20T12:12:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.451+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f14ec025030 con 0x7f15000ae050 2020-11-20T12:12:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.451+0000 7f1504a34700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150014a430 0x7f1500139d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f14f400c5e0 tx=0x7f14f4007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14d8056cd0 0x7f14d8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f14ec0718e0 con 0x7f15000ae050 2020-11-20T12:12:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f40072e0 con 0x7f150014a430 2020-11-20T12:12:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f14f40054e0 con 0x7f150014a430 2020-11-20T12:12:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150014a430 msgr2=0x7f1500139d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150014a430 0x7f1500139d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f14f400c5e0 tx=0x7f14f4007b70).stop 2020-11-20T12:12:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150014a430 msgr2=0x7f1500139d90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f14fcff9700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150014a430 0x7f1500139d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.452+0000 7f1505235700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14d8056cd0 0x7f14d8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.453+0000 7f1505235700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14d8056cd0 0x7f14d8059190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f14f8003fd0 tx=0x7f14f80090f0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.464+0000 7f1506c98700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013a280 0x7f150014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.464+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500060fb0 con 0x7f150013a280 2020-11-20T12:12:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.464+0000 7f1504a34700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013a280 0x7f150014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f1504a34700 1 -- 172.21.15.44:0/388672333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f15000613d0 con 0x7f150013a280 2020-11-20T12:12:51.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f1504a34700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013a280 0x7f150014d480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f14f400c5e0 tx=0x7f14f4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4014c00 con 0x7f150013a280 2020-11-20T12:12:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4767 (secure 0 0 0) 0x7f14f40070d0 con 0x7f150013a280 2020-11-20T12:12:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013a280 msgr2=0x7f150014d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f14fcff9700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013a280 0x7f150014d480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f14f400c5e0 tx=0x7f14f4015000).stop 2020-11-20T12:12:51.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f14fcff9700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013a280 msgr2=0x7f150014d480 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:51.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.465+0000 7f14fcff9700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f150013a280 0x7f150014d480 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0x7f14f400c5e0 tx=0x7f14f4015000).stop 2020-11-20T12:12:51.472 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:51.472 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:51.472 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:51.473 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:12:51.473 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 110, 2020-11-20T12:12:51.473 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:51.474 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:51.474 INFO:teuthology.orchestra.run.smithi044.stdout: 1 2020-11-20T12:12:51.474 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.474 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:12:51.475 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:51.475 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:51.475 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:51.475 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:51.475 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:51.476 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:51.476 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:51.476 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:51.476 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:51.476 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:51.477 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:51.477 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.477 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:51.477 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:51.477 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:51.478 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:51.478 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:51.478 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:51.478 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:51.478 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:51.479 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:51.479 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:51.479 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.479 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:51.480 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:51.480 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:51.480 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:51.480 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:51.481 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:51.481 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:51.481 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:51.481 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:51.481 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:51.482 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:51.482 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:51.482 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:51.482 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:51.482 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:51.483 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:51.483 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:51.483 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:51.483 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:51.483 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:51.484 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:51.484 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:51.484 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:51.484 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.484 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:51.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.485 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:51.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.485 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:51.485 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:51.486 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:51.486 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:51.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:51.486 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:51.487 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:51.488 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:51.488 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.488 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.489 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:51.489 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:51.489 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:51.490 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:51.490 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:51.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.491 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:51.491 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:51.491 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:51.491 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:51.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:51.492 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:51.492 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:51.493 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:51.493 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.494 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.494 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:51.494 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:51.495 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:51.495 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:51.495 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:51.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.496 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:51.496 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:51.496 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:51.496 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:51.499 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.501 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:12:51.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.503 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:12:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.504 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:12:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.505 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:12:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.506 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:12:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:51.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.508 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:51.508 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:51.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14d8056cd0 msgr2=0x7f14d8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f14d8056cd0 0x7f14d8059190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f14f8003fd0 tx=0x7f14f80090f0).stop 2020-11-20T12:12:51.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15000ae050 msgr2=0x7f150013c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 --2- 172.21.15.44:0/388672333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15000ae050 0x7f150013c070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f14ec006ef0 tx=0x7f14ec015000).stop 2020-11-20T12:12:51.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1505235700 1 -- 172.21.15.44:0/388672333 reap_dead start 2020-11-20T12:12:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1505235700 1 -- 172.21.15.44:0/388672333 reap_dead start 2020-11-20T12:12:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 shutdown_connections 2020-11-20T12:12:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 >> 172.21.15.44:0/388672333 conn(0x7f1500062bc0 msgr2=0x7f150005ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 shutdown_connections 2020-11-20T12:12:51.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.466+0000 7f1506c98700 1 -- 172.21.15.44:0/388672333 wait complete. 2020-11-20T12:12:51.512 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:12:51.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.645+0000 7f9a286db700 1 Processor -- start 2020-11-20T12:12:51.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.645+0000 7f9a286db700 1 -- start start 2020-11-20T12:12:51.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a286db700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 0x7f9a200a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a286db700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a200a9770 con 0x7f9a200aa020 2020-11-20T12:12:51.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a286db700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a200a6da0 con 0x7f9a200aa400 2020-11-20T12:12:51.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a286db700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200a56c0 con 0x7f9a200a3e10 2020-11-20T12:12:51.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a26477700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9a200aa020 0x7f9a200a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44290/0 (socket says 172.21.15.44:44290) 2020-11-20T12:12:51.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a26477700 1 -- 172.21.15.44:0/1748004125 learned_addr learned my addr 172.21.15.44:0/1748004125 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:12:51.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a26c78700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 msgr2=0x7f9a200a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:51.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a26c78700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 0x7f9a200a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:51.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964278578 0 0) 0x7f9a1c0010d0 con 0x7f9a200aa020 2020-11-20T12:12:51.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.646+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a08003240 con 0x7f9a200aa020 2020-11-20T12:12:51.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477691507 0 0) 0x7f9a200a6da0 con 0x7f9a200aa400 2020-11-20T12:12:51.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a1c0010d0 con 0x7f9a200aa400 2020-11-20T12:12:51.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 432553855 0 0) 0x7f9a08003240 con 0x7f9a200aa020 2020-11-20T12:12:51.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 msgr2=0x7f9a200a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:51.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 0x7f9a200a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:51.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 >> v1:172.21.15.79:6789/0 conn(0x7f9a200aa400 legacy=0x7f9a200a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a200a7040 con 0x7f9a200aa020 2020-11-20T12:12:51.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f9a1c001670 con 0x7f9a200aa020 2020-11-20T12:12:51.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 >> v1:172.21.15.44:6789/0 conn(0x7f9a200aa020 legacy=0x7f9a200a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a080038f0 0x7f9a08003cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 0x7f9a080044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a080049d0 0x7f9a08004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200a56c0 con 0x7f9a080038f0 2020-11-20T12:12:51.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08005340 con 0x7f9a080049d0 2020-11-20T12:12:51.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0800d940 con 0x7f9a200a3e10 2020-11-20T12:12:51.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26477700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a080038f0 0x7f9a08003cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25c76700 1 -- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 msgr2=0x7f9a080044e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:51.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25c76700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 0x7f9a080044e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:51.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26c78700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a080049d0 0x7f9a08004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26477700 1 -- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 msgr2=0x7f9a080044e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:12:51.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26477700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a200a3e10 0x7f9a080044e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:51.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26477700 1 -- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a080049d0 msgr2=0x7f9a08004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:12:51.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26477700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a080049d0 0x7f9a08004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26477700 1 -- 172.21.15.44:0/1748004125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a08003240 con 0x7f9a080038f0 2020-11-20T12:12:51.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a26477700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a080038f0 0x7f9a08003cd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f9a1c00c440 tx=0x7f9a200a5f30).ready entity=mon.0 client_cookie=f61c10b080aaf31a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00ce00 con 0x7f9a080038f0 2020-11-20T12:12:51.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c0072e0 con 0x7f9a080038f0 2020-11-20T12:12:51.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.647+0000 7f9a25475700 1 -- 172.21.15.44:0/1748004125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0055f0 con 0x7f9a080038f0 2020-11-20T12:12:51.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.648+0000 7f9a286db700 1 -- 172.21.15.44:0/1748004125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a080038f0 msgr2=0x7f9a08003cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.648+0000 7f9a286db700 1 --2- 172.21.15.44:0/1748004125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a080038f0 0x7f9a08003cd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f9a1c00c440 tx=0x7f9a200a5f30).stop 2020-11-20T12:12:51.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.648+0000 7f9a26c78700 1 -- 172.21.15.44:0/1748004125 reap_dead start 2020-11-20T12:12:51.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.648+0000 7f9a286db700 1 -- 172.21.15.44:0/1748004125 shutdown_connections 2020-11-20T12:12:51.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.648+0000 7f9a286db700 1 -- 172.21.15.44:0/1748004125 >> 172.21.15.44:0/1748004125 conn(0x7f9a20062690 msgr2=0x7f9a200a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:51.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.648+0000 7f9a286db700 1 -- 172.21.15.44:0/1748004125 shutdown_connections 2020-11-20T12:12:51.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.648+0000 7f9a286db700 1 -- 172.21.15.44:0/1748004125 wait complete. 2020-11-20T12:12:51.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.649+0000 7f9a286db700 1 Processor -- start 2020-11-20T12:12:51.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.649+0000 7f9a286db700 1 -- start start 2020-11-20T12:12:51.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a286db700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a080038f0 0x7f9a2013a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a286db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20066610 0x7f9a200abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a286db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ac140 0x7f9a20149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a286db700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200ac5a0 con 0x7f9a200ac140 2020-11-20T12:12:51.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a286db700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200ab1f0 con 0x7f9a20066610 2020-11-20T12:12:51.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a286db700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200acd70 con 0x7f9a080038f0 2020-11-20T12:12:51.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26477700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a080038f0 msgr2=0x7f9a2013a410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:12:51.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ac140 0x7f9a20149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26477700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a080038f0 0x7f9a2013a410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:12:51.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ac140 0x7f9a20149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40556/0 (socket says 172.21.15.44:40556) 2020-11-20T12:12:51.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 -- 172.21.15.44:0/3821736806 learned_addr learned my addr 172.21.15.44:0/3821736806 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:12:51.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a25c76700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20066610 0x7f9a200abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a080038f0 msgr2=0x7f9a2013a410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:12:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9a080038f0 0x7f9a2013a410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:12:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20066610 msgr2=0x7f9a200abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20066610 0x7f9a200abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.650+0000 7f9a26c78700 1 -- 172.21.15.44:0/3821736806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a200ad010 con 0x7f9a200ac140 2020-11-20T12:12:51.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.651+0000 7f9a26c78700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ac140 0x7f9a20149450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9a200aa880 tx=0x7f9a18008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.651+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18009550 con 0x7f9a200ac140 2020-11-20T12:12:51.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.651+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a18003c60 con 0x7f9a200ac140 2020-11-20T12:12:51.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.651+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a200ad2b0 con 0x7f9a200ac140 2020-11-20T12:12:51.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.651+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800d6b0 con 0x7f9a200ac140 2020-11-20T12:12:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.651+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a20067fc0 con 0x7f9a200ac140 2020-11-20T12:12:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.652+0000 7f9a286db700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20149940 0x7f9a20149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.652+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20061650 con 0x7f9a20149940 2020-11-20T12:12:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.652+0000 7f9a26477700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20149940 0x7f9a20149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.652+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9a1801b030 con 0x7f9a200ac140 2020-11-20T12:12:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.652+0000 7f9a26477700 1 -- 172.21.15.44:0/3821736806 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a200446d0 con 0x7f9a20149940 2020-11-20T12:12:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.652+0000 7f9a26477700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20149940 0x7f9a20149d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9a1c00c440 tx=0x7f9a1c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.652+0000 7f9a0e7fc700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f99f8056cd0 0x7f99f8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9a1806fba0 con 0x7f9a200ac140 2020-11-20T12:12:51.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a25c76700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f99f8056cd0 0x7f99f8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0010a0 con 0x7f9a20149940 2020-11-20T12:12:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f9a1c005960 con 0x7f9a20149940 2020-11-20T12:12:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20149940 msgr2=0x7f9a20149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a0e7fc700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20149940 0x7f9a20149d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9a1c00c440 tx=0x7f9a1c015000).stop 2020-11-20T12:12:51.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20149940 msgr2=0x7f9a20149d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a0e7fc700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20149940 0x7f9a20149d20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.653+0000 7f9a25c76700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f99f8056cd0 0x7f99f8059190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9a10007fd0 tx=0x7f9a10009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.661+0000 7f9a286db700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20130370 0x7f9a20130780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:12:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.661+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200008d0 con 0x7f9a20130370 2020-11-20T12:12:51.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.662+0000 7f9a26477700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20130370 0x7f9a20130780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:12:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.662+0000 7f9a26477700 1 -- 172.21.15.44:0/3821736806 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9a200613b0 con 0x7f9a20130370 2020-11-20T12:12:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.662+0000 7f9a26477700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20130370 0x7f9a20130780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9a1c007900 tx=0x7f9a1c01c000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:12:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.662+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c013660 con 0x7f9a20130370 2020-11-20T12:12:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4595 (secure 0 0 0) 0x7f9a1c000e50 con 0x7f9a20130370 2020-11-20T12:12:51.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20130370 msgr2=0x7f9a20130780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a0e7fc700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20130370 0x7f9a20130780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9a1c007900 tx=0x7f9a1c01c000).stop 2020-11-20T12:12:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a0e7fc700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20130370 msgr2=0x7f9a20130780 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:12:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a0e7fc700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a20130370 0x7f9a20130780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f99f8056cd0 msgr2=0x7f99f8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f99f8056cd0 0x7f99f8059190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9a10007fd0 tx=0x7f9a10009040).stop 2020-11-20T12:12:51.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ac140 msgr2=0x7f9a20149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:12:51.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ac140 0x7f9a20149450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9a200aa880 tx=0x7f9a18008040).stop 2020-11-20T12:12:51.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a26c78700 1 -- 172.21.15.44:0/3821736806 reap_dead start 2020-11-20T12:12:51.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 shutdown_connections 2020-11-20T12:12:51.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 --2- 172.21.15.44:0/3821736806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a200ac140 0x7f9a20149450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:12:51.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 >> 172.21.15.44:0/3821736806 conn(0x7f9a20062690 msgr2=0x7f9a200a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:12:51.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 shutdown_connections 2020-11-20T12:12:51.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:12:51.663+0000 7f9a286db700 1 -- 172.21.15.44:0/3821736806 wait complete. 2020-11-20T12:12:51.680 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:12:51.680 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:51.680 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:51.680 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:12:51.680 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 110, 2020-11-20T12:12:51.681 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:12:51.681 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:12:51.681 INFO:teuthology.orchestra.run.smithi044.stdout: 1 2020-11-20T12:12:51.681 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.682 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:12:51.682 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:51.682 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:12:51.682 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:12:51.682 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:51.683 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:51.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:51.683 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:51.683 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:51.683 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:51.684 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:51.684 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:51.684 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.684 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:12:51.685 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:12:51.685 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:51.685 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:51.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:51.685 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:51.686 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:51.686 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:51.686 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:51.686 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:51.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.687 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:12:51.687 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:12:51.687 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:12:51.687 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:12:51.688 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:12:51.688 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:12:51.688 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:51.688 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:12:51.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:12:51.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:12:51.689 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:12:51.689 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:12:51.689 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:12:51.690 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:12:51.690 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:12:51.690 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:12:51.690 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:12:51.690 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:12:51.691 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:12:51.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:12:51.691 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:12:51.691 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:12:51.692 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:12:51.692 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.692 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:12:51.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:12:51.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.693 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:12:51.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:12:51.693 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:51.693 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:51.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:51.694 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:12:51.694 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:51.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:12:51.695 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.696 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.696 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:12:51.696 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:12:51.697 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:51.697 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:51.697 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:51.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.698 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:12:51.698 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:12:51.698 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:51.698 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:51.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:51.699 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:12:51.699 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:51.700 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:12:51.700 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.700 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.700 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.701 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:12:51.701 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:12:51.701 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:51.702 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:51.702 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:51.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.702 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:12:51.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:12:51.703 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:12:51.703 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:12:51.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:12:51.704 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:12:51.704 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:12:51.705 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:12:51.705 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:12:51.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.705 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.706 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:12:51.706 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:12:51.706 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:12:51.706 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:12:51.707 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:12:51.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.707 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.707 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:12:51.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:12:51.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.708 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.708 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.708 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:51.709 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.709 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:12:51.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.709 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.710 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.710 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:51.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.710 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.710 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:12:51.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.711 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.711 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.711 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:12:51.712 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.712 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:12:51.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:12:51.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:12:51.712 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:12:51.713 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:12:51.713 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:12:51.713 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:12:51.713 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:12:51.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:12:51.714 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:12:51.714 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:12:51.715 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:13:07.619 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:13:07.626 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:13:11.716 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2020-11-20T12:13:11.716 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2020-11-20T12:13:11.716 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-20T12:13:11.717 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-20T12:13:11.721 INFO:tasks.ceph.mon.c:Started 2020-11-20T12:13:11.721 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:13:11.722 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:13:11.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.855+0000 7feb820db700 1 Processor -- start 2020-11-20T12:13:11.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.855+0000 7feb820db700 1 -- start start 2020-11-20T12:13:11.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.855+0000 7feb820db700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb7c060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.855+0000 7feb820db700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c13e080 con 0x7feb7c0669d0 2020-11-20T12:13:11.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.855+0000 7feb820db700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c060d60 con 0x7feb7c0ae050 2020-11-20T12:13:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.855+0000 7feb820db700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c060b40 con 0x7feb7c13ebf0 2020-11-20T12:13:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.855+0000 7feb7affd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb7c060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7bfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7feb7c0669d0 0x7feb7c137b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44320/0 (socket says 172.21.15.44:44320) 2020-11-20T12:13:11.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7feb7c0ae050 0x7feb7c13e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47192/0 (socket says 172.21.15.44:47192) 2020-11-20T12:13:11.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7bfff700 1 -- 172.21.15.44:0/953207567 learned_addr learned my addr 172.21.15.44:0/953207567 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:11.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7affd700 1 -- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 msgr2=0x7feb7c060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:13:11.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7affd700 1 -- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 msgr2=0x7feb7c060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:11.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7affd700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb7c060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:11.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7affd700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb7c060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:11.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670996354 0 0) 0x7feb7c060d60 con 0x7feb7c0ae050 2020-11-20T12:13:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb68003240 con 0x7feb7c0ae050 2020-11-20T12:13:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709225965 0 0) 0x7feb7c13e080 con 0x7feb7c0669d0 2020-11-20T12:13:11.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb7c060d60 con 0x7feb7c0669d0 2020-11-20T12:13:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3926312562 0 0) 0x7feb68003240 con 0x7feb7c0ae050 2020-11-20T12:13:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 msgr2=0x7feb7c060650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb7c060650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:11.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 >> v1:172.21.15.44:6789/0 conn(0x7feb7c0669d0 legacy=0x7feb7c137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c0ae430 con 0x7feb7c0ae050 2020-11-20T12:13:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7feb6c0014e0 con 0x7feb7c0ae050 2020-11-20T12:13:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 >> v1:172.21.15.79:6789/0 conn(0x7feb7c0ae050 legacy=0x7feb7c13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb68004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb68004e40 0x7feb68003480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68003970 0x7feb68003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c060b40 con 0x7feb68003970 2020-11-20T12:13:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb68004310 con 0x7feb68004e40 2020-11-20T12:13:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6800b780 con 0x7feb7c13ebf0 2020-11-20T12:13:11.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7affd700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb68004e40 0x7feb68003480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7b7fe700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb68004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.856+0000 7feb7bfff700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68003970 0x7feb68003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7affd700 1 -- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 msgr2=0x7feb68004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7affd700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13ebf0 0x7feb68004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:11.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7affd700 1 -- 172.21.15.44:0/953207567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68003970 msgr2=0x7feb68003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7affd700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68003970 0x7feb68003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7affd700 1 -- 172.21.15.44:0/953207567 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb68003240 con 0x7feb68004e40 2020-11-20T12:13:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7affd700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb68004e40 0x7feb68003480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7feb7c13f010 tx=0x7feb70008040).ready entity=mon.1 client_cookie=837513b968f1b0dc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000d830 con 0x7feb68004e40 2020-11-20T12:13:11.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7000d990 con 0x7feb68004e40 2020-11-20T12:13:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.857+0000 7feb7a7fc700 1 -- 172.21.15.44:0/953207567 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000dd10 con 0x7feb68004e40 2020-11-20T12:13:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.858+0000 7feb820db700 1 -- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb68004e40 msgr2=0x7feb68003480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.858+0000 7feb820db700 1 --2- 172.21.15.44:0/953207567 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb68004e40 0x7feb68003480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7feb7c13f010 tx=0x7feb70008040).stop 2020-11-20T12:13:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.858+0000 7feb7bfff700 1 -- 172.21.15.44:0/953207567 reap_dead start 2020-11-20T12:13:11.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.858+0000 7feb820db700 1 -- 172.21.15.44:0/953207567 shutdown_connections 2020-11-20T12:13:11.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.858+0000 7feb820db700 1 -- 172.21.15.44:0/953207567 >> 172.21.15.44:0/953207567 conn(0x7feb7c062bc0 msgr2=0x7feb7c062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:11.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.858+0000 7feb820db700 1 -- 172.21.15.44:0/953207567 shutdown_connections 2020-11-20T12:13:11.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.858+0000 7feb820db700 1 -- 172.21.15.44:0/953207567 wait complete. 2020-11-20T12:13:11.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.859+0000 7feb820db700 1 Processor -- start 2020-11-20T12:13:11.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.859+0000 7feb820db700 1 -- start start 2020-11-20T12:13:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb820db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68004e40 0x7feb7c135ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb820db700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 0x7feb7c0a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb820db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb7c14d110 0x7feb7c14d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb820db700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c13bc50 con 0x7feb68004e40 2020-11-20T12:13:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb820db700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c13d0a0 con 0x7feb7c14d110 2020-11-20T12:13:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb820db700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c136090 con 0x7feb7c13d7a0 2020-11-20T12:13:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68004e40 0x7feb7c135ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7affd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 0x7feb7c0a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb7c14d110 0x7feb7c14d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68004e40 0x7feb7c135ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40580/0 (socket says 172.21.15.44:40580) 2020-11-20T12:13:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7b7fe700 1 -- 172.21.15.44:0/4206726205 learned_addr learned my addr 172.21.15.44:0/4206726205 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7affd700 1 -- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 msgr2=0x7feb7c0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:13:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7affd700 1 -- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 msgr2=0x7feb7c0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:13:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7affd700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 0x7feb7c0a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7affd700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 0x7feb7c0a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7bfff700 1 -- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 msgr2=0x7feb7c0a1880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:13:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7bfff700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 0x7feb7c0a1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7bfff700 1 -- 172.21.15.44:0/4206726205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68004e40 msgr2=0x7feb7c135ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7bfff700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68004e40 0x7feb7c135ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7bfff700 1 -- 172.21.15.44:0/4206726205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c13d300 con 0x7feb7c14d110 2020-11-20T12:13:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.860+0000 7feb7b7fe700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68004e40 0x7feb7c135ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:13:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.861+0000 7feb7bfff700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb7c14d110 0x7feb7c14d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7feb7c13e210 tx=0x7feb74007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.861+0000 7feb5b7fe700 1 -- 172.21.15.44:0/4206726205 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400fbd0 con 0x7feb7c14d110 2020-11-20T12:13:11.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.861+0000 7feb5b7fe700 1 -- 172.21.15.44:0/4206726205 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7400fd30 con 0x7feb7c14d110 2020-11-20T12:13:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.861+0000 7feb5b7fe700 1 -- 172.21.15.44:0/4206726205 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74012580 con 0x7feb7c14d110 2020-11-20T12:13:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.861+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c136380 con 0x7feb7c14d110 2020-11-20T12:13:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.861+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c13c440 con 0x7feb7c14d110 2020-11-20T12:13:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.862+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c0a50c0 con 0x7feb7c14d110 2020-11-20T12:13:11.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.862+0000 7feb5b7fe700 1 -- 172.21.15.44:0/4206726205 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7feb74012930 con 0x7feb7c14d110 2020-11-20T12:13:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.862+0000 7feb5b7fe700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb54056d20 0x7feb540591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.862+0000 7feb5b7fe700 1 -- 172.21.15.44:0/4206726205 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7feb7406fbf0 con 0x7feb7c14d110 2020-11-20T12:13:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.863+0000 7feb7b7fe700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb54056d20 0x7feb540591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:11.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.863+0000 7feb7b7fe700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb54056d20 0x7feb540591e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7feb6c007520 tx=0x7feb6c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:11.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:11.866+0000 7feb5b7fe700 1 -- 172.21.15.44:0/4206726205 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7feb74044ec0 con 0x7feb7c14d110 2020-11-20T12:13:12.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.129+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feb7c0446d0 con 0x7feb7c14d110 2020-11-20T12:13:12.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.129+0000 7feb5b7fe700 1 -- 172.21.15.44:0/4206726205 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7feb74019030 con 0x7feb7c14d110 2020-11-20T12:13:12.133 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":110,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:13:12.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb54056d20 msgr2=0x7feb540591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:12.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb54056d20 0x7feb540591e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7feb6c007520 tx=0x7feb6c009040).stop 2020-11-20T12:13:12.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb7c14d110 msgr2=0x7feb7c14d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:12.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb7c14d110 0x7feb7c14d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7feb7c13e210 tx=0x7feb74007a00).stop 2020-11-20T12:13:12.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 shutdown_connections 2020-11-20T12:13:12.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7feb54056d20 0x7feb540591e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:12.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb7c14d110 0x7feb7c14d570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:12.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb68004e40 0x7feb7c135ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:12.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 --2- 172.21.15.44:0/4206726205 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feb7c13d7a0 0x7feb7c0a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:12.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 >> 172.21.15.44:0/4206726205 conn(0x7feb7c062bc0 msgr2=0x7feb7c05ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:12.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 shutdown_connections 2020-11-20T12:13:12.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:12.131+0000 7feb820db700 1 -- 172.21.15.44:0/4206726205 wait complete. 2020-11-20T12:13:15.147 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:13:15.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.276+0000 7fc75a53e700 1 Processor -- start 2020-11-20T12:13:15.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.276+0000 7fc75a53e700 1 -- start start 2020-11-20T12:13:15.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc75a53e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc75413a990 0x7fc75413adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:15.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc75a53e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc75413f800 0x7fc754141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:15.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc75a53e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7540a6260 con 0x7fc754060330 2020-11-20T12:13:15.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc75a53e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540a52e0 con 0x7fc75413f800 2020-11-20T12:13:15.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc75a53e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540a6490 con 0x7fc75413a990 2020-11-20T12:13:15.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc75413f800 0x7fc754141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:15.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc75413f800 0x7fc754141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40592/0 (socket says 172.21.15.44:40592) 2020-11-20T12:13:15.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 -- 172.21.15.44:0/3919574846 learned_addr learned my addr 172.21.15.44:0/3919574846 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:15.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc7537fe700 1 --2- 172.21.15.44:0/3919574846 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc75413a990 0x7fc75413adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:15.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 -- 172.21.15.44:0/3919574846 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc75413a990 msgr2=0x7fc75413adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:15.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 --2- 172.21.15.44:0/3919574846 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc75413a990 0x7fc75413adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 -- 172.21.15.44:0/3919574846 >> v1:172.21.15.44:6789/0 conn(0x7fc754060330 legacy=0x7fc7540a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:15.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 -- 172.21.15.44:0/3919574846 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540a6730 con 0x7fc75413f800 2020-11-20T12:13:15.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc758adb700 1 --2- 172.21.15.44:0/3919574846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc754141bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc74c009d50 tx=0x7fc74c009290).ready entity=mon.0 client_cookie=90d089b312c0c6d4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:15.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc752ffd700 1 -- 172.21.15.44:0/3919574846 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00bc80 con 0x7fc75413f800 2020-11-20T12:13:15.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc752ffd700 1 -- 172.21.15.44:0/3919574846 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c00bde0 con 0x7fc75413f800 2020-11-20T12:13:15.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.277+0000 7fc752ffd700 1 -- 172.21.15.44:0/3919574846 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c010620 con 0x7fc75413f800 2020-11-20T12:13:15.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 -- 172.21.15.44:0/3919574846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 msgr2=0x7fc754141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:15.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 --2- 172.21.15.44:0/3919574846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc754141bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc74c009d50 tx=0x7fc74c009290).stop 2020-11-20T12:13:15.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 -- 172.21.15.44:0/3919574846 shutdown_connections 2020-11-20T12:13:15.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 --2- 172.21.15.44:0/3919574846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc754141bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 --2- 172.21.15.44:0/3919574846 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc75413a990 0x7fc75413adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 -- 172.21.15.44:0/3919574846 >> 172.21.15.44:0/3919574846 conn(0x7fc754062690 msgr2=0x7fc7540a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 -- 172.21.15.44:0/3919574846 shutdown_connections 2020-11-20T12:13:15.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.278+0000 7fc75a53e700 1 -- 172.21.15.44:0/3919574846 wait complete. 2020-11-20T12:13:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.279+0000 7fc75a53e700 1 Processor -- start 2020-11-20T12:13:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.279+0000 7fc75a53e700 1 -- start start 2020-11-20T12:13:15.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc75a53e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 0x7fc7540ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc75a53e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc75413a990 0x7fc75413b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc75a53e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc7540abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc75a53e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540ad3e0 con 0x7fc75413f800 2020-11-20T12:13:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc75a53e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540ab140 con 0x7fc75413a990 2020-11-20T12:13:15.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc75a53e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540ab3e0 con 0x7fc754060330 2020-11-20T12:13:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc7540abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc7540abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40598/0 (socket says 172.21.15.44:40598) 2020-11-20T12:13:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 -- 172.21.15.44:0/2865892459 learned_addr learned my addr 172.21.15.44:0/2865892459 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc753fff700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 0x7fc7540ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:15.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc7537fe700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc75413a990 0x7fc75413b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc753fff700 1 -- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 msgr2=0x7fc7540ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:13:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc753fff700 1 -- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 msgr2=0x7fc7540ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:13:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc753fff700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 0x7fc7540ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:15.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc753fff700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 0x7fc7540ac950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 -- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 msgr2=0x7fc7540ac950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:13:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 0x7fc7540ac950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 -- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc75413a990 msgr2=0x7fc75413b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc75413a990 0x7fc75413b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.280+0000 7fc758adb700 1 -- 172.21.15.44:0/2865892459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540ab680 con 0x7fc75413f800 2020-11-20T12:13:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.281+0000 7fc758adb700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc7540abb20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fc74c0092d0 tx=0x7fc74c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.281+0000 7fc733fff700 1 -- 172.21.15.44:0/2865892459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00bc80 con 0x7fc75413f800 2020-11-20T12:13:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.281+0000 7fc733fff700 1 -- 172.21.15.44:0/2865892459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c010e70 con 0x7fc75413f800 2020-11-20T12:13:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.281+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75413b740 con 0x7fc75413f800 2020-11-20T12:13:15.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.281+0000 7fc733fff700 1 -- 172.21.15.44:0/2865892459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c019de0 con 0x7fc75413f800 2020-11-20T12:13:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.281+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7541377b0 con 0x7fc75413f800 2020-11-20T12:13:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.282+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7540ab810 con 0x7fc75413f800 2020-11-20T12:13:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.282+0000 7fc733fff700 1 -- 172.21.15.44:0/2865892459 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc74c00e040 con 0x7fc75413f800 2020-11-20T12:13:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.282+0000 7fc733fff700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc72c056d20 0x7fc72c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:15.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.282+0000 7fc733fff700 1 -- 172.21.15.44:0/2865892459 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc74c072070 con 0x7fc75413f800 2020-11-20T12:13:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.283+0000 7fc753fff700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc72c056d20 0x7fc72c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.283+0000 7fc753fff700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc72c056d20 0x7fc72c0591e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fc7440079a0 tx=0x7fc744009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:15.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.286+0000 7fc733fff700 1 -- 172.21.15.44:0/2865892459 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc74c047340 con 0x7fc75413f800 2020-11-20T12:13:15.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.557+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc754061650 con 0x7fc75413f800 2020-11-20T12:13:15.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.557+0000 7fc733fff700 1 -- 172.21.15.44:0/2865892459 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fc74c028070 con 0x7fc75413f800 2020-11-20T12:13:15.561 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":110,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:13:15.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc72c056d20 msgr2=0x7fc72c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:15.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc72c056d20 0x7fc72c0591e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fc7440079a0 tx=0x7fc744009040).stop 2020-11-20T12:13:15.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 msgr2=0x7fc7540abb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:15.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc7540abb20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fc74c0092d0 tx=0x7fc74c017000).stop 2020-11-20T12:13:15.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 shutdown_connections 2020-11-20T12:13:15.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc72c056d20 0x7fc72c0591e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc75413f800 0x7fc7540abb20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc754060330 0x7fc7540ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 --2- 172.21.15.44:0/2865892459 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc75413a990 0x7fc75413b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:15.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 >> 172.21.15.44:0/2865892459 conn(0x7fc754062690 msgr2=0x7fc754067760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:15.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 shutdown_connections 2020-11-20T12:13:15.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:15.559+0000 7fc75a53e700 1 -- 172.21.15.44:0/2865892459 wait complete. 2020-11-20T12:13:18.575 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:13:18.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.710+0000 7fd2e2849700 1 Processor -- start 2020-11-20T12:13:18.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.711+0000 7fd2e2849700 1 -- start start 2020-11-20T12:13:18.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.711+0000 7fd2e2849700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd2dc0ad350 0x7fd2dc0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:18.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.711+0000 7fd2e2849700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc0676f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:18.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.711+0000 7fd2e2849700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2dc0a4650 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.711+0000 7fd2e2849700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc135090 con 0x7fd2dc0ad350 2020-11-20T12:13:18.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.711+0000 7fd2e2849700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0a63b0 con 0x7fd2dc0ad730 2020-11-20T12:13:18.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1046700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc0676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:18.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd2dc0ad350 0x7fd2dc0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:18.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd2dc0ad350 0x7fd2dc0a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60886/0 (socket says 172.21.15.44:60886) 2020-11-20T12:13:18.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1046700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc0676f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52042/0 (socket says 172.21.15.44:52042) 2020-11-20T12:13:18.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 -- 172.21.15.44:0/22466536 learned_addr learned my addr 172.21.15.44:0/22466536 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:18.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 -- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 msgr2=0x7fd2dc0676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:18.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 --2- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc0676f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 -- 172.21.15.44:0/22466536 >> v1:172.21.15.79:6789/0 conn(0x7fd2dc0a40d0 legacy=0x7fd2dc1379f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 -- 172.21.15.44:0/22466536 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0a48f0 con 0x7fd2dc0ad350 2020-11-20T12:13:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e0845700 1 -- 172.21.15.44:0/22466536 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616707269 0 0) 0x7fd2dc0a4650 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1046700 1 --2- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc0676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:13:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.712+0000 7fd2e1847700 1 --2- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 0x7fd2dc0a5680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd2d8006fd0 tx=0x7fd2d8006eb0).ready entity=mon.1 client_cookie=44685d76adfff485 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e0845700 1 -- 172.21.15.44:0/22466536 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8009810 con 0x7fd2dc0ad350 2020-11-20T12:13:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e0845700 1 -- 172.21.15.44:0/22466536 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8009970 con 0x7fd2dc0ad350 2020-11-20T12:13:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e0845700 1 -- 172.21.15.44:0/22466536 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8009cf0 con 0x7fd2dc0ad350 2020-11-20T12:13:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 -- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 msgr2=0x7fd2dc0a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 --2- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 0x7fd2dc0a5680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd2d8006fd0 tx=0x7fd2d8006eb0).stop 2020-11-20T12:13:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 -- 172.21.15.44:0/22466536 shutdown_connections 2020-11-20T12:13:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 --2- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 0x7fd2dc0a5680 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 --2- 172.21.15.44:0/22466536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc0676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 -- 172.21.15.44:0/22466536 >> 172.21.15.44:0/22466536 conn(0x7fd2dc061b60 msgr2=0x7fd2dc061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:18.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 -- 172.21.15.44:0/22466536 shutdown_connections 2020-11-20T12:13:18.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.713+0000 7fd2e2849700 1 -- 172.21.15.44:0/22466536 wait complete. 2020-11-20T12:13:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.714+0000 7fd2e2849700 1 Processor -- start 2020-11-20T12:13:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.714+0000 7fd2e2849700 1 -- start start 2020-11-20T12:13:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e2849700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0a40d0 0x7fd2dc0667b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e2849700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 0x7fd2dc13be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e2849700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e2849700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc13ac70 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e2849700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc13af10 con 0x7fd2dc0ad350 2020-11-20T12:13:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e2849700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc13b1b0 con 0x7fd2dc0ad730 2020-11-20T12:13:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0a40d0 0x7fd2dc0667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0a40d0 0x7fd2dc0667b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40610/0 (socket says 172.21.15.44:40610) 2020-11-20T12:13:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 -- 172.21.15.44:0/3043587754 learned_addr learned my addr 172.21.15.44:0/3043587754 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e2048700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1046700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 0x7fd2dc13be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 -- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 msgr2=0x7fd2dc149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 -- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 msgr2=0x7fd2dc13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:18.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 0x7fd2dc13be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.715+0000 7fd2e1847700 1 -- 172.21.15.44:0/3043587754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc13c330 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.716+0000 7fd2e1847700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0a40d0 0x7fd2dc0667b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd2dc0626b0 tx=0x7fd2d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:18.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.716+0000 7fd2c9ffb700 1 -- 172.21.15.44:0/3043587754 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8011c80 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.716+0000 7fd2c9ffb700 1 -- 172.21.15.44:0/3043587754 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8011de0 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.716+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc13a7d0 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.716+0000 7fd2c9ffb700 1 -- 172.21.15.44:0/3043587754 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d801e670 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.716+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc14a600 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.717+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc13aaa0 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.718+0000 7fd2c9ffb700 1 -- 172.21.15.44:0/3043587754 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd2d8025030 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.718+0000 7fd2c9ffb700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd2b4056d20 0x7fd2b40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.718+0000 7fd2c9ffb700 1 -- 172.21.15.44:0/3043587754 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd2d8071920 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.718+0000 7fd2e1046700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd2b4056d20 0x7fd2b40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.719+0000 7fd2e1046700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd2b4056d20 0x7fd2b40591e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fd2dc066ec0 tx=0x7fd2cc006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.722+0000 7fd2c9ffb700 1 -- 172.21.15.44:0/3043587754 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd2d8046d20 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.989+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd2dc057d80 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.989+0000 7fd2c9ffb700 1 -- 172.21.15.44:0/3043587754 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fd2d8023080 con 0x7fd2dc0a40d0 2020-11-20T12:13:18.993 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:13:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd2b4056d20 msgr2=0x7fd2b40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:18.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd2b4056d20 0x7fd2b40591e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fd2dc066ec0 tx=0x7fd2cc006df0).stop 2020-11-20T12:13:18.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0a40d0 msgr2=0x7fd2dc0667b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:18.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0a40d0 0x7fd2dc0667b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fd2dc0626b0 tx=0x7fd2d8015000).stop 2020-11-20T12:13:18.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 shutdown_connections 2020-11-20T12:13:18.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd2b4056d20 0x7fd2b40591e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd2dc0ad730 0x7fd2dc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2dc0a40d0 0x7fd2dc0667b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 --2- 172.21.15.44:0/3043587754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2dc0ad350 0x7fd2dc13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 >> 172.21.15.44:0/3043587754 conn(0x7fd2dc061b60 msgr2=0x7fd2dc0a4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 shutdown_connections 2020-11-20T12:13:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:18.991+0000 7fd2e2849700 1 -- 172.21.15.44:0/3043587754 wait complete. 2020-11-20T12:13:19.005 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:13:19.006 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:13:19.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.144+0000 7f0a190bd700 1 Processor -- start 2020-11-20T12:13:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.144+0000 7f0a190bd700 1 -- start start 2020-11-20T12:13:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a190bd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a1413a990 0x7f0a1413ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a190bd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0a1413f800 0x7f0a14141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a190bd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140a6260 con 0x7f0a14060330 2020-11-20T12:13:19.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a190bd700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a140a52e0 con 0x7f0a1413a990 2020-11-20T12:13:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a190bd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a140a6490 con 0x7f0a1413f800 2020-11-20T12:13:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1259c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a1413a990 0x7f0a1413ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1259c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a1413a990 0x7f0a1413ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40620/0 (socket says 172.21.15.44:40620) 2020-11-20T12:13:19.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1259c700 1 -- 172.21.15.44:0/1501106809 learned_addr learned my addr 172.21.15.44:0/1501106809 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1359e700 1 --2- 172.21.15.44:0/1501106809 >> v2:172.21.15.79:3300/0 conn(0x7f0a1413f800 0x7f0a14141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1259c700 1 -- 172.21.15.44:0/1501106809 >> v2:172.21.15.79:3300/0 conn(0x7f0a1413f800 msgr2=0x7f0a14141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1259c700 1 --2- 172.21.15.44:0/1501106809 >> v2:172.21.15.79:3300/0 conn(0x7f0a1413f800 0x7f0a14141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1259c700 1 -- 172.21.15.44:0/1501106809 >> v1:172.21.15.79:6789/0 conn(0x7f0a14060330 legacy=0x7f0a140a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:19.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.145+0000 7f0a1259c700 1 -- 172.21.15.44:0/1501106809 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140a6730 con 0x7f0a1413a990 2020-11-20T12:13:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a1259c700 1 --2- 172.21.15.44:0/1501106809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a1413ae20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0a140a5a90 tx=0x7f0a08006eb0).ready entity=mon.0 client_cookie=3feeb831234357b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a11d9b700 1 -- 172.21.15.44:0/1501106809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080098d0 con 0x7f0a1413a990 2020-11-20T12:13:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a11d9b700 1 -- 172.21.15.44:0/1501106809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08009a30 con 0x7f0a1413a990 2020-11-20T12:13:19.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a11d9b700 1 -- 172.21.15.44:0/1501106809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08009db0 con 0x7f0a1413a990 2020-11-20T12:13:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a190bd700 1 -- 172.21.15.44:0/1501106809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 msgr2=0x7f0a1413ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a190bd700 1 --2- 172.21.15.44:0/1501106809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a1413ae20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0a140a5a90 tx=0x7f0a08006eb0).stop 2020-11-20T12:13:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a190bd700 1 -- 172.21.15.44:0/1501106809 shutdown_connections 2020-11-20T12:13:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a190bd700 1 --2- 172.21.15.44:0/1501106809 >> v2:172.21.15.79:3300/0 conn(0x7f0a1413f800 0x7f0a14141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a190bd700 1 --2- 172.21.15.44:0/1501106809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a1413ae20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.146+0000 7f0a190bd700 1 -- 172.21.15.44:0/1501106809 >> 172.21.15.44:0/1501106809 conn(0x7f0a14062690 msgr2=0x7f0a140a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.147+0000 7f0a190bd700 1 -- 172.21.15.44:0/1501106809 shutdown_connections 2020-11-20T12:13:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.147+0000 7f0a190bd700 1 -- 172.21.15.44:0/1501106809 wait complete. 2020-11-20T12:13:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 Processor -- start 2020-11-20T12:13:19.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 -- start start 2020-11-20T12:13:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a14060330 0x7f0a141367d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a140ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a1413f800 0x7f0a1414a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a141375c0 con 0x7f0a1413a990 2020-11-20T12:13:19.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14137860 con 0x7f0a1413f800 2020-11-20T12:13:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.148+0000 7f0a190bd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14137d00 con 0x7f0a14060330 2020-11-20T12:13:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a140ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a140ac400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40626/0 (socket says 172.21.15.44:40626) 2020-11-20T12:13:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 -- 172.21.15.44:0/44426478 learned_addr learned my addr 172.21.15.44:0/44426478 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:19.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a12d9d700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a14060330 0x7f0a141367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1359e700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a1413f800 0x7f0a1414a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a14060330 msgr2=0x7f0a141367d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a14060330 0x7f0a141367d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a1413f800 msgr2=0x7f0a1414a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a1413f800 0x7f0a1414a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 -- 172.21.15.44:0/44426478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140ac8f0 con 0x7f0a1413a990 2020-11-20T12:13:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a1259c700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a140ac400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0a08006ef0 tx=0x7f0a08015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08011d10 con 0x7f0a1413a990 2020-11-20T12:13:19.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08011e70 con 0x7f0a1413a990 2020-11-20T12:13:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a140a3390 con 0x7f0a1413a990 2020-11-20T12:13:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0801e670 con 0x7f0a1413a990 2020-11-20T12:13:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.149+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1414a880 con 0x7f0a1413a990 2020-11-20T12:13:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.150+0000 7f0a190bd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1412f100 0x7f0a1412f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.150+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14137440 con 0x7f0a1412f100 2020-11-20T12:13:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.150+0000 7f0a12d9d700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1412f100 0x7f0a1412f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f0a12d9d700 1 -- 172.21.15.44:0/44426478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a14137a20 con 0x7f0a1412f100 2020-11-20T12:13:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f0a12d9d700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1412f100 0x7f0a1412f4e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f0a0400f820 tx=0x7f0a04000b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0a08025030 con 0x7f0a1413a990 2020-11-20T12:13:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09f0056d20 0x7f09f00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0a080716e0 con 0x7f0a1413a990 2020-11-20T12:13:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f0a1359e700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09f0056d20 0x7f09f00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400d6b0 con 0x7f0a1412f100 2020-11-20T12:13:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f0a0400d810 con 0x7f0a1412f100 2020-11-20T12:13:19.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1412f100 msgr2=0x7f0a1412f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1412f100 0x7f0a1412f4e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f0a0400f820 tx=0x7f0a04000b80).stop 2020-11-20T12:13:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1412f100 msgr2=0x7f0a1412f4e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.151+0000 7f09faffd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1412f100 0x7f0a1412f4e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.152+0000 7f0a1359e700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09f0056d20 0x7f09f00591e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f0a140a6180 tx=0x7f0a0c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.159+0000 7f0a190bd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a140a7740 0x7f0a140a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.159+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14137700 con 0x7f0a140a7740 2020-11-20T12:13:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.159+0000 7f0a12d9d700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a140a7740 0x7f0a140a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a12d9d700 1 -- 172.21.15.44:0/44426478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0a14061650 con 0x7f0a140a7740 2020-11-20T12:13:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a12d9d700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a140a7740 0x7f0a140a7b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0a0400f820 tx=0x7f0a04016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04014c70 con 0x7f0a140a7740 2020-11-20T12:13:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7f0a0400d4a0 con 0x7f0a140a7740 2020-11-20T12:13:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a140a7740 msgr2=0x7f0a140a7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f09faffd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a140a7740 0x7f0a140a7b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0a0400f820 tx=0x7f0a04016000).stop 2020-11-20T12:13:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f09faffd700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a140a7740 msgr2=0x7f0a140a7b50 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f09faffd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a140a7740 0x7f0a140a7b50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09f0056d20 msgr2=0x7f09f00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a190bd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09f0056d20 0x7f09f00591e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f0a140a6180 tx=0x7f0a0c008040).stop 2020-11-20T12:13:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 msgr2=0x7f0a140ac400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a190bd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a140ac400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0a08006ef0 tx=0x7f0a08015000).stop 2020-11-20T12:13:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a1359e700 1 -- 172.21.15.44:0/44426478 reap_dead start 2020-11-20T12:13:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 shutdown_connections 2020-11-20T12:13:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.160+0000 7f0a190bd700 1 --2- 172.21.15.44:0/44426478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a1413a990 0x7f0a140ac400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.161+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 >> 172.21.15.44:0/44426478 conn(0x7f0a14062690 msgr2=0x7f0a141344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:19.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.161+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 shutdown_connections 2020-11-20T12:13:19.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:19.161+0000 7f0a190bd700 1 -- 172.21.15.44:0/44426478 wait complete. 2020-11-20T12:13:19.950 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:19.950 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:19.950 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:19.951 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:13:19.951 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 112, 2020-11-20T12:13:19.951 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:19.951 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:13:19.952 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:19.952 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:19.952 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:19.952 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:13:19.952 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:19.953 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:19.953 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:19.953 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:19.953 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:19.953 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:19.954 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:19.954 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:19.954 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:19.954 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:19.954 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:19.955 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:19.955 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:19.955 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:19.955 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:19.955 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:19.956 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:19.956 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:19.956 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:19.956 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:19.957 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:19.957 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:19.957 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:19.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.957 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:19.958 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:19.958 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:19.958 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:19.958 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:19.958 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:19.959 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:19.959 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:19.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:19.959 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:19.959 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:19.960 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:19.960 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:19.960 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:19.960 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:19.960 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:19.961 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:19.961 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:19.961 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:19.961 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:19.962 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:19.962 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:19.962 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:19.962 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:19.962 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:19.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.963 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:19.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.963 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:19.963 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:19.964 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:19.964 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:19.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:19.964 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:19.965 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:19.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:19.965 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:19.966 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:19.966 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.966 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:19.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.967 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:19.967 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:19.967 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:19.967 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:19.967 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:19.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.968 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:19.968 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:19.968 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:19.969 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:19.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:19.969 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:19.969 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:19.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:19.970 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:19.970 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:19.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.971 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:19.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.971 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:19.971 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:19.972 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:19.972 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:19.972 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:19.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.973 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:19.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:19.973 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:19.973 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:19.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:19.974 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:19.974 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:19.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:19.975 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:19.975 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:19.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.976 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:19.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.976 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:19.976 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:19.976 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:19.977 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:19.977 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:19.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:19.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.978 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:19.978 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:19.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.978 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:19.979 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:19.979 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:19.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.979 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:19.979 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:13:19.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.980 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:19.980 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:19.980 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:19.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.981 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:19.981 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:13:19.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.981 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:19.982 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:19.982 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:13:19.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.982 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:19.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:19.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.983 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:19.983 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:19.983 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:13:19.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:13:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:13:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:19.986 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:19.986 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:13:20.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.116+0000 7f94ad6c0700 1 Processor -- start 2020-11-20T12:13:20.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.116+0000 7f94ad6c0700 1 -- start start 2020-11-20T12:13:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94ad6c0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a80652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94ad6c0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94a8064570 0x7f94a8138840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94ad6c0700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a8136270 con 0x7f94a80a99e0 2020-11-20T12:13:20.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94ad6c0700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a8138d30 con 0x7f94a8064570 2020-11-20T12:13:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94ad6c0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8064b80 con 0x7f94a8064190 2020-11-20T12:13:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94a67fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94a8064570 0x7f94a8138840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94a67fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94a8064570 0x7f94a8138840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40638/0 (socket says 172.21.15.44:40638) 2020-11-20T12:13:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.117+0000 7f94a67fc700 1 -- 172.21.15.44:0/240193717 learned_addr learned my addr 172.21.15.44:0/240193717 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:20.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a6ffd700 1 --2- 172.21.15.44:0/240193717 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a80652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a67fc700 1 -- 172.21.15.44:0/240193717 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 msgr2=0x7f94a80652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:20.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a67fc700 1 --2- 172.21.15.44:0/240193717 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a80652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a67fc700 1 -- 172.21.15.44:0/240193717 >> v1:172.21.15.44:6789/0 conn(0x7f94a80a99e0 legacy=0x7f94a80a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a67fc700 1 -- 172.21.15.44:0/240193717 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8064e20 con 0x7f94a8064570 2020-11-20T12:13:20.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a6ffd700 1 --2- 172.21.15.44:0/240193717 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a80652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:13:20.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a67fc700 1 --2- 172.21.15.44:0/240193717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8138840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f949c006fd0 tx=0x7f949c006eb0).ready entity=mon.0 client_cookie=af81a91d15077a99 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a5ffb700 1 -- 172.21.15.44:0/240193717 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0098d0 con 0x7f94a8064570 2020-11-20T12:13:20.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a5ffb700 1 -- 172.21.15.44:0/240193717 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c009a30 con 0x7f94a8064570 2020-11-20T12:13:20.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.118+0000 7f94a5ffb700 1 -- 172.21.15.44:0/240193717 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c009db0 con 0x7f94a8064570 2020-11-20T12:13:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 -- 172.21.15.44:0/240193717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 msgr2=0x7f94a8138840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 --2- 172.21.15.44:0/240193717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8138840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f949c006fd0 tx=0x7f949c006eb0).stop 2020-11-20T12:13:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 -- 172.21.15.44:0/240193717 shutdown_connections 2020-11-20T12:13:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 --2- 172.21.15.44:0/240193717 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a80652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 --2- 172.21.15.44:0/240193717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8138840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 -- 172.21.15.44:0/240193717 >> 172.21.15.44:0/240193717 conn(0x7f94a80af040 msgr2=0x7f94a80af450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 -- 172.21.15.44:0/240193717 shutdown_connections 2020-11-20T12:13:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.119+0000 7f94ad6c0700 1 -- 172.21.15.44:0/240193717 wait complete. 2020-11-20T12:13:20.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 Processor -- start 2020-11-20T12:13:20.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 -- start start 2020-11-20T12:13:20.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a8136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a80a99e0 0x7f94a813ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81364b0 con 0x7f94a8064570 2020-11-20T12:13:20.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a813c8d0 con 0x7f94a80a99e0 2020-11-20T12:13:20.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.120+0000 7f94ad6c0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a813cb70 con 0x7f94a8064190 2020-11-20T12:13:20.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8139c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40644/0 (socket says 172.21.15.44:40644) 2020-11-20T12:13:20.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 -- 172.21.15.44:0/2506977380 learned_addr learned my addr 172.21.15.44:0/2506977380 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:20.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a6ffd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a8136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a77fe700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a80a99e0 0x7f94a813ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 msgr2=0x7f94a8136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f94a8064190 0x7f94a8136f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a80a99e0 msgr2=0x7f94a813ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a80a99e0 0x7f94a813ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 -- 172.21.15.44:0/2506977380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a813a6a0 con 0x7f94a8064570 2020-11-20T12:13:20.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94a67fc700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8139c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f949c006ef0 tx=0x7f949c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c011c80 con 0x7f94a8064570 2020-11-20T12:13:20.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c011de0 con 0x7f94a8064570 2020-11-20T12:13:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c01e670 con 0x7f94a8064570 2020-11-20T12:13:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a813a130 con 0x7f94a8064570 2020-11-20T12:13:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.121+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a8149d90 con 0x7f94a8064570 2020-11-20T12:13:20.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.122+0000 7f94ad6c0700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a813b030 0x7f94a814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.122+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a805cd80 con 0x7f94a813b030 2020-11-20T12:13:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.122+0000 7f94a6ffd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a813b030 0x7f94a814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f94a6ffd700 1 -- 172.21.15.44:0/2506977380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94a80a54f0 con 0x7f94a813b030 2020-11-20T12:13:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f949c025030 con 0x7f94a8064570 2020-11-20T12:13:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f94a6ffd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a813b030 0x7f94a814a3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9498010b00 tx=0x7f9498008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9484056d20 0x7f94840591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f949c0716d0 con 0x7f94a8064570 2020-11-20T12:13:20.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949800ea90 con 0x7f94a813b030 2020-11-20T12:13:20.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f94a77fe700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9484056d20 0x7f94840591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f949800ebf0 con 0x7f94a813b030 2020-11-20T12:13:20.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a813b030 msgr2=0x7f94a814a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a813b030 0x7f94a814a3a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9498010b00 tx=0x7f9498008040).stop 2020-11-20T12:13:20.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a813b030 msgr2=0x7f94a814a3a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:20.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.123+0000 7f948effd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a813b030 0x7f94a814a3a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.124+0000 7f94a77fe700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9484056d20 0x7f94840591e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f94a813c4e0 tx=0x7f94a0007f10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.132+0000 7f94ad6c0700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a814a890 0x7f94a814d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.132+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80446d0 con 0x7f94a814a890 2020-11-20T12:13:20.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.132+0000 7f94a6ffd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a814a890 0x7f94a814d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f94a6ffd700 1 -- 172.21.15.44:0/2506977380 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f94a814aca0 con 0x7f94a814a890 2020-11-20T12:13:20.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f94a6ffd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a814a890 0x7f94a814d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9498018000 tx=0x7f9498008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949801d770 con 0x7f94a814a890 2020-11-20T12:13:20.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f949800e880 con 0x7f94a814a890 2020-11-20T12:13:20.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a814a890 msgr2=0x7f94a814d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f948effd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a814a890 0x7f94a814d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9498018000 tx=0x7f9498008040).stop 2020-11-20T12:13:20.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f948effd700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a814a890 msgr2=0x7f94a814d360 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:20.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.133+0000 7f948effd700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94a814a890 0x7f94a814d360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9484056d20 msgr2=0x7f94840591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9484056d20 0x7f94840591e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f94a813c4e0 tx=0x7f94a0007f10).stop 2020-11-20T12:13:20.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 msgr2=0x7f94a8139c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 --2- 172.21.15.44:0/2506977380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94a8064570 0x7f94a8139c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f949c006ef0 tx=0x7f949c015000).stop 2020-11-20T12:13:20.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94a77fe700 1 -- 172.21.15.44:0/2506977380 reap_dead start 2020-11-20T12:13:20.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94a77fe700 1 -- 172.21.15.44:0/2506977380 reap_dead start 2020-11-20T12:13:20.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 shutdown_connections 2020-11-20T12:13:20.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 >> 172.21.15.44:0/2506977380 conn(0x7f94a80af040 msgr2=0x7f94a80aa120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:20.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 shutdown_connections 2020-11-20T12:13:20.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.134+0000 7f94ad6c0700 1 -- 172.21.15.44:0/2506977380 wait complete. 2020-11-20T12:13:20.146 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:20.146 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:13:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 112, 2020-11-20T12:13:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:13:20.147 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:13:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:20.148 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:20.149 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:20.150 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:20.151 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:20.152 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:20.153 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:20.153 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:20.153 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.153 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:20.154 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:20.154 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:20.154 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:20.154 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:20.155 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:20.155 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:20.155 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:20.155 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:20.155 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:20.156 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:20.156 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:20.156 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:20.156 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:20.156 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:20.157 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:20.157 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:20.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:20.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:20.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:20.158 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:20.158 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:20.158 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:20.158 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.158 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:20.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.159 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:20.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.159 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:20.159 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:20.160 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:20.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:20.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:20.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:20.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:20.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:20.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:20.163 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:20.163 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:20.163 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:20.163 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:20.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.164 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:20.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:20.164 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:20.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:20.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:20.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:20.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:20.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:20.166 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:20.167 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:20.168 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:20.168 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:20.168 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:20.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.169 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:20.169 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:20.169 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:20.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:20.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:20.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:20.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:20.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:20.171 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.171 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:20.172 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:20.172 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:20.172 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:20.173 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:20.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.173 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.173 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:20.174 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:20.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.174 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:20.174 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:20.174 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:20.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.175 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:13:20.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.175 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:20.176 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:20.176 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:20.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.176 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.177 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:13:20.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.177 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:20.177 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:20.177 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:13:20.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.178 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.178 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:20.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.178 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:20.179 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:20.179 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:20.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.179 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.180 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:20.180 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:20.181 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:13:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.321+0000 7f2f907c1700 1 Processor -- start 2020-11-20T12:13:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.321+0000 7f2f907c1700 1 -- start start 2020-11-20T12:13:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f907c1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2f88060330 0x7f2f880a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f907c1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f880a6260 con 0x7f2f8813b0a0 2020-11-20T12:13:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f907c1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f880a6490 con 0x7f2f880a4c30 2020-11-20T12:13:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f907c1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f880a52e0 con 0x7f2f88060330 2020-11-20T12:13:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f8e55d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2f88060330 0x7f2f880a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f8e55d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2f88060330 0x7f2f880a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40654/0 (socket says 172.21.15.44:40654) 2020-11-20T12:13:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f8e55d700 1 -- 172.21.15.44:0/236865949 learned_addr learned my addr 172.21.15.44:0/236865949 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f8e55d700 1 -- 172.21.15.44:0/236865949 >> v1:172.21.15.79:6789/0 conn(0x7f2f880a4c30 legacy=0x7f2f8813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f8e55d700 1 -- 172.21.15.44:0/236865949 >> v1:172.21.15.44:6789/0 conn(0x7f2f8813b0a0 legacy=0x7f2f88141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f8e55d700 1 -- 172.21.15.44:0/236865949 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f880a6730 con 0x7f2f88060330 2020-11-20T12:13:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.322+0000 7f2f8e55d700 1 --2- 172.21.15.44:0/236865949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f88060330 0x7f2f880a4740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f2f84006fd0 tx=0x7f2f84006eb0).ready entity=mon.0 client_cookie=87853efe5bc285f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f8d55b700 1 -- 172.21.15.44:0/236865949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84014070 con 0x7f2f88060330 2020-11-20T12:13:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f8d55b700 1 -- 172.21.15.44:0/236865949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f840098d0 con 0x7f2f88060330 2020-11-20T12:13:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f8d55b700 1 -- 172.21.15.44:0/236865949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84009c50 con 0x7f2f88060330 2020-11-20T12:13:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f907c1700 1 -- 172.21.15.44:0/236865949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f88060330 msgr2=0x7f2f880a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f907c1700 1 --2- 172.21.15.44:0/236865949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f88060330 0x7f2f880a4740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f2f84006fd0 tx=0x7f2f84006eb0).stop 2020-11-20T12:13:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f907c1700 1 -- 172.21.15.44:0/236865949 shutdown_connections 2020-11-20T12:13:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f907c1700 1 --2- 172.21.15.44:0/236865949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f88060330 0x7f2f880a4740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f907c1700 1 -- 172.21.15.44:0/236865949 >> 172.21.15.44:0/236865949 conn(0x7f2f88062690 msgr2=0x7f2f880a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f907c1700 1 -- 172.21.15.44:0/236865949 shutdown_connections 2020-11-20T12:13:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.323+0000 7f2f907c1700 1 -- 172.21.15.44:0/236865949 wait complete. 2020-11-20T12:13:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.324+0000 7f2f907c1700 1 Processor -- start 2020-11-20T12:13:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.324+0000 7f2f907c1700 1 -- start start 2020-11-20T12:13:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f907c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f88060330 0x7f2f880aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f907c1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f880a4c30 0x7f2f88136400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f907c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8813b0a0 0x7f2f88149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f907c1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881368f0 con 0x7f2f8813b0a0 2020-11-20T12:13:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f907c1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88146270 con 0x7f2f88060330 2020-11-20T12:13:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f907c1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88146510 con 0x7f2f880a4c30 2020-11-20T12:13:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f8ed5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8813b0a0 0x7f2f88149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f8e55d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f88060330 0x7f2f880aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f8ed5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8813b0a0 0x7f2f88149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40662/0 (socket says 172.21.15.44:40662) 2020-11-20T12:13:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f8e55d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f88060330 0x7f2f880aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:60940/0 (socket says 172.21.15.44:60940) 2020-11-20T12:13:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f8ed5e700 1 -- 172.21.15.44:0/4151409521 learned_addr learned my addr 172.21.15.44:0/4151409521 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.325+0000 7f2f8dd5c700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f880a4c30 0x7f2f88136400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f8ed5e700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f880a4c30 msgr2=0x7f2f88136400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f8ed5e700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f880a4c30 0x7f2f88136400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f8ed5e700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f88060330 msgr2=0x7f2f880aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f8ed5e700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f88060330 0x7f2f880aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f8ed5e700 1 -- 172.21.15.44:0/4151409521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f881467b0 con 0x7f2f8813b0a0 2020-11-20T12:13:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f8ed5e700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8813b0a0 0x7f2f88149320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2f8000ca90 tx=0x7f2f80001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000ce60 con 0x7f2f8813b0a0 2020-11-20T12:13:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8000a730 con 0x7f2f8813b0a0 2020-11-20T12:13:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f88146c50 con 0x7f2f8813b0a0 2020-11-20T12:13:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000aad0 con 0x7f2f8813b0a0 2020-11-20T12:13:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.326+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f881499d0 con 0x7f2f8813b0a0 2020-11-20T12:13:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.327+0000 7f2f907c1700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f88145dd0 0x7f2f8813d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.327+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88146940 con 0x7f2f88145dd0 2020-11-20T12:13:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.327+0000 7f2f8e55d700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f88145dd0 0x7f2f8813d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.327+0000 7f2f8e55d700 1 -- 172.21.15.44:0/4151409521 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f88061650 con 0x7f2f88145dd0 2020-11-20T12:13:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.327+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2f80019030 con 0x7f2f8813b0a0 2020-11-20T12:13:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f8e55d700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f88145dd0 0x7f2f8813d740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2f84006ef0 tx=0x7f2f84016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f60056d20 0x7f2f600591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2f8006f830 con 0x7f2f8813b0a0 2020-11-20T12:13:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f8dd5c700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f60056d20 0x7f2f600591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84021070 con 0x7f2f88145dd0 2020-11-20T12:13:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f2f8401b630 con 0x7f2f88145dd0 2020-11-20T12:13:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f88145dd0 msgr2=0x7f2f8813d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f88145dd0 0x7f2f8813d740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2f84006ef0 tx=0x7f2f84016000).stop 2020-11-20T12:13:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f88145dd0 msgr2=0x7f2f8813d740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f767fc700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f88145dd0 0x7f2f8813d740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.328+0000 7f2f8dd5c700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f60056d20 0x7f2f600591e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f2f880632f0 tx=0x7f2f78000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.336+0000 7f2f907c1700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f8813dc30 0x7f2f88134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.336+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f880446d0 con 0x7f2f8813dc30 2020-11-20T12:13:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.336+0000 7f2f8e55d700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f8813dc30 0x7f2f88134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f8e55d700 1 -- 172.21.15.44:0/4151409521 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f880680f0 con 0x7f2f8813dc30 2020-11-20T12:13:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f8e55d700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f8813dc30 0x7f2f88134ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2f8400bf60 tx=0x7f2f84021000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84019b90 con 0x7f2f8813dc30 2020-11-20T12:13:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f2f8401b420 con 0x7f2f8813dc30 2020-11-20T12:13:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f8813dc30 msgr2=0x7f2f88134ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f767fc700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f8813dc30 0x7f2f88134ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2f8400bf60 tx=0x7f2f84021000).stop 2020-11-20T12:13:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f767fc700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f8813dc30 msgr2=0x7f2f88134ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.337+0000 7f2f767fc700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2f8813dc30 0x7f2f88134ca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f60056d20 msgr2=0x7f2f600591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2f60056d20 0x7f2f600591e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f2f880632f0 tx=0x7f2f78000b80).stop 2020-11-20T12:13:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8813b0a0 msgr2=0x7f2f88149320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 --2- 172.21.15.44:0/4151409521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8813b0a0 0x7f2f88149320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2f8000ca90 tx=0x7f2f80001360).stop 2020-11-20T12:13:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f8ed5e700 1 -- 172.21.15.44:0/4151409521 reap_dead start 2020-11-20T12:13:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f8ed5e700 1 -- 172.21.15.44:0/4151409521 reap_dead start 2020-11-20T12:13:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 shutdown_connections 2020-11-20T12:13:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 >> 172.21.15.44:0/4151409521 conn(0x7f2f88062690 msgr2=0x7f2f880a3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 shutdown_connections 2020-11-20T12:13:20.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.338+0000 7f2f907c1700 1 -- 172.21.15.44:0/4151409521 wait complete. 2020-11-20T12:13:20.349 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:13:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 112, 2020-11-20T12:13:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:13:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 4, 2020-11-20T12:13:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:20.378 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:20.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:20.378 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:20.378 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:20.379 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:20.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:20.379 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:20.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:20.379 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:20.380 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:20.381 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:13:20.381 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:13:20.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.515+0000 7f220cd74700 1 Processor -- start 2020-11-20T12:13:20.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.515+0000 7f220cd74700 1 -- start start 2020-11-20T12:13:20.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.516+0000 7f220cd74700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f22080ae050 0x7f2208060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.516+0000 7f220cd74700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f220813e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.516+0000 7f220cd74700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220813e080 con 0x7f220813e9d0 2020-11-20T12:13:20.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.516+0000 7f220cd74700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2208060b40 con 0x7f22080ae050 2020-11-20T12:13:20.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.516+0000 7f220cd74700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220813ee80 con 0x7f22080669d0 2020-11-20T12:13:20.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.516+0000 7f2206d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f220813e9d0 0x7f2208137b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44422/0 (socket says 172.21.15.44:44422) 2020-11-20T12:13:20.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.516+0000 7f2206d9d700 1 -- 172.21.15.44:0/1574496879 learned_addr learned my addr 172.21.15.44:0/1574496879 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:20.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f2205d9b700 1 --2- 172.21.15.44:0/1574496879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f220813e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f220659c700 1 --2- 172.21.15.44:0/1574496879 >> v2:172.21.15.79:3300/0 conn(0x7f22080ae050 0x7f2208060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f220559a700 1 -- 172.21.15.44:0/1574496879 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863879935 0 0) 0x7f220813e080 con 0x7f220813e9d0 2020-11-20T12:13:20.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f220559a700 1 -- 172.21.15.44:0/1574496879 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21ec003240 con 0x7f220813e9d0 2020-11-20T12:13:20.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f2205d9b700 1 -- 172.21.15.44:0/1574496879 >> v2:172.21.15.79:3300/0 conn(0x7f22080ae050 msgr2=0x7f2208060650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:20.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f2205d9b700 1 --2- 172.21.15.44:0/1574496879 >> v2:172.21.15.79:3300/0 conn(0x7f22080ae050 0x7f2208060650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f2205d9b700 1 -- 172.21.15.44:0/1574496879 >> v1:172.21.15.44:6789/0 conn(0x7f220813e9d0 legacy=0x7f2208137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f2205d9b700 1 -- 172.21.15.44:0/1574496879 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2208060d60 con 0x7f22080669d0 2020-11-20T12:13:20.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f2205d9b700 1 --2- 172.21.15.44:0/1574496879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f220813e4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f21fc00b9a0 tx=0x7f21fc006df0).ready entity=mon.2 client_cookie=dbb8eade23a731b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f220559a700 1 -- 172.21.15.44:0/1574496879 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc009680 con 0x7f22080669d0 2020-11-20T12:13:20.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f220559a700 1 -- 172.21.15.44:0/1574496879 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc0097e0 con 0x7f22080669d0 2020-11-20T12:13:20.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.517+0000 7f220559a700 1 -- 172.21.15.44:0/1574496879 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc009b60 con 0x7f22080669d0 2020-11-20T12:13:20.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 -- 172.21.15.44:0/1574496879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 msgr2=0x7f220813e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 --2- 172.21.15.44:0/1574496879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f220813e4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f21fc00b9a0 tx=0x7f21fc006df0).stop 2020-11-20T12:13:20.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 -- 172.21.15.44:0/1574496879 shutdown_connections 2020-11-20T12:13:20.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 --2- 172.21.15.44:0/1574496879 >> v2:172.21.15.79:3300/0 conn(0x7f22080ae050 0x7f2208060650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 --2- 172.21.15.44:0/1574496879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f220813e4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 -- 172.21.15.44:0/1574496879 >> 172.21.15.44:0/1574496879 conn(0x7f2208062bc0 msgr2=0x7f2208062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:20.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 -- 172.21.15.44:0/1574496879 shutdown_connections 2020-11-20T12:13:20.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.518+0000 7f220cd74700 1 -- 172.21.15.44:0/1574496879 wait complete. 2020-11-20T12:13:20.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.519+0000 7f220cd74700 1 Processor -- start 2020-11-20T12:13:20.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.519+0000 7f220cd74700 1 -- start start 2020-11-20T12:13:20.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220cd74700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f22080ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220cd74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22080ae050 0x7f220813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220cd74700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220813e9d0 0x7f2208136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220cd74700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2208140380 con 0x7f22080ae050 2020-11-20T12:13:20.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220cd74700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2208140620 con 0x7f220813e9d0 2020-11-20T12:13:20.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220cd74700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22081408c0 con 0x7f22080669d0 2020-11-20T12:13:20.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22080ae050 0x7f220813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22080ae050 0x7f220813c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40678/0 (socket says 172.21.15.44:40678) 2020-11-20T12:13:20.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 -- 172.21.15.44:0/2759630916 learned_addr learned my addr 172.21.15.44:0/2759630916 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:20.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220659c700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f22080ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2206d9d700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220813e9d0 0x7f2208136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 -- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 msgr2=0x7f22080ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f22080ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 -- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220813e9d0 msgr2=0x7f2208136a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220813e9d0 0x7f2208136a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 -- 172.21.15.44:0/2759630916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2208140070 con 0x7f22080ae050 2020-11-20T12:13:20.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f2205d9b700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22080ae050 0x7f220813c1c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f21fc010000 tx=0x7f21fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f21f67fc700 1 -- 172.21.15.44:0/2759630916 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc011c40 con 0x7f22080ae050 2020-11-20T12:13:20.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f21f67fc700 1 -- 172.21.15.44:0/2759630916 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc011da0 con 0x7f22080ae050 2020-11-20T12:13:20.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22080abe00 con 0x7f22080ae050 2020-11-20T12:13:20.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.520+0000 7f21f67fc700 1 -- 172.21.15.44:0/2759630916 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc01e670 con 0x7f22080ae050 2020-11-20T12:13:20.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.521+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22081370e0 con 0x7f22080ae050 2020-11-20T12:13:20.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.522+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22080608c0 con 0x7f22080ae050 2020-11-20T12:13:20.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.522+0000 7f21f67fc700 1 -- 172.21.15.44:0/2759630916 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f21fc025030 con 0x7f22080ae050 2020-11-20T12:13:20.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.522+0000 7f21f67fc700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21dc056d20 0x7f21dc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:20.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.522+0000 7f21f67fc700 1 -- 172.21.15.44:0/2759630916 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f21fc0716a0 con 0x7f22080ae050 2020-11-20T12:13:20.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.522+0000 7f220659c700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21dc056d20 0x7f21dc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:20.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.523+0000 7f220659c700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21dc056d20 0x7f21dc0591e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f21f8006fd0 tx=0x7f21f8006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:20.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.528+0000 7f21f67fc700 1 -- 172.21.15.44:0/2759630916 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f21fc046aa0 con 0x7f22080ae050 2020-11-20T12:13:20.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.796+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f22080a50c0 con 0x7f22080ae050 2020-11-20T12:13:20.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.797+0000 7f21f67fc700 1 -- 172.21.15.44:0/2759630916 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f21fc023070 con 0x7f22080ae050 2020-11-20T12:13:20.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21dc056d20 msgr2=0x7f21dc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21dc056d20 0x7f21dc0591e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f21f8006fd0 tx=0x7f21f8006df0).stop 2020-11-20T12:13:20.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22080ae050 msgr2=0x7f220813c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:20.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22080ae050 0x7f220813c1c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f21fc010000 tx=0x7f21fc015000).stop 2020-11-20T12:13:20.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 shutdown_connections 2020-11-20T12:13:20.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f21dc056d20 0x7f21dc0591e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220813e9d0 0x7f2208136a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22080669d0 0x7f22080ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 --2- 172.21.15.44:0/2759630916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22080ae050 0x7f220813c1c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 >> 172.21.15.44:0/2759630916 conn(0x7f2208062bc0 msgr2=0x7f2208134090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 shutdown_connections 2020-11-20T12:13:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:20.799+0000 7f220cd74700 1 -- 172.21.15.44:0/2759630916 wait complete. 2020-11-20T12:13:20.815 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:13:30.817 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:13:30.817 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:13:30.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.955+0000 7fdd8f02f700 1 Processor -- start 2020-11-20T12:13:30.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.955+0000 7fdd8f02f700 1 -- start start 2020-11-20T12:13:30.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8f02f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd880ad440 0x7fdd880a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:30.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8f02f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a9dd0 0x7fdd88142350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:30.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8f02f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880a70f0 con 0x7fdd880a4b00 2020-11-20T12:13:30.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8f02f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880a5880 con 0x7fdd880ad440 2020-11-20T12:13:30.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8f02f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880a4170 con 0x7fdd880a9dd0 2020-11-20T12:13:30.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8d82c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fdd880a4b00 0x7fdd880a96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44432/0 (socket says 172.21.15.44:44432) 2020-11-20T12:13:30.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8d82c700 1 -- 172.21.15.44:0/2639612536 learned_addr learned my addr 172.21.15.44:0/2639612536 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:30.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8e82e700 1 --2- 172.21.15.44:0/2639612536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a9dd0 0x7fdd88142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:30.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.956+0000 7fdd8e02d700 1 --2- 172.21.15.44:0/2639612536 >> v2:172.21.15.79:3300/0 conn(0x7fdd880ad440 0x7fdd880a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8d02b700 1 -- 172.21.15.44:0/2639612536 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95245843 0 0) 0x7fdd880a70f0 con 0x7fdd880a4b00 2020-11-20T12:13:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8d02b700 1 -- 172.21.15.44:0/2639612536 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd70003240 con 0x7fdd880a4b00 2020-11-20T12:13:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8e82e700 1 -- 172.21.15.44:0/2639612536 >> v2:172.21.15.79:3300/0 conn(0x7fdd880ad440 msgr2=0x7fdd880a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8e82e700 1 --2- 172.21.15.44:0/2639612536 >> v2:172.21.15.79:3300/0 conn(0x7fdd880ad440 0x7fdd880a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:30.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8e82e700 1 -- 172.21.15.44:0/2639612536 >> v1:172.21.15.44:6789/0 conn(0x7fdd880a4b00 legacy=0x7fdd880a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8e82e700 1 -- 172.21.15.44:0/2639612536 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880ad820 con 0x7fdd880a9dd0 2020-11-20T12:13:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8e82e700 1 --2- 172.21.15.44:0/2639612536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a9dd0 0x7fdd88142350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fdd80009c90 tx=0x7fdd800091d0).ready entity=mon.2 client_cookie=50d0cdecb5bce55b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8d02b700 1 -- 172.21.15.44:0/2639612536 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8000ba50 con 0x7fdd880a9dd0 2020-11-20T12:13:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8d02b700 1 -- 172.21.15.44:0/2639612536 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8000bbb0 con 0x7fdd880a9dd0 2020-11-20T12:13:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.957+0000 7fdd8d02b700 1 -- 172.21.15.44:0/2639612536 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80011400 con 0x7fdd880a9dd0 2020-11-20T12:13:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 -- 172.21.15.44:0/2639612536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a9dd0 msgr2=0x7fdd88142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/2639612536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a9dd0 0x7fdd88142350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fdd80009c90 tx=0x7fdd800091d0).stop 2020-11-20T12:13:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 -- 172.21.15.44:0/2639612536 shutdown_connections 2020-11-20T12:13:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/2639612536 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a9dd0 0x7fdd88142350 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/2639612536 >> v2:172.21.15.79:3300/0 conn(0x7fdd880ad440 0x7fdd880a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 -- 172.21.15.44:0/2639612536 >> 172.21.15.44:0/2639612536 conn(0x7fdd88061b60 msgr2=0x7fdd88061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 -- 172.21.15.44:0/2639612536 shutdown_connections 2020-11-20T12:13:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.958+0000 7fdd8f02f700 1 -- 172.21.15.44:0/2639612536 wait complete. 2020-11-20T12:13:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 Processor -- start 2020-11-20T12:13:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 -- start start 2020-11-20T12:13:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a4b00 0x7fdd881341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd880a9dd0 0x7fdd88132980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd880ad440 0x7fdd88146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:30.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88145910 con 0x7fdd880a9dd0 2020-11-20T12:13:30.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880aad40 con 0x7fdd880ad440 2020-11-20T12:13:30.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.959+0000 7fdd8f02f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880aafe0 con 0x7fdd880a4b00 2020-11-20T12:13:30.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd880a9dd0 0x7fdd88132980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:30.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd880a9dd0 0x7fdd88132980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40690/0 (socket says 172.21.15.44:40690) 2020-11-20T12:13:30.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 -- 172.21.15.44:0/1003976050 learned_addr learned my addr 172.21.15.44:0/1003976050 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:30.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8e02d700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a4b00 0x7fdd881341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:30.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8e82e700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd880ad440 0x7fdd88146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:30.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 -- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a4b00 msgr2=0x7fdd881341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:30.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a4b00 0x7fdd881341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:30.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 -- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd880ad440 msgr2=0x7fdd88146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:30.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd880ad440 0x7fdd88146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:30.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 -- 172.21.15.44:0/1003976050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880ab280 con 0x7fdd880a9dd0 2020-11-20T12:13:30.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8d82c700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd880a9dd0 0x7fdd88132980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fdd7800c6a0 tx=0x7fdd78007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd767fc700 1 -- 172.21.15.44:0/1003976050 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd780072e0 con 0x7fdd880a9dd0 2020-11-20T12:13:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd767fc700 1 -- 172.21.15.44:0/1003976050 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd78007440 con 0x7fdd880a9dd0 2020-11-20T12:13:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd880ab520 con 0x7fdd880a9dd0 2020-11-20T12:13:30.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd767fc700 1 -- 172.21.15.44:0/1003976050 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78005820 con 0x7fdd880a9dd0 2020-11-20T12:13:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.960+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd88132e70 con 0x7fdd880a9dd0 2020-11-20T12:13:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.961+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd88136b50 con 0x7fdd880a9dd0 2020-11-20T12:13:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.962+0000 7fdd767fc700 1 -- 172.21.15.44:0/1003976050 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fdd78010070 con 0x7fdd880a9dd0 2020-11-20T12:13:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.962+0000 7fdd767fc700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdd60056d20 0x7fdd600591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.962+0000 7fdd767fc700 1 -- 172.21.15.44:0/1003976050 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fdd7806f6a0 con 0x7fdd880a9dd0 2020-11-20T12:13:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.962+0000 7fdd8e02d700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdd60056d20 0x7fdd600591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.962+0000 7fdd8e02d700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdd60056d20 0x7fdd600591e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fdd84006fd0 tx=0x7fdd84006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:30.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:30.966+0000 7fdd767fc700 1 -- 172.21.15.44:0/1003976050 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fdd88136b50 con 0x7fdd880a9dd0 2020-11-20T12:13:31.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.245+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdd88136b50 con 0x7fdd880a9dd0 2020-11-20T12:13:31.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.245+0000 7fdd767fc700 1 -- 172.21.15.44:0/1003976050 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fdd88136b50 con 0x7fdd880a9dd0 2020-11-20T12:13:31.249 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:13:31.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdd60056d20 msgr2=0x7fdd600591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdd60056d20 0x7fdd600591e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fdd84006fd0 tx=0x7fdd84006df0).stop 2020-11-20T12:13:31.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd880a9dd0 msgr2=0x7fdd88132980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd880a9dd0 0x7fdd88132980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fdd7800c6a0 tx=0x7fdd78007ac0).stop 2020-11-20T12:13:31.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 shutdown_connections 2020-11-20T12:13:31.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdd60056d20 0x7fdd600591e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd880ad440 0x7fdd88146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdd880a4b00 0x7fdd881341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 --2- 172.21.15.44:0/1003976050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd880a9dd0 0x7fdd88132980 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 >> 172.21.15.44:0/1003976050 conn(0x7fdd88061b60 msgr2=0x7fdd88061f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:31.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 shutdown_connections 2020-11-20T12:13:31.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.247+0000 7fdd8f02f700 1 -- 172.21.15.44:0/1003976050 wait complete. 2020-11-20T12:13:31.262 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:13:31.263 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:13:31.263 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:13:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.424+0000 7f267959e700 1 Processor -- start 2020-11-20T12:13:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.424+0000 7f267959e700 1 -- start start 2020-11-20T12:13:31.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f267959e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a9880 0x7f26740a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f267959e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26740a8af0 0x7f26740a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f267959e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26740a8f80 con 0x7f26740a6600 2020-11-20T12:13:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f267959e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26740a71b0 con 0x7f26740a8af0 2020-11-20T12:13:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f267959e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26740a6b80 con 0x7f26740a9880 2020-11-20T12:13:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f266bfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f26740a6600 0x7f26740a7630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44446/0 (socket says 172.21.15.44:44446) 2020-11-20T12:13:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f266bfff700 1 -- 172.21.15.44:0/3642250694 learned_addr learned my addr 172.21.15.44:0/3642250694 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:31.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f2673fff700 1 --2- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a9880 0x7f26740a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f2678d9d700 1 --2- 172.21.15.44:0/3642250694 >> v2:172.21.15.79:3300/0 conn(0x7f26740a8af0 0x7f26740a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.425+0000 7f26737fe700 1 -- 172.21.15.44:0/3642250694 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383061459 0 0) 0x7f26740a8f80 con 0x7f26740a6600 2020-11-20T12:13:31.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f26737fe700 1 -- 172.21.15.44:0/3642250694 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2654003240 con 0x7f26740a6600 2020-11-20T12:13:31.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f2678d9d700 1 -- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a9880 msgr2=0x7f26740a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:31.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f2678d9d700 1 --2- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a9880 0x7f26740a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f2678d9d700 1 -- 172.21.15.44:0/3642250694 >> v1:172.21.15.44:6789/0 conn(0x7f26740a6600 legacy=0x7f26740a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f2678d9d700 1 -- 172.21.15.44:0/3642250694 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26740a6e20 con 0x7f26740a8af0 2020-11-20T12:13:31.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f2673fff700 1 --2- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a9880 0x7f26740a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:13:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f2678d9d700 1 --2- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 0x7f26740a7d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f266c009c90 tx=0x7f266c0091d0).ready entity=mon.1 client_cookie=a9cf0c2f89188f9a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f26737fe700 1 -- 172.21.15.44:0/3642250694 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c00ba50 con 0x7f26740a8af0 2020-11-20T12:13:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f26737fe700 1 -- 172.21.15.44:0/3642250694 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c00bbb0 con 0x7f26740a8af0 2020-11-20T12:13:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.426+0000 7f26737fe700 1 -- 172.21.15.44:0/3642250694 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c011400 con 0x7f26740a8af0 2020-11-20T12:13:31.434 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 -- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 msgr2=0x7f26740a7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 --2- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 0x7f26740a7d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f266c009c90 tx=0x7f266c0091d0).stop 2020-11-20T12:13:31.435 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 -- 172.21.15.44:0/3642250694 shutdown_connections 2020-11-20T12:13:31.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 --2- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 0x7f26740a7d40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 --2- 172.21.15.44:0/3642250694 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a9880 0x7f26740a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 -- 172.21.15.44:0/3642250694 >> 172.21.15.44:0/3642250694 conn(0x7f2674061b60 msgr2=0x7f2674061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:31.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 -- 172.21.15.44:0/3642250694 shutdown_connections 2020-11-20T12:13:31.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.427+0000 7f267959e700 1 -- 172.21.15.44:0/3642250694 wait complete. 2020-11-20T12:13:31.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 Processor -- start 2020-11-20T12:13:31.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 -- start start 2020-11-20T12:13:31.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a6600 0x7f267413a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 0x7f2674065360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a9880 0x7f2674131e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267414ce80 con 0x7f26740a9880 2020-11-20T12:13:31.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2674064460 con 0x7f26740a8af0 2020-11-20T12:13:31.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.428+0000 7f267959e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2674064700 con 0x7f26740a6600 2020-11-20T12:13:31.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a9880 0x7f2674131e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a9880 0x7f2674131e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40706/0 (socket says 172.21.15.44:40706) 2020-11-20T12:13:31.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 -- 172.21.15.44:0/1762812515 learned_addr learned my addr 172.21.15.44:0/1762812515 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:31.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2673fff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a6600 0x7f267413a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f266bfff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 0x7f2674065360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a6600 msgr2=0x7f267413a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a6600 0x7f267413a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 msgr2=0x7f2674065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26740a8af0 0x7f2674065360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 -- 172.21.15.44:0/1762812515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26740649a0 con 0x7f26740a9880 2020-11-20T12:13:31.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2673fff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26740a6600 0x7f267413a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:13:31.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2678d9d700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a9880 0x7f2674131e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f266c009210 tx=0x7f266c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c010780 con 0x7f26740a9880 2020-11-20T12:13:31.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c0108e0 con 0x7f26740a9880 2020-11-20T12:13:31.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.429+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2674067450 con 0x7f26740a9880 2020-11-20T12:13:31.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.430+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c010c80 con 0x7f26740a9880 2020-11-20T12:13:31.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.430+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26740a31e0 con 0x7f26740a9880 2020-11-20T12:13:31.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f267959e700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2674065850 0x7f2674132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26740a3490 con 0x7f2674065850 2020-11-20T12:13:31.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f2673fff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2674065850 0x7f2674132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f266c017070 con 0x7f26740a9880 2020-11-20T12:13:31.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f2673fff700 1 -- 172.21.15.44:0/1762812515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2674064b30 con 0x7f2674065850 2020-11-20T12:13:31.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f2673fff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2674065850 0x7f2674132300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f2664010e50 tx=0x7f2664008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f2670ff9700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2650056d20 0x7f26500591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f266c072450 con 0x7f26740a9880 2020-11-20T12:13:31.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f266bfff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2650056d20 0x7f26500591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.431+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266400ecc0 con 0x7f2674065850 2020-11-20T12:13:31.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.432+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f266400ee20 con 0x7f2674065850 2020-11-20T12:13:31.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.432+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2674065850 msgr2=0x7f2674132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.432+0000 7f2670ff9700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2674065850 0x7f2674132300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f2664010e50 tx=0x7f2664008040).stop 2020-11-20T12:13:31.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.432+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2674065850 msgr2=0x7f2674132300 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:31.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.432+0000 7f2670ff9700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2674065850 0x7f2674132300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.432+0000 7f266bfff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2650056d20 0x7f26500591e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f26740a2f40 tx=0x7f265c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.440+0000 7f267959e700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a1880 0x7f26740a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.440+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2674061180 con 0x7f26740a1880 2020-11-20T12:13:31.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2673fff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a1880 0x7f26740a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2673fff700 1 -- 172.21.15.44:0/1762812515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2674057d80 con 0x7f26740a1880 2020-11-20T12:13:31.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2673fff700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a1880 0x7f26740a1c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f2664010e50 tx=0x7f266401c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266401e880 con 0x7f26740a1880 2020-11-20T12:13:31.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4779 (secure 0 0 0) 0x7f266400eab0 con 0x7f26740a1880 2020-11-20T12:13:31.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a1880 msgr2=0x7f26740a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2670ff9700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a1880 0x7f26740a1c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f2664010e50 tx=0x7f266401c000).stop 2020-11-20T12:13:31.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2670ff9700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a1880 msgr2=0x7f26740a1c60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:31.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.441+0000 7f2670ff9700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a1880 0x7f26740a1c60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2650056d20 msgr2=0x7f26500591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2650056d20 0x7f26500591e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f26740a2f40 tx=0x7f265c009040).stop 2020-11-20T12:13:31.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a9880 msgr2=0x7f2674131e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 --2- 172.21.15.44:0/1762812515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26740a9880 0x7f2674131e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f266c009210 tx=0x7f266c016000).stop 2020-11-20T12:13:31.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f2678d9d700 1 -- 172.21.15.44:0/1762812515 reap_dead start 2020-11-20T12:13:31.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f2678d9d700 1 -- 172.21.15.44:0/1762812515 reap_dead start 2020-11-20T12:13:31.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 shutdown_connections 2020-11-20T12:13:31.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 >> 172.21.15.44:0/1762812515 conn(0x7f2674061b60 msgr2=0x7f26741377a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:31.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 shutdown_connections 2020-11-20T12:13:31.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.442+0000 7f267959e700 1 -- 172.21.15.44:0/1762812515 wait complete. 2020-11-20T12:13:31.457 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:31.457 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:31.457 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:31.457 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:13:31.457 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 112, 2020-11-20T12:13:31.458 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:31.458 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:13:31.458 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:31.458 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:31.459 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.459 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:13:31.459 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:31.459 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:31.459 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:31.460 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.460 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.460 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.461 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.461 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.461 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.461 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.461 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.462 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.462 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:31.462 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:31.462 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.462 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.463 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.463 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.463 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.463 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.463 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.464 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.464 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.464 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:31.464 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:31.465 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:31.465 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:31.465 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:31.465 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:31.466 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:31.466 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:31.466 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:31.467 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:31.467 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:31.468 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:31.468 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:31.468 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:31.469 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:31.469 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.470 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.470 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.470 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.471 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.471 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.471 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.472 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.472 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.473 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:31.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.473 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:31.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.474 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:31.474 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:31.475 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.475 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.476 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.476 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:31.477 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.478 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:31.479 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.479 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.479 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.480 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:31.480 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:31.481 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.481 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.482 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.483 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:31.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:31.484 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.484 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.485 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.485 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:31.485 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.487 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:31.487 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.488 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.489 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:31.489 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:31.490 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.490 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.491 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.492 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:31.492 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:31.492 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.493 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.494 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:31.494 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.496 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:31.496 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.497 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.498 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:31.498 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:31.499 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.499 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.499 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.500 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.501 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:31.501 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:31.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.502 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.503 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.503 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:31.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.504 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.504 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:13:31.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.505 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.505 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.506 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:31.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.506 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.507 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:13:31.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.508 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.508 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.508 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:13:31.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.509 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:31.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.510 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.510 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.511 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:13:31.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.512 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.512 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:13:31.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.513 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.513 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.514 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:13:31.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.515 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:31.516 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:31.517 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:13:31.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.664+0000 7f9d62bc7700 1 Processor -- start 2020-11-20T12:13:31.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d62bc7700 1 -- start start 2020-11-20T12:13:31.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d62bc7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9d5c0a3e10 0x7f9d5c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d62bc7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5c0a9770 con 0x7f9d5c0aa020 2020-11-20T12:13:31.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d62bc7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5c0a6da0 con 0x7f9d5c0aa400 2020-11-20T12:13:31.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d62bc7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0a56c0 con 0x7f9d5c0a3e10 2020-11-20T12:13:31.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d60963700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9d5c0aa020 0x7f9d5c0a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44462/0 (socket says 172.21.15.44:44462) 2020-11-20T12:13:31.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d60963700 1 -- 172.21.15.44:0/2080131699 learned_addr learned my addr 172.21.15.44:0/2080131699 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:31.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.665+0000 7f9d61164700 1 --2- 172.21.15.44:0/2080131699 >> v2:172.21.15.79:3300/0 conn(0x7f9d5c0a3e10 0x7f9d5c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/2080131699 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679391324 0 0) 0x7f9d500010d0 con 0x7f9d5c0aa020 2020-11-20T12:13:31.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/2080131699 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d40003240 con 0x7f9d5c0aa020 2020-11-20T12:13:31.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/2080131699 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617807998 0 0) 0x7f9d5c0a6da0 con 0x7f9d5c0aa400 2020-11-20T12:13:31.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/2080131699 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d500010d0 con 0x7f9d5c0aa400 2020-11-20T12:13:31.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d61164700 1 -- 172.21.15.44:0/2080131699 >> v1:172.21.15.79:6789/0 conn(0x7f9d5c0aa400 legacy=0x7f9d5c0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d61164700 1 -- 172.21.15.44:0/2080131699 >> v1:172.21.15.44:6789/0 conn(0x7f9d5c0aa020 legacy=0x7f9d5c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d61164700 1 -- 172.21.15.44:0/2080131699 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0a7040 con 0x7f9d5c0a3e10 2020-11-20T12:13:31.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d61164700 1 --2- 172.21.15.44:0/2080131699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 0x7f9d5c0a9280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f9d54009c90 tx=0x7f9d540091d0).ready entity=mon.1 client_cookie=4e9dcccf796cb9b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/2080131699 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400b9c0 con 0x7f9d5c0a3e10 2020-11-20T12:13:31.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/2080131699 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5400bb20 con 0x7f9d5c0a3e10 2020-11-20T12:13:31.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.666+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/2080131699 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400bea0 con 0x7f9d5c0a3e10 2020-11-20T12:13:31.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.667+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2080131699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 msgr2=0x7f9d5c0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.667+0000 7f9d62bc7700 1 --2- 172.21.15.44:0/2080131699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 0x7f9d5c0a9280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f9d54009c90 tx=0x7f9d540091d0).stop 2020-11-20T12:13:31.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.667+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2080131699 shutdown_connections 2020-11-20T12:13:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.667+0000 7f9d62bc7700 1 --2- 172.21.15.44:0/2080131699 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 0x7f9d5c0a9280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.667+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2080131699 >> 172.21.15.44:0/2080131699 conn(0x7f9d5c062690 msgr2=0x7f9d5c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:31.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.667+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2080131699 shutdown_connections 2020-11-20T12:13:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.667+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2080131699 wait complete. 2020-11-20T12:13:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.668+0000 7f9d62bc7700 1 Processor -- start 2020-11-20T12:13:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.668+0000 7f9d62bc7700 1 -- start start 2020-11-20T12:13:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d62bc7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 0x7f9d5c134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d62bc7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d5c0aa020 0x7f9d5c0677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d62bc7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d5c0aa400 0x7f9d5c066be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d62bc7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c065730 con 0x7f9d5c0aa020 2020-11-20T12:13:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d62bc7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c14ad20 con 0x7f9d5c0a3e10 2020-11-20T12:13:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d62bc7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c14a3c0 con 0x7f9d5c0aa400 2020-11-20T12:13:31.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d5c0aa020 0x7f9d5c0677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d5c0aa020 0x7f9d5c0677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40722/0 (socket says 172.21.15.44:40722) 2020-11-20T12:13:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 -- 172.21.15.44:0/2260274570 learned_addr learned my addr 172.21.15.44:0/2260274570 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d61164700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d5c0aa400 0x7f9d5c066be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d60963700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 0x7f9d5c134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d5c0aa400 msgr2=0x7f9d5c066be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d5c0aa400 0x7f9d5c066be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 msgr2=0x7f9d5c134690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 0x7f9d5c134690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 -- 172.21.15.44:0/2260274570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c14a880 con 0x7f9d5c0aa020 2020-11-20T12:13:31.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d61164700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d5c0aa400 0x7f9d5c066be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:13:31.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d60963700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0a3e10 0x7f9d5c134690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:13:31.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d5c0aa020 0x7f9d5c0677f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f9d4800c710 tx=0x7f9d48007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800e560 con 0x7f9d5c0aa020 2020-11-20T12:13:31.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4800e6c0 con 0x7f9d5c0aa020 2020-11-20T12:13:31.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c060400 con 0x7f9d5c0aa020 2020-11-20T12:13:31.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.669+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800ea60 con 0x7f9d5c0aa020 2020-11-20T12:13:31.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.670+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c067ce0 con 0x7f9d5c0aa020 2020-11-20T12:13:31.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.670+0000 7f9d62bc7700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0666c0 0x7f9d5c0670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.670+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c067fc0 con 0x7f9d5c0666c0 2020-11-20T12:13:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.671+0000 7f9d60963700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0666c0 0x7f9d5c0670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.671+0000 7f9d60963700 1 -- 172.21.15.44:0/2260274570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d5c14a6c0 con 0x7f9d5c0666c0 2020-11-20T12:13:31.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.671+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9d48019030 con 0x7f9d5c0aa020 2020-11-20T12:13:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.671+0000 7f9d60963700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0666c0 0x7f9d5c0670d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f9d5000d9d0 tx=0x7f9d50009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.671+0000 7f9d58ff9700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d34056d50 0x7f9d34059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.671+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9d4806f600 con 0x7f9d5c0aa020 2020-11-20T12:13:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d61164700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d34056d50 0x7f9d34059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50007320 con 0x7f9d5c0666c0 2020-11-20T12:13:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f9d50007480 con 0x7f9d5c0666c0 2020-11-20T12:13:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0666c0 msgr2=0x7f9d5c0670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d58ff9700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0666c0 0x7f9d5c0670d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f9d5000d9d0 tx=0x7f9d50009040).stop 2020-11-20T12:13:31.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0666c0 msgr2=0x7f9d5c0670d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:31.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d58ff9700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c0666c0 0x7f9d5c0670d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.672+0000 7f9d61164700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d34056d50 0x7f9d34059210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f9d54009210 tx=0x7f9d54016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.680+0000 7f9d62bc7700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c149dc0 0x7f9d5c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.680+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c14a1d0 con 0x7f9d5c149dc0 2020-11-20T12:13:31.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.681+0000 7f9d60963700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c149dc0 0x7f9d5c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.681+0000 7f9d60963700 1 -- 172.21.15.44:0/2260274570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9d5c14aa10 con 0x7f9d5c149dc0 2020-11-20T12:13:31.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.681+0000 7f9d60963700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c149dc0 0x7f9d5c14d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f9d50014000 tx=0x7f9d50009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.681+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50007890 con 0x7f9d5c149dc0 2020-11-20T12:13:31.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7f9d5000be80 con 0x7f9d5c149dc0 2020-11-20T12:13:31.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c149dc0 msgr2=0x7f9d5c14d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d58ff9700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c149dc0 0x7f9d5c14d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f9d50014000 tx=0x7f9d50009040).stop 2020-11-20T12:13:31.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d58ff9700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c149dc0 msgr2=0x7f9d5c14d390 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:31.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d58ff9700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d5c149dc0 0x7f9d5c14d390 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0x7f9d50014000 tx=0x7f9d50009040).stop 2020-11-20T12:13:31.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d34056d50 msgr2=0x7f9d34059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d34056d50 0x7f9d34059210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f9d54009210 tx=0x7f9d54016000).stop 2020-11-20T12:13:31.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d5c0aa020 msgr2=0x7f9d5c0677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 --2- 172.21.15.44:0/2260274570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d5c0aa020 0x7f9d5c0677f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f9d4800c710 tx=0x7f9d48007ae0).stop 2020-11-20T12:13:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d61164700 1 -- 172.21.15.44:0/2260274570 reap_dead start 2020-11-20T12:13:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d61164700 1 -- 172.21.15.44:0/2260274570 reap_dead start 2020-11-20T12:13:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 shutdown_connections 2020-11-20T12:13:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 >> 172.21.15.44:0/2260274570 conn(0x7f9d5c062690 msgr2=0x7f9d5c0a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:31.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 shutdown_connections 2020-11-20T12:13:31.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.682+0000 7f9d62bc7700 1 -- 172.21.15.44:0/2260274570 wait complete. 2020-11-20T12:13:31.703 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:31.704 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:31.704 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:13:31.704 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 112, 2020-11-20T12:13:31.704 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:31.704 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:13:31.705 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:31.705 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:31.705 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.705 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:13:31.706 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:31.706 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:31.706 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:31.706 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.706 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.707 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.707 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.707 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.708 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.708 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.708 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.708 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:31.708 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:31.709 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.709 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.709 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.710 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.710 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.710 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.710 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.711 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.711 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:31.711 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:31.711 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:31.712 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:31.712 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:31.712 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:31.712 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:31.712 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:31.713 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:31.713 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:31.713 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:31.713 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:31.714 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:31.714 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:31.714 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:31.714 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.714 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.715 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.715 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.715 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.716 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.716 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.716 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.716 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:31.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:31.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.717 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:31.717 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:31.718 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.718 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.718 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:31.719 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.719 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:31.720 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.720 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.720 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.720 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:31.721 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:31.721 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.721 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.721 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.722 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.724 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:31.725 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.729 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:31.730 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.732 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:31.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:31.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.732 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.732 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.733 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:31.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.733 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:13:31.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.734 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.734 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.734 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:31.734 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.734 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.735 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:13:31.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.735 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.735 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.735 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:13:31.736 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.736 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.736 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:31.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.736 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.737 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.737 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:31.737 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.737 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.738 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:31.738 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:31.739 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:13:31.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.876+0000 7f9d50679700 1 Processor -- start 2020-11-20T12:13:31.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.876+0000 7f9d50679700 1 -- start start 2020-11-20T12:13:31.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d50679700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813a990 0x7f9d4813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d50679700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9d4813f800 0x7f9d48141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d50679700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d480a6260 con 0x7f9d48060330 2020-11-20T12:13:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d50679700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480a52e0 con 0x7f9d4813f800 2020-11-20T12:13:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d50679700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480a6490 con 0x7f9d4813a990 2020-11-20T12:13:31.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4ec16700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9d4813f800 0x7f9d48141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4e415700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9d48060330 0x7f9d480a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47356/0 (socket says 172.21.15.44:47356) 2020-11-20T12:13:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4dc14700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813a990 0x7f9d4813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4e415700 1 -- 172.21.15.44:0/530044921 learned_addr learned my addr 172.21.15.44:0/530044921 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4d413700 1 -- 172.21.15.44:0/530044921 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037299699 0 0) 0x7f9d480a6260 con 0x7f9d48060330 2020-11-20T12:13:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4d413700 1 -- 172.21.15.44:0/530044921 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d30003240 con 0x7f9d48060330 2020-11-20T12:13:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4ec16700 1 -- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813a990 msgr2=0x7f9d4813adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4ec16700 1 --2- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813a990 0x7f9d4813adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4ec16700 1 -- 172.21.15.44:0/530044921 >> v1:172.21.15.79:6789/0 conn(0x7f9d48060330 legacy=0x7f9d480a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4ec16700 1 -- 172.21.15.44:0/530044921 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480a6730 con 0x7f9d4813f800 2020-11-20T12:13:31.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.877+0000 7f9d4dc14700 1 --2- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813a990 0x7f9d4813adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:13:31.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d4ec16700 1 --2- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813f800 0x7f9d48141bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9d40009c90 tx=0x7f9d400091d0).ready entity=mon.1 client_cookie=18593116ade75047 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d4d413700 1 -- 172.21.15.44:0/530044921 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000b9c0 con 0x7f9d4813f800 2020-11-20T12:13:31.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d4d413700 1 -- 172.21.15.44:0/530044921 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000bb20 con 0x7f9d4813f800 2020-11-20T12:13:31.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d4d413700 1 -- 172.21.15.44:0/530044921 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000bea0 con 0x7f9d4813f800 2020-11-20T12:13:31.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 -- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813f800 msgr2=0x7f9d48141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 --2- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813f800 0x7f9d48141bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9d40009c90 tx=0x7f9d400091d0).stop 2020-11-20T12:13:31.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 -- 172.21.15.44:0/530044921 shutdown_connections 2020-11-20T12:13:31.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 --2- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813f800 0x7f9d48141bf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 --2- 172.21.15.44:0/530044921 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813a990 0x7f9d4813adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 -- 172.21.15.44:0/530044921 >> 172.21.15.44:0/530044921 conn(0x7f9d48062690 msgr2=0x7f9d480a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:31.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 -- 172.21.15.44:0/530044921 shutdown_connections 2020-11-20T12:13:31.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.878+0000 7f9d50679700 1 -- 172.21.15.44:0/530044921 wait complete. 2020-11-20T12:13:31.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.879+0000 7f9d50679700 1 Processor -- start 2020-11-20T12:13:31.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.879+0000 7f9d50679700 1 -- start start 2020-11-20T12:13:31.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d50679700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d48060330 0x7f9d48065510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d50679700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813a990 0x7f9d48151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d50679700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813f800 0x7f9d4812a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d50679700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48152420 con 0x7f9d48060330 2020-11-20T12:13:31.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d50679700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4812a130 con 0x7f9d4813a990 2020-11-20T12:13:31.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d50679700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4812a3d0 con 0x7f9d4813f800 2020-11-20T12:13:31.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d4e415700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d48060330 0x7f9d48065510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d4e415700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d48060330 0x7f9d48065510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40738/0 (socket says 172.21.15.44:40738) 2020-11-20T12:13:31.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d4e415700 1 -- 172.21.15.44:0/1619187020 learned_addr learned my addr 172.21.15.44:0/1619187020 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:31.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d4dc14700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813a990 0x7f9d48151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.880+0000 7f9d4ec16700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813f800 0x7f9d4812a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d4e415700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813f800 msgr2=0x7f9d4812a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d4e415700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d4813f800 0x7f9d4812a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d4e415700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813a990 msgr2=0x7f9d48151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d4e415700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4813a990 0x7f9d48151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d4e415700 1 -- 172.21.15.44:0/1619187020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4813f190 con 0x7f9d48060330 2020-11-20T12:13:31.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d4e415700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d48060330 0x7f9d48065510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9d4400c6f0 tx=0x7f9d44007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d440072e0 con 0x7f9d48060330 2020-11-20T12:13:31.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d44007440 con 0x7f9d48060330 2020-11-20T12:13:31.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d48151f90 con 0x7f9d48060330 2020-11-20T12:13:31.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44005800 con 0x7f9d48060330 2020-11-20T12:13:31.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.881+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4812afa0 con 0x7f9d48060330 2020-11-20T12:13:31.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.882+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9d44010070 con 0x7f9d48060330 2020-11-20T12:13:31.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d50679700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48137380 0x7f9d48137760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48152290 con 0x7f9d48137380 2020-11-20T12:13:31.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d367fc700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d20056d20 0x7f9d200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9d4406f920 con 0x7f9d48060330 2020-11-20T12:13:31.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d4dc14700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48137380 0x7f9d48137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d4ec16700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d20056d20 0x7f9d200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d4dc14700 1 -- 172.21.15.44:0/1619187020 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d48061650 con 0x7f9d48137380 2020-11-20T12:13:31.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d4ec16700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d20056d20 0x7f9d200591e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9d40009440 tx=0x7f9d40016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.883+0000 7f9d4dc14700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48137380 0x7f9d48137760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f9d48141e90 tx=0x7f9d38006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.884+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38018070 con 0x7f9d48137380 2020-11-20T12:13:31.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.884+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f9d38014440 con 0x7f9d48137380 2020-11-20T12:13:31.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.884+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48137380 msgr2=0x7f9d48137760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.884+0000 7f9d367fc700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48137380 0x7f9d48137760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f9d48141e90 tx=0x7f9d38006df0).stop 2020-11-20T12:13:31.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.884+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48137380 msgr2=0x7f9d48137760 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:31.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.884+0000 7f9d367fc700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48137380 0x7f9d48137760 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0x7f9d48141e90 tx=0x7f9d38006df0).stop 2020-11-20T12:13:31.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.892+0000 7f9d50679700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48066100 0x7f9d48066510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:31.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.892+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48129a60 con 0x7f9d48066100 2020-11-20T12:13:31.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.893+0000 7f9d4dc14700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48066100 0x7f9d48066510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:31.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.893+0000 7f9d4dc14700 1 -- 172.21.15.44:0/1619187020 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9d480446d0 con 0x7f9d48066100 2020-11-20T12:13:31.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.893+0000 7f9d4dc14700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48066100 0x7f9d48066510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f9d38017000 tx=0x7f9d3801a000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:31.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.893+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38013b90 con 0x7f9d48066100 2020-11-20T12:13:31.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7f9d38027070 con 0x7f9d48066100 2020-11-20T12:13:31.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48066100 msgr2=0x7f9d48066510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d367fc700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48066100 0x7f9d48066510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f9d38017000 tx=0x7f9d3801a000).stop 2020-11-20T12:13:31.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d367fc700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48066100 msgr2=0x7f9d48066510 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:31.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d367fc700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d48066100 0x7f9d48066510 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:31.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d20056d20 msgr2=0x7f9d200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d20056d20 0x7f9d200591e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9d40009440 tx=0x7f9d40016000).stop 2020-11-20T12:13:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d48060330 msgr2=0x7f9d48065510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 --2- 172.21.15.44:0/1619187020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d48060330 0x7f9d48065510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9d4400c6f0 tx=0x7f9d44007ad0).stop 2020-11-20T12:13:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d4ec16700 1 -- 172.21.15.44:0/1619187020 reap_dead start 2020-11-20T12:13:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d4ec16700 1 -- 172.21.15.44:0/1619187020 reap_dead start 2020-11-20T12:13:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 shutdown_connections 2020-11-20T12:13:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 >> 172.21.15.44:0/1619187020 conn(0x7f9d48062690 msgr2=0x7f9d480a8030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 shutdown_connections 2020-11-20T12:13:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:31.894+0000 7f9d50679700 1 -- 172.21.15.44:0/1619187020 wait complete. 2020-11-20T12:13:31.907 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:13:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 112, 2020-11-20T12:13:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:13:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:13:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:31.939 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:31.940 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:13:31.941 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:13:31.941 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:13:31.941 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:13:31.941 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:13:31.942 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:13:31.954 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:13:31.955 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:13:31.955 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:13:32.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.125+0000 7fcf5919e700 1 Processor -- start 2020-11-20T12:13:32.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.125+0000 7fcf5919e700 1 -- start start 2020-11-20T12:13:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf5919e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcf54140810 0x7fcf54142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf5919e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf540a6260 con 0x7fcf5413a990 2020-11-20T12:13:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf5919e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf540a6490 con 0x7fcf54060330 2020-11-20T12:13:32.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf5919e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf540a52e0 con 0x7fcf54140810 2020-11-20T12:13:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf4bfff700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fcf5413a990 legacy=0x7fcf5413ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:13:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf52d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fcf54060330 0x7fcf540a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47406/0 (socket says 172.21.15.44:47406) 2020-11-20T12:13:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf5359e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcf54140810 0x7fcf54142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.126+0000 7fcf52d9d700 1 -- 172.21.15.44:0/2368425580 learned_addr learned my addr 172.21.15.44:0/2368425580 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:32.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5359e700 1 -- 172.21.15.44:0/2368425580 >> v1:172.21.15.79:6789/0 conn(0x7fcf54060330 legacy=0x7fcf540a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5359e700 1 -- 172.21.15.44:0/2368425580 >> v1:172.21.15.44:6789/0 conn(0x7fcf5413a990 legacy=0x7fcf5413ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5359e700 1 -- 172.21.15.44:0/2368425580 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf540a6730 con 0x7fcf54140810 2020-11-20T12:13:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5359e700 1 --2- 172.21.15.44:0/2368425580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 0x7fcf54142c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fcf4c009c90 tx=0x7fcf4c0091d0).ready entity=mon.1 client_cookie=782d3cda63392a2d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5259c700 1 -- 172.21.15.44:0/2368425580 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00bb80 con 0x7fcf54140810 2020-11-20T12:13:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5259c700 1 -- 172.21.15.44:0/2368425580 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c00bce0 con 0x7fcf54140810 2020-11-20T12:13:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5259c700 1 -- 172.21.15.44:0/2368425580 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c011610 con 0x7fcf54140810 2020-11-20T12:13:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5919e700 1 -- 172.21.15.44:0/2368425580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 msgr2=0x7fcf54142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:32.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5919e700 1 --2- 172.21.15.44:0/2368425580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 0x7fcf54142c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fcf4c009c90 tx=0x7fcf4c0091d0).stop 2020-11-20T12:13:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5919e700 1 -- 172.21.15.44:0/2368425580 shutdown_connections 2020-11-20T12:13:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5919e700 1 --2- 172.21.15.44:0/2368425580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 0x7fcf54142c00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5919e700 1 -- 172.21.15.44:0/2368425580 >> 172.21.15.44:0/2368425580 conn(0x7fcf54062690 msgr2=0x7fcf540a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5919e700 1 -- 172.21.15.44:0/2368425580 shutdown_connections 2020-11-20T12:13:32.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.127+0000 7fcf5919e700 1 -- 172.21.15.44:0/2368425580 wait complete. 2020-11-20T12:13:32.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.129+0000 7fcf5919e700 1 Processor -- start 2020-11-20T12:13:32.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.129+0000 7fcf5919e700 1 -- start start 2020-11-20T12:13:32.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.129+0000 7fcf5919e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf54060330 0x7fcf5413be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:32.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.129+0000 7fcf5919e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5413a990 0x7fcf5413dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:32.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf5919e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 0x7fcf540abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:32.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf5919e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5413cab0 con 0x7fcf5413a990 2020-11-20T12:13:32.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf5919e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5413c350 con 0x7fcf54140810 2020-11-20T12:13:32.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf5919e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5413c570 con 0x7fcf54060330 2020-11-20T12:13:32.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf4bfff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5413a990 msgr2=0x7fcf5413dfb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:32.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf4bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5413a990 0x7fcf5413dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:32.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf54060330 0x7fcf5413be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:32.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf54060330 0x7fcf5413be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52226/0 (socket says 172.21.15.44:52226) 2020-11-20T12:13:32.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf5359e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 0x7fcf540abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:32.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 -- 172.21.15.44:0/293729845 learned_addr learned my addr 172.21.15.44:0/293729845 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:32.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 -- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 msgr2=0x7fcf540abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:32.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 0x7fcf540abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:32.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 -- 172.21.15.44:0/293729845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5413a990 msgr2=0x7fcf5413dfb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:13:32.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5413a990 0x7fcf5413dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:32.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 -- 172.21.15.44:0/293729845 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5413c810 con 0x7fcf54060330 2020-11-20T12:13:32.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.130+0000 7fcf52d9d700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf54060330 0x7fcf5413be60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcf4400c740 tx=0x7fcf44007c80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:32.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.131+0000 7fcf4affd700 1 -- 172.21.15.44:0/293729845 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf440072e0 con 0x7fcf54060330 2020-11-20T12:13:32.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.131+0000 7fcf4affd700 1 -- 172.21.15.44:0/293729845 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44007440 con 0x7fcf54060330 2020-11-20T12:13:32.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.131+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5413ae20 con 0x7fcf54060330 2020-11-20T12:13:32.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.131+0000 7fcf4affd700 1 -- 172.21.15.44:0/293729845 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44005800 con 0x7fcf54060330 2020-11-20T12:13:32.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.131+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf540ac4a0 con 0x7fcf54060330 2020-11-20T12:13:32.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.132+0000 7fcf4affd700 1 -- 172.21.15.44:0/293729845 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcf44010070 con 0x7fcf54060330 2020-11-20T12:13:32.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.132+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf54061650 con 0x7fcf54060330 2020-11-20T12:13:32.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.132+0000 7fcf4affd700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf30056cd0 0x7fcf30059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:32.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.132+0000 7fcf4affd700 1 -- 172.21.15.44:0/293729845 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fcf4406f850 con 0x7fcf54060330 2020-11-20T12:13:32.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.132+0000 7fcf4bfff700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf30056cd0 0x7fcf30059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:32.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.133+0000 7fcf4bfff700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf30056cd0 0x7fcf30059190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fcf3c0036a0 tx=0x7fcf3c006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:32.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.137+0000 7fcf4affd700 1 -- 172.21.15.44:0/293729845 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcf44044b20 con 0x7fcf54060330 2020-11-20T12:13:32.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.422+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcf54060710 con 0x7fcf54060330 2020-11-20T12:13:32.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.422+0000 7fcf4affd700 1 -- 172.21.15.44:0/293729845 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fcf44048140 con 0x7fcf54060330 2020-11-20T12:13:32.426 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:13:32.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf30056cd0 msgr2=0x7fcf30059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:32.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf30056cd0 0x7fcf30059190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fcf3c0036a0 tx=0x7fcf3c006ae0).stop 2020-11-20T12:13:32.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf54060330 msgr2=0x7fcf5413be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:32.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf54060330 0x7fcf5413be60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcf4400c740 tx=0x7fcf44007c80).stop 2020-11-20T12:13:32.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 shutdown_connections 2020-11-20T12:13:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcf30056cd0 0x7fcf30059190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf54140810 0x7fcf540abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcf54060330 0x7fcf5413be60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:32.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 --2- 172.21.15.44:0/293729845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf5413a990 0x7fcf5413dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 >> 172.21.15.44:0/293729845 conn(0x7fcf54062690 msgr2=0x7fcf54138390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 shutdown_connections 2020-11-20T12:13:32.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:32.424+0000 7fcf5919e700 1 -- 172.21.15.44:0/293729845 wait complete. 2020-11-20T12:13:35.441 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:13:35.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.573+0000 7f88f8f92700 1 Processor -- start 2020-11-20T12:13:35.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.573+0000 7f88f8f92700 1 -- start start 2020-11-20T12:13:35.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88f8f92700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f88f40a3e10 0x7f88f40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88f8f92700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40a9770 con 0x7f88f40aa020 2020-11-20T12:13:35.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88f8f92700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40a6da0 con 0x7f88f40aa400 2020-11-20T12:13:35.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88f8f92700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40a56c0 con 0x7f88f40a3e10 2020-11-20T12:13:35.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88f2d9d700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f88f40a3e10 msgr2=0x7f88f40a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:35.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88f259c700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f88f40aa020 legacy=0x7f88f40a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:13:35.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88f2d9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f88f40a3e10 0x7f88f40a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:35.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88ebfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f88f40aa400 0x7f88f40a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47420/0 (socket says 172.21.15.44:47420) 2020-11-20T12:13:35.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.574+0000 7f88ebfff700 1 -- 172.21.15.44:0/2373544481 learned_addr learned my addr 172.21.15.44:0/2373544481 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:35.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278369903 0 0) 0x7f88f40a6da0 con 0x7f88f40aa400 2020-11-20T12:13:35.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d4003240 con 0x7f88f40aa400 2020-11-20T12:13:35.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 208383746 0 0) 0x7f88d4003240 con 0x7f88f40aa400 2020-11-20T12:13:35.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 >> v2:172.21.15.44:3300/0 conn(0x7f88f40a3e10 msgr2=0x7f88f40a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:35.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 --2- 172.21.15.44:0/2373544481 >> v2:172.21.15.44:3300/0 conn(0x7f88f40a3e10 0x7f88f40a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:35.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 >> v1:172.21.15.44:6789/0 conn(0x7f88f40aa020 legacy=0x7f88f40a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:35.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40a7040 con 0x7f88f40aa400 2020-11-20T12:13:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f88dc0014e0 con 0x7f88f40aa400 2020-11-20T12:13:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 >> v1:172.21.15.79:6789/0 conn(0x7f88f40aa400 legacy=0x7f88f40a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d4003730 0x7f88d4003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88d40044e0 0x7f88d4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88d4004e50 0x7f88d400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40a56c0 con 0x7f88d4003730 2020-11-20T12:13:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40041f0 con 0x7f88d40044e0 2020-11-20T12:13:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d400bc20 con 0x7f88d4004e50 2020-11-20T12:13:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f259c700 1 -- 172.21.15.44:0/2373544481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d4003730 msgr2=0x7f88d4003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f259c700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d4003730 0x7f88d4003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88ebfff700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88d40044e0 0x7f88d4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.575+0000 7f88f2d9d700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88d4004e50 0x7f88d400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88ebfff700 1 -- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88d4004e50 msgr2=0x7f88d400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88ebfff700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88d4004e50 0x7f88d400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88ebfff700 1 -- 172.21.15.44:0/2373544481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d4003730 msgr2=0x7f88d4003b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88ebfff700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d4003730 0x7f88d4003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88ebfff700 1 -- 172.21.15.44:0/2373544481 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4003240 con 0x7f88d40044e0 2020-11-20T12:13:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f2d9d700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88d4004e50 0x7f88d400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:13:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f2d9d700 1 -- 172.21.15.44:0/2373544481 reap_dead start 2020-11-20T12:13:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88ebfff700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88d40044e0 0x7f88d4004960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f88dc00c440 tx=0x7f88f40a6f30).ready entity=mon.1 client_cookie=955595d7f66d3d29 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00ce00 con 0x7f88d40044e0 2020-11-20T12:13:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc0072e0 con 0x7f88d40044e0 2020-11-20T12:13:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f1d9b700 1 -- 172.21.15.44:0/2373544481 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc005840 con 0x7f88d40044e0 2020-11-20T12:13:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f8f92700 1 -- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88d40044e0 msgr2=0x7f88d4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f8f92700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88d40044e0 0x7f88d4004960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f88dc00c440 tx=0x7f88f40a6f30).stop 2020-11-20T12:13:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f8f92700 1 -- 172.21.15.44:0/2373544481 shutdown_connections 2020-11-20T12:13:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f8f92700 1 --2- 172.21.15.44:0/2373544481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88d40044e0 0x7f88d4004960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f8f92700 1 -- 172.21.15.44:0/2373544481 >> 172.21.15.44:0/2373544481 conn(0x7f88f4062690 msgr2=0x7f88f40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f8f92700 1 -- 172.21.15.44:0/2373544481 shutdown_connections 2020-11-20T12:13:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.576+0000 7f88f8f92700 1 -- 172.21.15.44:0/2373544481 wait complete. 2020-11-20T12:13:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.577+0000 7f88f8f92700 1 Processor -- start 2020-11-20T12:13:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f8f92700 1 -- start start 2020-11-20T12:13:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f8f92700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40044e0 0x7f88f4133100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f8f92700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f40aca20 0x7f88f40acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f8f92700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f41310c0 0x7f88f4131500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f8f92700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40ac0b0 con 0x7f88d40044e0 2020-11-20T12:13:35.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f8f92700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40a3880 con 0x7f88f41310c0 2020-11-20T12:13:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f259c700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40044e0 msgr2=0x7f88f4133100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f8f92700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40ac2e0 con 0x7f88f40aca20 2020-11-20T12:13:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40044e0 0x7f88f4133100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88ebfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f40aca20 0x7f88f40acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:35.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88ebfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f40aca20 0x7f88f40acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52246/0 (socket says 172.21.15.44:52246) 2020-11-20T12:13:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88ebfff700 1 -- 172.21.15.44:0/45120069 learned_addr learned my addr 172.21.15.44:0/45120069 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.578+0000 7f88f2d9d700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f41310c0 0x7f88f4131500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ebfff700 1 -- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f41310c0 msgr2=0x7f88f4131500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ebfff700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f41310c0 0x7f88f4131500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ebfff700 1 -- 172.21.15.44:0/45120069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40044e0 msgr2=0x7f88f4133100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:13:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ebfff700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40044e0 0x7f88f4133100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ebfff700 1 -- 172.21.15.44:0/45120069 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40ac580 con 0x7f88f40aca20 2020-11-20T12:13:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88f2d9d700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f41310c0 0x7f88f4131500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:13:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ebfff700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f40aca20 0x7f88f40acf50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f88dc00d010 tx=0x7f88dc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:35.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ea7fc700 1 -- 172.21.15.44:0/45120069 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0055e0 con 0x7f88f40aca20 2020-11-20T12:13:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ea7fc700 1 -- 172.21.15.44:0/45120069 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc005bb0 con 0x7f88f40aca20 2020-11-20T12:13:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88ea7fc700 1 -- 172.21.15.44:0/45120069 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0013e0 con 0x7f88f40aca20 2020-11-20T12:13:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f40ad440 con 0x7f88f40aca20 2020-11-20T12:13:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.579+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f41319f0 con 0x7f88f40aca20 2020-11-20T12:13:35.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.580+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f40ac710 con 0x7f88f40aca20 2020-11-20T12:13:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.581+0000 7f88ea7fc700 1 -- 172.21.15.44:0/45120069 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f88dc025030 con 0x7f88f40aca20 2020-11-20T12:13:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.581+0000 7f88ea7fc700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88d0056cd0 0x7f88d0059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.581+0000 7f88ea7fc700 1 -- 172.21.15.44:0/45120069 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f88dc070a40 con 0x7f88f40aca20 2020-11-20T12:13:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.581+0000 7f88f259c700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88d0056cd0 0x7f88d0059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:35.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.582+0000 7f88f259c700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88d0056cd0 0x7f88d0059190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f88e40035d0 tx=0x7f88e4006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:35.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.585+0000 7f88ea7fc700 1 -- 172.21.15.44:0/45120069 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f88dc045dc0 con 0x7f88f40aca20 2020-11-20T12:13:35.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.863+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f88f4061650 con 0x7f88f40aca20 2020-11-20T12:13:35.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.864+0000 7f88ea7fc700 1 -- 172.21.15.44:0/45120069 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f88dc023070 con 0x7f88f40aca20 2020-11-20T12:13:35.868 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":112,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:13:35.869 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88d0056cd0 msgr2=0x7f88d0059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:35.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88d0056cd0 0x7f88d0059190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f88e40035d0 tx=0x7f88e4006ae0).stop 2020-11-20T12:13:35.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f40aca20 msgr2=0x7f88f40acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:35.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f40aca20 0x7f88f40acf50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f88dc00d010 tx=0x7f88dc015000).stop 2020-11-20T12:13:35.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 shutdown_connections 2020-11-20T12:13:35.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88d0056cd0 0x7f88d0059190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:35.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f41310c0 0x7f88f4131500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:35.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88d40044e0 0x7f88f4133100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:35.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 --2- 172.21.15.44:0/45120069 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f40aca20 0x7f88f40acf50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:35.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 >> 172.21.15.44:0/45120069 conn(0x7f88f4062690 msgr2=0x7f88f4139da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:35.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 shutdown_connections 2020-11-20T12:13:35.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:35.866+0000 7f88f8f92700 1 -- 172.21.15.44:0/45120069 wait complete. 2020-11-20T12:13:37.705 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:13:37.717 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:13:38.882 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:13:39.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.016+0000 7fa1432c0700 1 Processor -- start 2020-11-20T12:13:39.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.016+0000 7fa1432c0700 1 -- start start 2020-11-20T12:13:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa1432c0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 0x7fa13c138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:39.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa1432c0700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0a6d90 con 0x7fa13c0a4e10 2020-11-20T12:13:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa1432c0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0a7030 con 0x7fa13c061340 2020-11-20T12:13:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa1432c0700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0a5900 con 0x7fa13c0a74d0 2020-11-20T12:13:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa14185d700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 msgr2=0x7fa13c138190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa14085b700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fa13c0a4e10 legacy=0x7fa13c1433f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:13:39.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa14185d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 0x7fa13c138190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:39.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa14105c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa13c061340 0x7fa13c137a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47438/0 (socket says 172.21.15.44:47438) 2020-11-20T12:13:39.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.017+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 learned_addr learned my addr 172.21.15.44:0/3833598188 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:39.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.218+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> v1:172.21.15.44:6789/0 conn(0x7fa13c0a4e10 legacy=0x7fa13c1433f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:13:39.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.218+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 msgr2=0x7fa13c138190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:39.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.218+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 0x7fa13c138190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:13:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.619+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> v1:172.21.15.44:6789/0 conn(0x7fa13c0a4e10 legacy=0x7fa13c1433f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:13:39.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.619+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 msgr2=0x7fa13c138190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:39.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:39.619+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 0x7fa13c138190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:13:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:40.420+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> v1:172.21.15.44:6789/0 conn(0x7fa13c0a4e10 legacy=0x7fa13c1433f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:13:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:40.420+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 msgr2=0x7fa13c138190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:40.420+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 0x7fa13c138190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:13:42.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 msgr2=0x7fa13c138190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa13c0a74d0 0x7fa13c138190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:42.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 >> v1:172.21.15.79:6789/0 conn(0x7fa13c061340 legacy=0x7fa13c137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:42.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 >> v1:172.21.15.44:6789/0 conn(0x7fa13c0a4e10 legacy=0x7fa13c1433f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:42.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:42.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:42.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c008920 con 0x7fa11c0033b0 2020-11-20T12:13:42.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c008a60 con 0x7fa11c005dd0 2020-11-20T12:13:42.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:42.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:42.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:13:42.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:42.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:42.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:13:42.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:42.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:42.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:42.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.017+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:42.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:42.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:42.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:13:42.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:42.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:13:42.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:42.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:42.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:42.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:13:42.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.218+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:13:42.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:42.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:42.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:13:42.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:42.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:13:42.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:42.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:42.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:42.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:13:42.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:42.619+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:13:43.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:43.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:43.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:13:43.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:43.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:13:43.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:43.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:43.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:43.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:13:43.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:43.420+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:13:45.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 msgr2=0x7fa11c008270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:45.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa11c005dd0 0x7fa11c008270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:45.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 msgr2=0x7fa11c005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:45.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c0033b0 0x7fa11c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:45.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 reap_dead start 2020-11-20T12:13:45.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 0x7fa11c006240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:45.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:45.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c061720 con 0x7fa11c006730 2020-11-20T12:13:45.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c008920 con 0x7fa11c003820 2020-11-20T12:13:45.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa12affd700 1 -- 172.21.15.44:0/3833598188 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c008a60 con 0x7fa11c00aeb0 2020-11-20T12:13:45.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 msgr2=0x7fa11c006240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:45.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 0x7fa11c006240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:45.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.017+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:45.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.018+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:13:45.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.018+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:45.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.018+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:45.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.018+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:45.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.218+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 msgr2=0x7fa11c006240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:45.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.218+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 0x7fa11c006240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:13:45.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.219+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:45.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.219+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:13:45.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.219+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:45.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.219+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:45.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.219+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:13:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.619+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 msgr2=0x7fa11c006240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.619+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 0x7fa11c006240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:13:45.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.620+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:45.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.620+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:13:45.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.620+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:45.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.620+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:45.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:45.620+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:13:46.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.420+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 msgr2=0x7fa11c006240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:46.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.420+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 0x7fa11c006240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:13:46.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.421+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:46.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.421+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:13:46.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.421+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:13:46.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.421+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:13:46.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.421+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:13:46.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.796+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944729717 0 0) 0x7fa13c061720 con 0x7fa11c006730 2020-11-20T12:13:46.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.796+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa124003240 con 0x7fa11c006730 2020-11-20T12:13:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1558214446 0 0) 0x7fa124003240 con 0x7fa11c006730 2020-11-20T12:13:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 msgr2=0x7fa11c00d2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.79:3300/0 conn(0x7fa11c00aeb0 0x7fa11c00d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 msgr2=0x7fa11c006240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 --2- 172.21.15.44:0/3833598188 >> v2:172.21.15.44:3300/0 conn(0x7fa11c003820 0x7fa11c006240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0a6d90 con 0x7fa11c006730 2020-11-20T12:13:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fa12c00e3c0 con 0x7fa11c006730 2020-11-20T12:13:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 >> v1:172.21.15.79:6789/0 conn(0x7fa11c006730 legacy=0x7fa11c006bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124003730 0x7fa124003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa124004060 0x7fa124004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa124004be0 0x7fa12400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c008920 con 0x7fa124003730 2020-11-20T12:13:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c008a60 con 0x7fa124004060 2020-11-20T12:13:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa124005300 con 0x7fa124004be0 2020-11-20T12:13:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14105c700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124003730 msgr2=0x7fa124003b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14105c700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124003730 0x7fa124003b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa124004060 0x7fa124004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14185d700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa124004be0 0x7fa12400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa124004be0 msgr2=0x7fa12400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:13:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa124004be0 0x7fa12400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124003730 msgr2=0x7fa124003b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.798+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124003730 0x7fa124003b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.799+0000 7fa14085b700 1 -- 172.21.15.44:0/3833598188 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa124003240 con 0x7fa124004060 2020-11-20T12:13:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.799+0000 7fa14185d700 1 -- 172.21.15.44:0/3833598188 reap_dead start 2020-11-20T12:13:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.799+0000 7fa14085b700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa124004060 0x7fa124004540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fa13c0a5250 tx=0x7fa12c008040).ready entity=mon.1 client_cookie=3477b6956d0a00d5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.800+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c005880 con 0x7fa124004060 2020-11-20T12:13:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.800+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c0145c0 con 0x7fa124004060 2020-11-20T12:13:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.800+0000 7fa12bfff700 1 -- 172.21.15.44:0/3833598188 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c014940 con 0x7fa124004060 2020-11-20T12:13:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.801+0000 7fa1432c0700 1 -- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa124004060 msgr2=0x7fa124004540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:46.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.801+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa124004060 0x7fa124004540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fa13c0a5250 tx=0x7fa12c008040).stop 2020-11-20T12:13:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.801+0000 7fa1432c0700 1 -- 172.21.15.44:0/3833598188 shutdown_connections 2020-11-20T12:13:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.801+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3833598188 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa124004060 0x7fa124004540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.801+0000 7fa1432c0700 1 -- 172.21.15.44:0/3833598188 >> 172.21.15.44:0/3833598188 conn(0x7fa13c0ad6e0 msgr2=0x7fa13c0adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.801+0000 7fa1432c0700 1 -- 172.21.15.44:0/3833598188 shutdown_connections 2020-11-20T12:13:46.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.801+0000 7fa1432c0700 1 -- 172.21.15.44:0/3833598188 wait complete. 2020-11-20T12:13:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.802+0000 7fa1432c0700 1 Processor -- start 2020-11-20T12:13:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.802+0000 7fa1432c0700 1 -- start start 2020-11-20T12:13:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa1432c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124004060 0x7fa13c0a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa1432c0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa13c067760 0x7fa13c14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:46.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa1432c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa13c14d5f0 0x7fa13c14da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa1432c0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0a4150 con 0x7fa124004060 2020-11-20T12:13:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa1432c0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0a22f0 con 0x7fa13c14d5f0 2020-11-20T12:13:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa1432c0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c141c70 con 0x7fa13c067760 2020-11-20T12:13:46.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa14105c700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124004060 msgr2=0x7fa13c0a4540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa14105c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124004060 0x7fa13c0a4540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa14185d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa13c14d5f0 0x7fa13c14da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa14085b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa13c067760 0x7fa13c14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.803+0000 7fa14085b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa13c067760 0x7fa13c14d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52308/0 (socket says 172.21.15.44:52308) 2020-11-20T12:13:46.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14085b700 1 -- 172.21.15.44:0/3695386235 learned_addr learned my addr 172.21.15.44:0/3695386235 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14185d700 1 -- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa13c067760 msgr2=0x7fa13c14d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:46.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14185d700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa13c067760 0x7fa13c14d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14185d700 1 -- 172.21.15.44:0/3695386235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124004060 msgr2=0x7fa13c0a4540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:13:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14185d700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124004060 0x7fa13c0a4540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14185d700 1 -- 172.21.15.44:0/3695386235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c14df40 con 0x7fa13c14d5f0 2020-11-20T12:13:46.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14085b700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa13c067760 0x7fa13c14d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:13:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa14185d700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa13c14d5f0 0x7fa13c14da50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fa13c13bc30 tx=0x7fa134010040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa1297fa700 1 -- 172.21.15.44:0/3695386235 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1340183b0 con 0x7fa13c14d5f0 2020-11-20T12:13:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa1297fa700 1 -- 172.21.15.44:0/3695386235 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa134014040 con 0x7fa13c14d5f0 2020-11-20T12:13:46.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c0a1990 con 0x7fa13c14d5f0 2020-11-20T12:13:46.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa1297fa700 1 -- 172.21.15.44:0/3695386235 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134018740 con 0x7fa13c14d5f0 2020-11-20T12:13:46.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.804+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c13bce0 con 0x7fa13c14d5f0 2020-11-20T12:13:46.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.805+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c0446d0 con 0x7fa13c14d5f0 2020-11-20T12:13:46.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.806+0000 7fa1297fa700 1 -- 172.21.15.44:0/3695386235 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa134018ab0 con 0x7fa13c14d5f0 2020-11-20T12:13:46.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.806+0000 7fa1297fa700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa114056cd0 0x7fa114059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.806+0000 7fa1297fa700 1 -- 172.21.15.44:0/3695386235 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa1340700e0 con 0x7fa13c14d5f0 2020-11-20T12:13:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.806+0000 7fa14105c700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa114056cd0 0x7fa114059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.806+0000 7fa14105c700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa114056cd0 0x7fa114059190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa138000f20 tx=0x7fa138009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:46.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:46.810+0000 7fa1297fa700 1 -- 172.21.15.44:0/3695386235 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa1340453b0 con 0x7fa13c14d5f0 2020-11-20T12:13:47.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.080+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa13c0697f0 con 0x7fa13c14d5f0 2020-11-20T12:13:47.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.081+0000 7fa1297fa700 1 -- 172.21.15.44:0/3695386235 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fa1340489d0 con 0x7fa13c14d5f0 2020-11-20T12:13:47.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.082+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa114056cd0 msgr2=0x7fa114059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa114056cd0 0x7fa114059190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa138000f20 tx=0x7fa138009040).stop 2020-11-20T12:13:47.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa13c14d5f0 msgr2=0x7fa13c14da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa13c14d5f0 0x7fa13c14da50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fa13c13bc30 tx=0x7fa134010040).stop 2020-11-20T12:13:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 shutdown_connections 2020-11-20T12:13:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa114056cd0 0x7fa114059190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa13c14d5f0 0x7fa13c14da50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa124004060 0x7fa13c0a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:47.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 --2- 172.21.15.44:0/3695386235 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa13c067760 0x7fa13c14d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 >> 172.21.15.44:0/3695386235 conn(0x7fa13c0ad6e0 msgr2=0x7fa13c0a5280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 shutdown_connections 2020-11-20T12:13:47.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.083+0000 7fa1432c0700 1 -- 172.21.15.44:0/3695386235 wait complete. 2020-11-20T12:13:47.645 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:13:47.646 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:13:47.646 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:13:47.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.803+0000 7fb8d79a9700 1 Processor -- start 2020-11-20T12:13:47.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.803+0000 7fb8d79a9700 1 -- start start 2020-11-20T12:13:47.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.804+0000 7fb8d79a9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8d00aa020 0x7fb8d00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.804+0000 7fb8d79a9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d013ac60 0x7fb8d00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.804+0000 7fb8d79a9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d00a9770 con 0x7fb8d00aa400 2020-11-20T12:13:47.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.804+0000 7fb8d79a9700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d00a56c0 con 0x7fb8d00aa020 2020-11-20T12:13:47.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.804+0000 7fb8d79a9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00a6da0 con 0x7fb8d013ac60 2020-11-20T12:13:47.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.804+0000 7fb8d5745700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fb8d00aa020 msgr2=0x7fb8d00a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.804+0000 7fb8d5745700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8d00aa020 0x7fb8d00a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d5f46700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d013ac60 0x7fb8d00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d4f44700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb8d00aa400 0x7fb8d00a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47502/0 (socket says 172.21.15.44:47502) 2020-11-20T12:13:47.810 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d4f44700 1 -- 172.21.15.44:0/2158158171 learned_addr learned my addr 172.21.15.44:0/2158158171 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:13:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d5f46700 1 --2- 172.21.15.44:0/2158158171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d013ac60 0x7fb8d00a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52318/0 (socket says 172.21.15.44:52318) 2020-11-20T12:13:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d5f46700 1 -- 172.21.15.44:0/2158158171 >> v2:172.21.15.44:3300/0 conn(0x7fb8d00aa020 msgr2=0x7fb8d00a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8bffff700 1 -- 172.21.15.44:0/2158158171 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459035999 0 0) 0x7fb8c00010d0 con 0x7fb8d00aa400 2020-11-20T12:13:47.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d5f46700 1 --2- 172.21.15.44:0/2158158171 >> v2:172.21.15.44:3300/0 conn(0x7fb8d00aa020 0x7fb8d00a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d5f46700 1 -- 172.21.15.44:0/2158158171 >> v1:172.21.15.79:6789/0 conn(0x7fb8d00aa400 legacy=0x7fb8d00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d5f46700 1 -- 172.21.15.44:0/2158158171 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d00a7040 con 0x7fb8d013ac60 2020-11-20T12:13:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8d5f46700 1 --2- 172.21.15.44:0/2158158171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d013ac60 0x7fb8d00a3700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb8c8009c90 tx=0x7fb8c80091d0).ready entity=mon.2 client_cookie=874f49bcafa91c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:47.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8bffff700 1 -- 172.21.15.44:0/2158158171 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800bbc0 con 0x7fb8d013ac60 2020-11-20T12:13:47.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8bffff700 1 -- 172.21.15.44:0/2158158171 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c800bd20 con 0x7fb8d013ac60 2020-11-20T12:13:47.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.805+0000 7fb8bffff700 1 -- 172.21.15.44:0/2158158171 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8010680 con 0x7fb8d013ac60 2020-11-20T12:13:47.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 -- 172.21.15.44:0/2158158171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d013ac60 msgr2=0x7fb8d00a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 --2- 172.21.15.44:0/2158158171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d013ac60 0x7fb8d00a3700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb8c8009c90 tx=0x7fb8c80091d0).stop 2020-11-20T12:13:47.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 -- 172.21.15.44:0/2158158171 shutdown_connections 2020-11-20T12:13:47.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 --2- 172.21.15.44:0/2158158171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d013ac60 0x7fb8d00a3700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:47.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 --2- 172.21.15.44:0/2158158171 >> v2:172.21.15.44:3300/0 conn(0x7fb8d00aa020 0x7fb8d00a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:47.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 -- 172.21.15.44:0/2158158171 >> 172.21.15.44:0/2158158171 conn(0x7fb8d0062690 msgr2=0x7fb8d00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:47.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 -- 172.21.15.44:0/2158158171 shutdown_connections 2020-11-20T12:13:47.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.806+0000 7fb8d79a9700 1 -- 172.21.15.44:0/2158158171 wait complete. 2020-11-20T12:13:47.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.807+0000 7fb8d79a9700 1 Processor -- start 2020-11-20T12:13:47.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.807+0000 7fb8d79a9700 1 -- start start 2020-11-20T12:13:47.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d79a9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d00aa020 0x7fb8d00ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d79a9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d00aa400 0x7fb8d0148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d79a9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d013ac60 0x7fb8d00ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d79a9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00ad260 con 0x7fb8d00aa400 2020-11-20T12:13:47.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d79a9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00acfc0 con 0x7fb8d013ac60 2020-11-20T12:13:47.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d79a9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00ad540 con 0x7fb8d00aa020 2020-11-20T12:13:47.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d4f44700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d00aa400 msgr2=0x7fb8d0148da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:47.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d4f44700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d00aa400 0x7fb8d0148da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:47.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d5f46700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d013ac60 0x7fb8d00ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:47.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d5745700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d00aa020 0x7fb8d00ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:47.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d5f46700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d013ac60 0x7fb8d00ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:32936/0 (socket says 172.21.15.44:32936) 2020-11-20T12:13:47.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d5f46700 1 -- 172.21.15.44:0/3415025991 learned_addr learned my addr 172.21.15.44:0/3415025991 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:47.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.808+0000 7fb8d5745700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d00aa020 0x7fb8d00ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52320/0 (socket says 172.21.15.44:52320) 2020-11-20T12:13:47.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d5745700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d013ac60 msgr2=0x7fb8d00ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d5745700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d013ac60 0x7fb8d00ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:47.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d5745700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d00aa400 msgr2=0x7fb8d0148da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:13:47.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d5745700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d00aa400 0x7fb8d0148da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:47.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d5745700 1 -- 172.21.15.44:0/3415025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d00665d0 con 0x7fb8d00aa020 2020-11-20T12:13:47.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d5745700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d00aa020 0x7fb8d00ac9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb8cc009660 tx=0x7fb8cc006d90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:47.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc006170 con 0x7fb8d00aa020 2020-11-20T12:13:47.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc0062d0 con 0x7fb8d00aa020 2020-11-20T12:13:47.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d00aac60 con 0x7fb8d00aa020 2020-11-20T12:13:47.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0038e0 con 0x7fb8d00aa020 2020-11-20T12:13:47.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.809+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d0137a80 con 0x7fb8d00aa020 2020-11-20T12:13:47.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.810+0000 7fb8d79a9700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065150 0x7fb8d0065530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.810+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0137d00 con 0x7fb8d0065150 2020-11-20T12:13:47.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.810+0000 7fb8d4f44700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065150 0x7fb8d0065530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:47.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb8cc00f070 con 0x7fb8d00aa020 2020-11-20T12:13:47.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8d4f44700 1 -- 172.21.15.44:0/3415025991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8d00ad700 con 0x7fb8d0065150 2020-11-20T12:13:47.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8d4f44700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065150 0x7fb8d0065530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb8d01354e0 tx=0x7fb8c0007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:47.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8a8056cd0 0x7fb8a8059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb8cc06f7c0 con 0x7fb8d00aa020 2020-11-20T12:13:47.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8d5f46700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8a8056cd0 0x7fb8a8059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:47.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007320 con 0x7fb8d0065150 2020-11-20T12:13:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb8c0007480 con 0x7fb8d0065150 2020-11-20T12:13:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065150 msgr2=0x7fb8d0065530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065150 0x7fb8d0065530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb8d01354e0 tx=0x7fb8c0007a00).stop 2020-11-20T12:13:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065150 msgr2=0x7fb8d0065530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:47.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.811+0000 7fb8bd7fa700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065150 0x7fb8d0065530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:47.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.812+0000 7fb8d5f46700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8a8056cd0 0x7fb8a8059190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fb8c8006db0 tx=0x7fb8c8018000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:47.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.820+0000 7fb8d79a9700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065a20 0x7fb8d0131c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:47.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.820+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0061650 con 0x7fb8d0065a20 2020-11-20T12:13:47.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8d4f44700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065a20 0x7fb8d0131c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:47.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8d4f44700 1 -- 172.21.15.44:0/3415025991 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb8d00446d0 con 0x7fb8d0065a20 2020-11-20T12:13:47.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8d4f44700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065a20 0x7fb8d0131c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb8c0010000 tx=0x7fb8c0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:47.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0013c00 con 0x7fb8d0065a20 2020-11-20T12:13:47.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4767 (secure 0 0 0) 0x7fb8c00011f0 con 0x7fb8d0065a20 2020-11-20T12:13:47.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065a20 msgr2=0x7fb8d0131c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8bd7fa700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065a20 0x7fb8d0131c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb8c0010000 tx=0x7fb8c0016000).stop 2020-11-20T12:13:47.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8bd7fa700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065a20 msgr2=0x7fb8d0131c70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:47.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.821+0000 7fb8bd7fa700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d0065a20 0x7fb8d0131c70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:47.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8a8056cd0 msgr2=0x7fb8a8059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8a8056cd0 0x7fb8a8059190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fb8c8006db0 tx=0x7fb8c8018000).stop 2020-11-20T12:13:47.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d00aa020 msgr2=0x7fb8d00ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:47.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 --2- 172.21.15.44:0/3415025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d00aa020 0x7fb8d00ac9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb8cc009660 tx=0x7fb8cc006d90).stop 2020-11-20T12:13:47.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d5f46700 1 -- 172.21.15.44:0/3415025991 reap_dead start 2020-11-20T12:13:47.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d5f46700 1 -- 172.21.15.44:0/3415025991 reap_dead start 2020-11-20T12:13:47.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 shutdown_connections 2020-11-20T12:13:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 >> 172.21.15.44:0/3415025991 conn(0x7fb8d0062690 msgr2=0x7fb8d013c540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 shutdown_connections 2020-11-20T12:13:47.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:47.822+0000 7fb8d79a9700 1 -- 172.21.15.44:0/3415025991 wait complete. 2020-11-20T12:13:47.833 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:47.834 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:47.834 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:47.834 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:13:47.834 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 114, 2020-11-20T12:13:47.834 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:47.835 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:47.835 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:47.835 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:47.835 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:13:47.835 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:47.836 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:47.836 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:47.836 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:47.836 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:47.836 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:47.837 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:47.837 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:47.837 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:47.837 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:47.838 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:47.838 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:47.838 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:47.838 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:47.838 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:47.839 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:47.839 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:47.839 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:47.839 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:47.839 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:47.840 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:47.840 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:47.840 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:47.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.840 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:47.841 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:47.841 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:47.841 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:47.841 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:47.841 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:47.842 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:47.842 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:47.842 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:47.842 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:47.842 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:47.843 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:47.843 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:47.843 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:47.843 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:47.843 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:47.844 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:47.844 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:47.844 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:47.844 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:47.844 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:47.845 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:47.845 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:47.845 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:47.845 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:47.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.846 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:47.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.846 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:47.846 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:47.846 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:47.847 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:47.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.847 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:47.847 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:47.847 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:47.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:47.848 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:47.848 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:47.849 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.849 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:47.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.849 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:47.849 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:47.850 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:47.850 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:47.850 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:47.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.851 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:47.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:47.851 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:47.851 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:47.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.852 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:47.852 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:47.852 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:47.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:47.853 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:47.853 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:47.853 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.854 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:47.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.854 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:47.854 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:47.854 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:47.855 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:47.855 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:47.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.855 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:47.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:47.856 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:47.856 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:47.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.856 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:47.857 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:47.857 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:47.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:47.858 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:47.858 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:47.858 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.858 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:47.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.859 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:47.859 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:47.859 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:47.859 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:47.859 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:47.860 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.860 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:47.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.860 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:47.860 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:47.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.861 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:47.861 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:47.861 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:47.861 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.862 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:47.862 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:13:47.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.862 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:47.862 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:47.863 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:13:47.863 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.863 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:47.863 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:13:47.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.864 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:47.864 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:47.864 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:13:47.864 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.864 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:47.865 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:47.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.865 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:47.865 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:47.865 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:13:47.866 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.866 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:47.866 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:13:47.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:47.866 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:47.867 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:47.867 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:47.867 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:47.867 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:47.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:47.868 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:47.868 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:47.869 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:13:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.012+0000 7f4bd4829700 1 Processor -- start 2020-11-20T12:13:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.012+0000 7f4bd4829700 1 -- start start 2020-11-20T12:13:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bd4829700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00aad00 0x7f4bd00a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bd4829700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd01385f0 con 0x7f4bd00a86d0 2020-11-20T12:13:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bd4829700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd00aca50 con 0x7f4bd00ab0e0 2020-11-20T12:13:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bd4829700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd013e250 con 0x7f4bd00aad00 2020-11-20T12:13:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bced9d700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f4bd00a86d0 legacy=0x7f4bd00ac340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:13:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bce59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00aad00 0x7f4bd00a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bce59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00aad00 0x7f4bd00a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52332/0 (socket says 172.21.15.44:52332) 2020-11-20T12:13:48.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bc7fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4bd00ab0e0 0x7f4bd00abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47520/0 (socket says 172.21.15.44:47520) 2020-11-20T12:13:48.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bce59c700 1 -- 172.21.15.44:0/1842209375 learned_addr learned my addr 172.21.15.44:0/1842209375 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:48.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bcdd9b700 1 -- 172.21.15.44:0/1842209375 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159414784 0 0) 0x7f4bd00aca50 con 0x7f4bd00ab0e0 2020-11-20T12:13:48.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bce59c700 1 -- 172.21.15.44:0/1842209375 >> v1:172.21.15.79:6789/0 conn(0x7f4bd00ab0e0 legacy=0x7f4bd00abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:48.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bce59c700 1 -- 172.21.15.44:0/1842209375 >> v1:172.21.15.44:6789/0 conn(0x7f4bd00a86d0 legacy=0x7f4bd00ac340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:13:48.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.013+0000 7f4bce59c700 1 -- 172.21.15.44:0/1842209375 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd013e4f0 con 0x7f4bd00aad00 2020-11-20T12:13:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bce59c700 1 --2- 172.21.15.44:0/1842209375 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00aad00 0x7f4bd00a94d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4bc000ba20 tx=0x7f4bc0006eb0).ready entity=mon.2 client_cookie=43151fc1335c3c32 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bcdd9b700 1 -- 172.21.15.44:0/1842209375 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0009920 con 0x7f4bd00aad00 2020-11-20T12:13:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bcdd9b700 1 -- 172.21.15.44:0/1842209375 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc0009a80 con 0x7f4bd00aad00 2020-11-20T12:13:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bcdd9b700 1 -- 172.21.15.44:0/1842209375 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0009e00 con 0x7f4bd00aad00 2020-11-20T12:13:48.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bd4829700 1 -- 172.21.15.44:0/1842209375 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00aad00 msgr2=0x7f4bd00a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:48.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bd4829700 1 --2- 172.21.15.44:0/1842209375 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00aad00 0x7f4bd00a94d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4bc000ba20 tx=0x7f4bc0006eb0).stop 2020-11-20T12:13:48.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bd4829700 1 -- 172.21.15.44:0/1842209375 shutdown_connections 2020-11-20T12:13:48.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bd4829700 1 --2- 172.21.15.44:0/1842209375 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00aad00 0x7f4bd00a94d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:48.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bd4829700 1 -- 172.21.15.44:0/1842209375 >> 172.21.15.44:0/1842209375 conn(0x7f4bd0060170 msgr2=0x7f4bd0060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:48.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bd4829700 1 -- 172.21.15.44:0/1842209375 shutdown_connections 2020-11-20T12:13:48.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.014+0000 7f4bd4829700 1 -- 172.21.15.44:0/1842209375 wait complete. 2020-11-20T12:13:48.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.015+0000 7f4bd4829700 1 Processor -- start 2020-11-20T12:13:48.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.015+0000 7f4bd4829700 1 -- start start 2020-11-20T12:13:48.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bd4829700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00a86d0 0x7f4bd014a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:48.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bd4829700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bd00aad00 0x7f4bd009df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bd4829700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00ab0e0 0x7f4bd013ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bd4829700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd013b720 con 0x7f4bd00ab0e0 2020-11-20T12:13:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bd4829700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd013b010 con 0x7f4bd00aad00 2020-11-20T12:13:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bd4829700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd013a3e0 con 0x7f4bd00a86d0 2020-11-20T12:13:48.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bced9d700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00ab0e0 msgr2=0x7f4bd013ab20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:13:48.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bced9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00ab0e0 0x7f4bd013ab20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:13:48.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bce59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00a86d0 0x7f4bd014a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:48.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bc7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bd00aad00 0x7f4bd009df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:48.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bc7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bd00aad00 0x7f4bd009df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:32952/0 (socket says 172.21.15.44:32952) 2020-11-20T12:13:48.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bce59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00a86d0 0x7f4bd014a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52338/0 (socket says 172.21.15.44:52338) 2020-11-20T12:13:48.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.016+0000 7f4bc7fff700 1 -- 172.21.15.44:0/2272786176 learned_addr learned my addr 172.21.15.44:0/2272786176 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:13:48.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bce59c700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bd00aad00 msgr2=0x7f4bd009df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:48.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bce59c700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bd00aad00 0x7f4bd009df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:48.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bce59c700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00ab0e0 msgr2=0x7f4bd013ab20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:13:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bce59c700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bd00ab0e0 0x7f4bd013ab20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:13:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bce59c700 1 -- 172.21.15.44:0/2272786176 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd013a680 con 0x7f4bd00a86d0 2020-11-20T12:13:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bce59c700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00a86d0 0x7f4bd014a310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4bc000ba20 tx=0x7f4bc0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc001e460 con 0x7f4bd00a86d0 2020-11-20T12:13:48.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc001e5c0 con 0x7f4bd00a86d0 2020-11-20T12:13:48.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc001e940 con 0x7f4bd00a86d0 2020-11-20T12:13:48.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd00a7d80 con 0x7f4bd00a86d0 2020-11-20T12:13:48.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.017+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0139ab0 con 0x7f4bd00a86d0 2020-11-20T12:13:48.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.018+0000 7f4bd4829700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd0139d00 0x7f4bd014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:48.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.018+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd005c5f0 con 0x7f4bd0139d00 2020-11-20T12:13:48.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.018+0000 7f4bc7fff700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd0139d00 0x7f4bd014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:48.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.018+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4bc001a040 con 0x7f4bd00a86d0 2020-11-20T12:13:48.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.018+0000 7f4bc7fff700 1 -- 172.21.15.44:0/2272786176 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bd013a810 con 0x7f4bd0139d00 2020-11-20T12:13:48.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.018+0000 7f4bc7fff700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd0139d00 0x7f4bd014aa50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4bd00a4910 tx=0x7f4bb8007c20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:48.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4bac056cd0 0x7f4bac059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:48.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4bc0071620 con 0x7f4bd00a86d0 2020-11-20T12:13:48.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bced9d700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4bac056cd0 0x7f4bac059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:48.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8011650 con 0x7f4bd0139d00 2020-11-20T12:13:48.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4bb80117b0 con 0x7f4bd0139d00 2020-11-20T12:13:48.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd0139d00 msgr2=0x7f4bd014aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd0139d00 0x7f4bd014aa50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4bd00a4910 tx=0x7f4bb8007c20).stop 2020-11-20T12:13:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd0139d00 msgr2=0x7f4bd014aa50 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bc67fc700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd0139d00 0x7f4bd014aa50 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0x7f4bd00a4910 tx=0x7f4bb8007c20).stop 2020-11-20T12:13:48.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.019+0000 7f4bced9d700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4bac056cd0 0x7f4bac059190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f4bc8006fd0 tx=0x7f4bc8006cf0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.028+0000 7f4bd4829700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd014af40 0x7f4bd014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:13:48.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.028+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0069680 con 0x7f4bd014af40 2020-11-20T12:13:48.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.029+0000 7f4bc7fff700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd014af40 0x7f4bd014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:13:48.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.029+0000 7f4bc7fff700 1 -- 172.21.15.44:0/2272786176 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4bd00a80d0 con 0x7f4bd014af40 2020-11-20T12:13:48.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.029+0000 7f4bc7fff700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd014af40 0x7f4bd014d8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4bd00a4910 tx=0x7f4bb8017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:13:48.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.029+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8014db0 con 0x7f4bd014af40 2020-11-20T12:13:48.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4595 (secure 0 0 0) 0x7f4bb8011440 con 0x7f4bd014af40 2020-11-20T12:13:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd014af40 msgr2=0x7f4bd014d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bc67fc700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd014af40 0x7f4bd014d8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4bd00a4910 tx=0x7f4bb8017000).stop 2020-11-20T12:13:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bc67fc700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd014af40 msgr2=0x7f4bd014d8e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:13:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bc67fc700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd014af40 0x7f4bd014d8e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:13:48.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4bac056cd0 msgr2=0x7f4bac059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:48.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4bac056cd0 0x7f4bac059190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f4bc8006fd0 tx=0x7f4bc8006cf0).stop 2020-11-20T12:13:48.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00a86d0 msgr2=0x7f4bd014a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:13:48.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 --2- 172.21.15.44:0/2272786176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4bd00a86d0 0x7f4bd014a310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f4bc000ba20 tx=0x7f4bc0015000).stop 2020-11-20T12:13:48.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bced9d700 1 -- 172.21.15.44:0/2272786176 reap_dead start 2020-11-20T12:13:48.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bced9d700 1 -- 172.21.15.44:0/2272786176 reap_dead start 2020-11-20T12:13:48.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 shutdown_connections 2020-11-20T12:13:48.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 >> 172.21.15.44:0/2272786176 conn(0x7f4bd0060170 msgr2=0x7f4bd00ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:13:48.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 shutdown_connections 2020-11-20T12:13:48.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:13:48.030+0000 7f4bd4829700 1 -- 172.21.15.44:0/2272786176 wait complete. 2020-11-20T12:13:48.041 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:13:48.041 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:48.041 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:48.041 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:13:48.041 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 114, 2020-11-20T12:13:48.042 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:13:48.042 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:13:48.042 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:13:48.042 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:48.042 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:13:48.043 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:48.043 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:13:48.043 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:13:48.043 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:48.043 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:48.044 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:48.044 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:48.044 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:48.044 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:48.044 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:48.045 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:48.045 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:48.045 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:13:48.045 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:13:48.045 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:48.046 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:48.046 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:48.046 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:48.046 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:48.046 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:48.047 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:48.047 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:48.047 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:48.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.047 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:13:48.048 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:13:48.048 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:13:48.048 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:13:48.048 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:13:48.048 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:13:48.049 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:48.049 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:13:48.049 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:13:48.049 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:13:48.049 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:13:48.050 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:13:48.050 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:13:48.050 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:13:48.050 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:13:48.050 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:13:48.051 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:13:48.051 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:13:48.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:13:48.051 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:13:48.051 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:13:48.052 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:13:48.052 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:13:48.052 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:48.052 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:13:48.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.053 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:13:48.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.053 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:13:48.053 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:13:48.054 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:48.054 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:48.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:48.054 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:13:48.055 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:48.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:48.055 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:13:48.056 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:48.056 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.056 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:48.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.056 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:13:48.057 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:13:48.057 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:48.057 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:48.057 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:48.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.058 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:13:48.058 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:13:48.058 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:48.058 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:48.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.059 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:48.059 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:13:48.059 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:48.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:48.060 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:13:48.060 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:48.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.061 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:48.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.061 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:13:48.061 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:13:48.061 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:13:48.063 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:13:48.063 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:13:48.063 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:13:48.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:13:48.064 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:13:48.064 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:48.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:13:48.065 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:13:48.065 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:13:48.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.065 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:48.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.066 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:13:48.066 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:13:48.066 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:13:48.066 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:13:48.066 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:13:48.067 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.067 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:48.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.067 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:13:48.067 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:13:48.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.068 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:48.068 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:48.068 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:48.068 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.069 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:48.069 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:13:48.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.069 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:48.069 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:48.070 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:48.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.070 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:48.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:13:48.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:13:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:13:48.071 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:13:48.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:13:48.072 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:13:48.072 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:13:48.072 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:13:48.072 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:13:48.073 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:13:48.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:13:48.073 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:13:48.073 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:13:48.074 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:14:07.803 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:14:07.807 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:14:08.075 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:14:08.076 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:14:08.076 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:14:08.077 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:14:08.079 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:14:08.080 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:14:08.080 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:08.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.214+0000 7f1ca9d7d700 1 Processor -- start 2020-11-20T12:14:08.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.214+0000 7f1ca9d7d700 1 -- start start 2020-11-20T12:14:08.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca9d7d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1ca40a1a40 0x7f1ca40aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:08.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca9d7d700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca4067470 con 0x7f1ca40666d0 2020-11-20T12:14:08.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca9d7d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca4067710 con 0x7f1ca41382c0 2020-11-20T12:14:08.220 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca9d7d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40a9b30 con 0x7f1ca40a1a40 2020-11-20T12:14:08.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca2ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1ca40666d0 0x7f1ca4137bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44666/0 (socket says 172.21.15.44:44666) 2020-11-20T12:14:08.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca2ffd700 1 -- 172.21.15.44:0/3380536294 learned_addr learned my addr 172.21.15.44:0/3380536294 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:08.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca37fe700 1 --2- 172.21.15.44:0/3380536294 >> v2:172.21.15.79:3300/0 conn(0x7f1ca40a1a40 0x7f1ca40aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:08.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca27fc700 1 -- 172.21.15.44:0/3380536294 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446565665 0 0) 0x7f1ca4067710 con 0x7f1ca41382c0 2020-11-20T12:14:08.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.215+0000 7f1ca27fc700 1 -- 172.21.15.44:0/3380536294 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c90003240 con 0x7f1ca41382c0 2020-11-20T12:14:08.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca37fe700 1 -- 172.21.15.44:0/3380536294 >> v1:172.21.15.79:6789/0 conn(0x7f1ca41382c0 legacy=0x7f1ca41454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:08.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca37fe700 1 -- 172.21.15.44:0/3380536294 >> v1:172.21.15.44:6789/0 conn(0x7f1ca40666d0 legacy=0x7f1ca4137bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:08.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca37fe700 1 -- 172.21.15.44:0/3380536294 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40679b0 con 0x7f1ca40a1a40 2020-11-20T12:14:08.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca37fe700 1 --2- 172.21.15.44:0/3380536294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40a1a40 0x7f1ca40aad50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f1c9400b9a0 tx=0x7f1c94006df0).ready entity=mon.1 client_cookie=dd179cd27e167f9f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca27fc700 1 -- 172.21.15.44:0/3380536294 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940097b0 con 0x7f1ca40a1a40 2020-11-20T12:14:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca27fc700 1 -- 172.21.15.44:0/3380536294 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94009910 con 0x7f1ca40a1a40 2020-11-20T12:14:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca27fc700 1 -- 172.21.15.44:0/3380536294 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94009c90 con 0x7f1ca40a1a40 2020-11-20T12:14:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/3380536294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40a1a40 msgr2=0x7f1ca40aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:08.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/3380536294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40a1a40 0x7f1ca40aad50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f1c9400b9a0 tx=0x7f1c94006df0).stop 2020-11-20T12:14:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/3380536294 shutdown_connections 2020-11-20T12:14:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/3380536294 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40a1a40 0x7f1ca40aad50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/3380536294 >> 172.21.15.44:0/3380536294 conn(0x7f1ca40af030 msgr2=0x7f1ca40af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/3380536294 shutdown_connections 2020-11-20T12:14:08.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.216+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/3380536294 wait complete. 2020-11-20T12:14:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 Processor -- start 2020-11-20T12:14:08.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 -- start start 2020-11-20T12:14:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40666d0 0x7f1ca41371b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ca40a1a40 0x7f1ca40621b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 0x7f1ca4135730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca413f120 con 0x7f1ca41382c0 2020-11-20T12:14:08.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca413f380 con 0x7f1ca40666d0 2020-11-20T12:14:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca9d7d700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca413f620 con 0x7f1ca40a1a40 2020-11-20T12:14:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 0x7f1ca4135730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 0x7f1ca4135730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40928/0 (socket says 172.21.15.44:40928) 2020-11-20T12:14:08.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca3fff700 1 -- 172.21.15.44:0/661886736 learned_addr learned my addr 172.21.15.44:0/661886736 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca2ffd700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ca40a1a40 0x7f1ca40621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca37fe700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40666d0 0x7f1ca41371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca3fff700 1 -- 172.21.15.44:0/661886736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 msgr2=0x7f1ca4135730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:14:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca3fff700 1 -- 172.21.15.44:0/661886736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 msgr2=0x7f1ca4135730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:14:08.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.218+0000 7f1ca3fff700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 0x7f1ca4135730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:08.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca3fff700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 0x7f1ca4135730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:08.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca2ffd700 1 -- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40666d0 msgr2=0x7f1ca41371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:08.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca2ffd700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40666d0 0x7f1ca41371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:08.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca2ffd700 1 -- 172.21.15.44:0/661886736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 msgr2=0x7f1ca4135730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:08.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca2ffd700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 0x7f1ca4135730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:08.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca2ffd700 1 -- 172.21.15.44:0/661886736 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4062c10 con 0x7f1ca40a1a40 2020-11-20T12:14:08.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca37fe700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40666d0 0x7f1ca41371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:08.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca2ffd700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ca40a1a40 0x7f1ca40621b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1c9800c710 tx=0x7f1c98007b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:08.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1c837fe700 1 -- 172.21.15.44:0/661886736 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980072e0 con 0x7f1ca40a1a40 2020-11-20T12:14:08.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1c837fe700 1 -- 172.21.15.44:0/661886736 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98007440 con 0x7f1ca40a1a40 2020-11-20T12:14:08.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca40ad350 con 0x7f1ca40a1a40 2020-11-20T12:14:08.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1c837fe700 1 -- 172.21.15.44:0/661886736 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98005800 con 0x7f1ca40a1a40 2020-11-20T12:14:08.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.219+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca40626a0 con 0x7f1ca40a1a40 2020-11-20T12:14:08.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.220+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca405c2a0 con 0x7f1ca40a1a40 2020-11-20T12:14:08.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.220+0000 7f1c837fe700 1 -- 172.21.15.44:0/661886736 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1c9801a060 con 0x7f1ca40a1a40 2020-11-20T12:14:08.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.221+0000 7f1c837fe700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c7c056d20 0x7f1c7c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:08.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.221+0000 7f1c837fe700 1 -- 172.21.15.44:0/661886736 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1c9806f7f0 con 0x7f1ca40a1a40 2020-11-20T12:14:08.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.221+0000 7f1ca37fe700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c7c056d20 0x7f1c7c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:08.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.221+0000 7f1ca37fe700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c7c056d20 0x7f1c7c0591e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f1c94006e30 tx=0x7f1c94015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:08.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.225+0000 7f1c837fe700 1 -- 172.21.15.44:0/661886736 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1c98044ac0 con 0x7f1ca40a1a40 2020-11-20T12:14:08.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.497+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1ca4067230 con 0x7f1ca40a1a40 2020-11-20T12:14:08.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.498+0000 7f1c837fe700 1 -- 172.21.15.44:0/661886736 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f1c98018020 con 0x7f1ca40a1a40 2020-11-20T12:14:08.501 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:08.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c7c056d20 msgr2=0x7f1c7c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:08.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c7c056d20 0x7f1c7c0591e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f1c94006e30 tx=0x7f1c94015000).stop 2020-11-20T12:14:08.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ca40a1a40 msgr2=0x7f1ca40621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:08.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ca40a1a40 0x7f1ca40621b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1c9800c710 tx=0x7f1c98007b80).stop 2020-11-20T12:14:08.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 shutdown_connections 2020-11-20T12:14:08.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1c7c056d20 0x7f1c7c0591e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:08.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ca41382c0 0x7f1ca4135730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:08.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1ca40666d0 0x7f1ca41371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:08.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 --2- 172.21.15.44:0/661886736 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1ca40a1a40 0x7f1ca40621b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:08.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 >> 172.21.15.44:0/661886736 conn(0x7f1ca40af030 msgr2=0x7f1ca40af410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:08.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 shutdown_connections 2020-11-20T12:14:08.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:08.499+0000 7f1ca9d7d700 1 -- 172.21.15.44:0/661886736 wait complete. 2020-11-20T12:14:11.515 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.649+0000 7f182b9ce700 1 Processor -- start 2020-11-20T12:14:11.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.649+0000 7f182b9ce700 1 -- start start 2020-11-20T12:14:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f182b9ce700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 0x7f18240a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f182b9ce700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240a9770 con 0x7f18240aa020 2020-11-20T12:14:11.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f182b9ce700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240a6da0 con 0x7f182413ac60 2020-11-20T12:14:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f182b9ce700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240a56c0 con 0x7f18240aa400 2020-11-20T12:14:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1828f69700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 0x7f18240a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1828f69700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 0x7f18240a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40948/0 (socket says 172.21.15.44:40948) 2020-11-20T12:14:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1828f69700 1 -- 172.21.15.44:0/886666669 learned_addr learned my addr 172.21.15.44:0/886666669 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f182976a700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f18240aa020 0x7f18240a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44694/0 (socket says 172.21.15.44:44694) 2020-11-20T12:14:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1828f69700 1 -- 172.21.15.44:0/886666669 >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 msgr2=0x7f18240a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:14:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1828f69700 1 -- 172.21.15.44:0/886666669 >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 msgr2=0x7f18240a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1828f69700 1 --2- 172.21.15.44:0/886666669 >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 0x7f18240a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1828f69700 1 --2- 172.21.15.44:0/886666669 >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 0x7f18240a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:11.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190704019 0 0) 0x7f18240a6da0 con 0x7f182413ac60 2020-11-20T12:14:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.650+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f180c003240 con 0x7f182413ac60 2020-11-20T12:14:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2965022394 0 0) 0x7f180c003240 con 0x7f182413ac60 2020-11-20T12:14:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 msgr2=0x7f18240a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:14:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 --2- 172.21.15.44:0/886666669 >> v2:172.21.15.44:3300/0 conn(0x7f18240aa400 0x7f18240a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 >> v1:172.21.15.44:6789/0 conn(0x7f18240aa020 legacy=0x7f18240a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240a7040 con 0x7f182413ac60 2020-11-20T12:14:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f181c001860 con 0x7f182413ac60 2020-11-20T12:14:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 >> v1:172.21.15.79:6789/0 conn(0x7f182413ac60 legacy=0x7f18240a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c003730 0x7f180c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f180c0044e0 0x7f180c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f180c004e50 0x7f180c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240a56c0 con 0x7f180c0044e0 2020-11-20T12:14:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c004260 con 0x7f180c003730 2020-11-20T12:14:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c00bc20 con 0x7f180c004e50 2020-11-20T12:14:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1828f69700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f180c0044e0 0x7f180c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f182976a700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c003730 0x7f180c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1829f6b700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f180c004e50 0x7f180c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1828f69700 1 -- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f180c004e50 msgr2=0x7f180c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1828f69700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f180c004e50 0x7f180c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1828f69700 1 -- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c003730 msgr2=0x7f180c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1828f69700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c003730 0x7f180c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1828f69700 1 -- 172.21.15.44:0/886666669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c003240 con 0x7f180c0044e0 2020-11-20T12:14:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1829f6b700 1 -- 172.21.15.44:0/886666669 reap_dead start 2020-11-20T12:14:11.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.651+0000 7f1828f69700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f180c0044e0 0x7f180c004960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f181400de10 tx=0x7f181400d430).ready entity=mon.0 client_cookie=fd1ba99cd1ce3425 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400fde0 con 0x7f180c0044e0 2020-11-20T12:14:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1814011450 con 0x7f180c0044e0 2020-11-20T12:14:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f1813fff700 1 -- 172.21.15.44:0/886666669 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18140117f0 con 0x7f180c0044e0 2020-11-20T12:14:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f182b9ce700 1 -- 172.21.15.44:0/886666669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f180c0044e0 msgr2=0x7f180c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:11.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f182b9ce700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f180c0044e0 0x7f180c004960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f181400de10 tx=0x7f181400d430).stop 2020-11-20T12:14:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f182b9ce700 1 -- 172.21.15.44:0/886666669 shutdown_connections 2020-11-20T12:14:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f182b9ce700 1 --2- 172.21.15.44:0/886666669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f180c0044e0 0x7f180c004960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f182b9ce700 1 -- 172.21.15.44:0/886666669 >> 172.21.15.44:0/886666669 conn(0x7f1824062690 msgr2=0x7f18240a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f182b9ce700 1 -- 172.21.15.44:0/886666669 shutdown_connections 2020-11-20T12:14:11.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.652+0000 7f182b9ce700 1 -- 172.21.15.44:0/886666669 wait complete. 2020-11-20T12:14:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.653+0000 7f182b9ce700 1 Processor -- start 2020-11-20T12:14:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.653+0000 7f182b9ce700 1 -- start start 2020-11-20T12:14:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f182b9ce700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c0044e0 0x7f1824066760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f182b9ce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 0x7f18240ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f182b9ce700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f182414e390 0x7f18240acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f182b9ce700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240add40 con 0x7f18240650a0 2020-11-20T12:14:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f182b9ce700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182414d780 con 0x7f180c0044e0 2020-11-20T12:14:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f182b9ce700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182414d9b0 con 0x7f182414e390 2020-11-20T12:14:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1828f69700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 0x7f18240ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1828f69700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 0x7f18240ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40962/0 (socket says 172.21.15.44:40962) 2020-11-20T12:14:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1828f69700 1 -- 172.21.15.44:0/1664862579 learned_addr learned my addr 172.21.15.44:0/1664862579 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f182976a700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c0044e0 0x7f1824066760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1829f6b700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f182414e390 0x7f18240acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1828f69700 1 -- 172.21.15.44:0/1664862579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 msgr2=0x7f18240ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:14:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1828f69700 1 -- 172.21.15.44:0/1664862579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 msgr2=0x7f18240ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:14:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1828f69700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 0x7f18240ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.654+0000 7f1828f69700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 0x7f18240ac5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182976a700 1 -- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f182414e390 msgr2=0x7f18240acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182976a700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f182414e390 0x7f18240acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182976a700 1 -- 172.21.15.44:0/1664862579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 msgr2=0x7f18240ac5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182976a700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 0x7f18240ac5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182976a700 1 -- 172.21.15.44:0/1664862579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182414dc50 con 0x7f180c0044e0 2020-11-20T12:14:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f1829f6b700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f182414e390 0x7f18240acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182976a700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c0044e0 0x7f1824066760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1820001570 tx=0x7f1820007b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f18117fa700 1 -- 172.21.15.44:0/1664862579 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820011580 con 0x7f180c0044e0 2020-11-20T12:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f18117fa700 1 -- 172.21.15.44:0/1664862579 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18200116e0 con 0x7f180c0044e0 2020-11-20T12:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182414def0 con 0x7f180c0044e0 2020-11-20T12:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f18117fa700 1 -- 172.21.15.44:0/1664862579 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820011a60 con 0x7f180c0044e0 2020-11-20T12:14:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.655+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18241445d0 con 0x7f180c0044e0 2020-11-20T12:14:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.656+0000 7f18117fa700 1 -- 172.21.15.44:0/1664862579 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f182001b030 con 0x7f180c0044e0 2020-11-20T12:14:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.656+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182414e080 con 0x7f180c0044e0 2020-11-20T12:14:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.657+0000 7f18117fa700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17fc056d50 0x7f17fc059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.657+0000 7f18117fa700 1 -- 172.21.15.44:0/1664862579 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f182006f8c0 con 0x7f180c0044e0 2020-11-20T12:14:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.657+0000 7f1828f69700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17fc056d50 0x7f17fc059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.657+0000 7f1828f69700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17fc056d50 0x7f17fc059210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f181400d520 tx=0x7f181400d430).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.661+0000 7f18117fa700 1 -- 172.21.15.44:0/1664862579 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1820044a80 con 0x7f180c0044e0 2020-11-20T12:14:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.953+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f18240a9770 con 0x7f180c0044e0 2020-11-20T12:14:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.953+0000 7f18117fa700 1 -- 172.21.15.44:0/1664862579 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f18240a9770 con 0x7f180c0044e0 2020-11-20T12:14:11.957 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:11.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17fc056d50 msgr2=0x7f17fc059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17fc056d50 0x7f17fc059210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f181400d520 tx=0x7f181400d430).stop 2020-11-20T12:14:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c0044e0 msgr2=0x7f1824066760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c0044e0 0x7f1824066760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1820001570 tx=0x7f1820007b80).stop 2020-11-20T12:14:11.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 shutdown_connections 2020-11-20T12:14:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f17fc056d50 0x7f17fc059210 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f182414e390 0x7f18240acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f180c0044e0 0x7f1824066760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 --2- 172.21.15.44:0/1664862579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18240650a0 0x7f18240ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 >> 172.21.15.44:0/1664862579 conn(0x7f1824062690 msgr2=0x7f1824060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:11.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 shutdown_connections 2020-11-20T12:14:11.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:11.955+0000 7f182b9ce700 1 -- 172.21.15.44:0/1664862579 wait complete. 2020-11-20T12:14:14.971 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:15.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.112+0000 7fe6675e3700 1 Processor -- start 2020-11-20T12:14:15.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.112+0000 7fe6675e3700 1 -- start start 2020-11-20T12:14:15.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe6675e3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe66013a990 0x7fe66013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:15.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe6675e3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 0x7fe660141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:15.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe6675e3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6600a6260 con 0x7fe660060330 2020-11-20T12:14:15.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe6675e3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6600a52e0 con 0x7fe66013f800 2020-11-20T12:14:15.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe6675e3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6600a6490 con 0x7fe66013a990 2020-11-20T12:14:15.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe665b80700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 0x7fe660141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:15.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe665b80700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 0x7fe660141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40974/0 (socket says 172.21.15.44:40974) 2020-11-20T12:14:15.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe665b80700 1 -- 172.21.15.44:0/3574763135 learned_addr learned my addr 172.21.15.44:0/3574763135 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:15.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe665b80700 1 -- 172.21.15.44:0/3574763135 >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 msgr2=0x7fe660141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:14:15.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe665b80700 1 -- 172.21.15.44:0/3574763135 >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 msgr2=0x7fe660141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:15.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe665b80700 1 --2- 172.21.15.44:0/3574763135 >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 0x7fe660141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe665b80700 1 --2- 172.21.15.44:0/3574763135 >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 0x7fe660141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.113+0000 7fe664b7e700 1 --2- 172.21.15.44:0/3574763135 >> v2:172.21.15.79:3300/0 conn(0x7fe66013a990 0x7fe66013ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe664b7e700 1 -- 172.21.15.44:0/3574763135 >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 msgr2=0x7fe660141bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe664b7e700 1 --2- 172.21.15.44:0/3574763135 >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 0x7fe660141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe664b7e700 1 -- 172.21.15.44:0/3574763135 >> v1:172.21.15.79:6789/0 conn(0x7fe660060330 legacy=0x7fe6600a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe664b7e700 1 -- 172.21.15.44:0/3574763135 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600a6730 con 0x7fe66013a990 2020-11-20T12:14:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe64ffff700 1 -- 172.21.15.44:0/3574763135 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110041979 0 0) 0x7fe6600a6260 con 0x7fe660060330 2020-11-20T12:14:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe664b7e700 1 --2- 172.21.15.44:0/3574763135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe66013ae20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe6600a3a60 tx=0x7fe650006df0).ready entity=mon.1 client_cookie=db71f08e3d0be647 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe64ffff700 1 -- 172.21.15.44:0/3574763135 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6500097b0 con 0x7fe66013a990 2020-11-20T12:14:15.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe64ffff700 1 -- 172.21.15.44:0/3574763135 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650009910 con 0x7fe66013a990 2020-11-20T12:14:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.114+0000 7fe64ffff700 1 -- 172.21.15.44:0/3574763135 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650009c90 con 0x7fe66013a990 2020-11-20T12:14:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 -- 172.21.15.44:0/3574763135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 msgr2=0x7fe66013ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 --2- 172.21.15.44:0/3574763135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe66013ae20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe6600a3a60 tx=0x7fe650006df0).stop 2020-11-20T12:14:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 -- 172.21.15.44:0/3574763135 shutdown_connections 2020-11-20T12:14:15.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 --2- 172.21.15.44:0/3574763135 >> v2:172.21.15.44:3300/0 conn(0x7fe66013f800 0x7fe660141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 --2- 172.21.15.44:0/3574763135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe66013ae20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 -- 172.21.15.44:0/3574763135 >> 172.21.15.44:0/3574763135 conn(0x7fe660062690 msgr2=0x7fe6600a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 -- 172.21.15.44:0/3574763135 shutdown_connections 2020-11-20T12:14:15.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.115+0000 7fe6675e3700 1 -- 172.21.15.44:0/3574763135 wait complete. 2020-11-20T12:14:15.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 Processor -- start 2020-11-20T12:14:15.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 -- start start 2020-11-20T12:14:15.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 0x7fe660130190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:15.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe6601376d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:15.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe66013f800 0x7fe6600ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:15.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660147da0 con 0x7fe660060330 2020-11-20T12:14:15.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660148d20 con 0x7fe66013a990 2020-11-20T12:14:15.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.116+0000 7fe6675e3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe660137bc0 con 0x7fe66013f800 2020-11-20T12:14:15.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe66537f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 0x7fe660130190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe66537f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 0x7fe660130190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40976/0 (socket says 172.21.15.44:40976) 2020-11-20T12:14:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe66537f700 1 -- 172.21.15.44:0/2417607230 learned_addr learned my addr 172.21.15.44:0/2417607230 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe66537f700 1 -- 172.21.15.44:0/2417607230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 msgr2=0x7fe660130190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:14:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe664b7e700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe6601376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:15.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe66537f700 1 -- 172.21.15.44:0/2417607230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 msgr2=0x7fe660130190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:14:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe66537f700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 0x7fe660130190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe665b80700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe66013f800 0x7fe6600ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe66537f700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 0x7fe660130190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe664b7e700 1 -- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe66013f800 msgr2=0x7fe6600ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe664b7e700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe66013f800 0x7fe6600ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe664b7e700 1 -- 172.21.15.44:0/2417607230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 msgr2=0x7fe660130190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:15.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe664b7e700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 0x7fe660130190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe664b7e700 1 -- 172.21.15.44:0/2417607230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600ad460 con 0x7fe66013a990 2020-11-20T12:14:15.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe664b7e700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe6601376d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fe6600adec0 tx=0x7fe650015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:15.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe64d7fa700 1 -- 172.21.15.44:0/2417607230 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65001e460 con 0x7fe66013a990 2020-11-20T12:14:15.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.117+0000 7fe64d7fa700 1 -- 172.21.15.44:0/2417607230 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65001e5c0 con 0x7fe66013a990 2020-11-20T12:14:15.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.118+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6600ad700 con 0x7fe66013a990 2020-11-20T12:14:15.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.118+0000 7fe64d7fa700 1 -- 172.21.15.44:0/2417607230 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65001e960 con 0x7fe66013a990 2020-11-20T12:14:15.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.118+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe660130b30 con 0x7fe66013a990 2020-11-20T12:14:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.119+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6600adb60 con 0x7fe66013a990 2020-11-20T12:14:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.119+0000 7fe64d7fa700 1 -- 172.21.15.44:0/2417607230 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe65001a040 con 0x7fe66013a990 2020-11-20T12:14:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.119+0000 7fe64d7fa700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe638056d20 0x7fe6380591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.119+0000 7fe64d7fa700 1 -- 172.21.15.44:0/2417607230 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe650071dc0 con 0x7fe66013a990 2020-11-20T12:14:15.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.119+0000 7fe66537f700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe638056d20 0x7fe6380591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.120+0000 7fe66537f700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe638056d20 0x7fe6380591e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fe65c007fd0 tx=0x7fe65c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:15.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.123+0000 7fe64d7fa700 1 -- 172.21.15.44:0/2417607230 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe650047070 con 0x7fe66013a990 2020-11-20T12:14:15.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.388+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe660061650 con 0x7fe66013a990 2020-11-20T12:14:15.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.388+0000 7fe64d7fa700 1 -- 172.21.15.44:0/2417607230 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fe65004a690 con 0x7fe66013a990 2020-11-20T12:14:15.392 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe638056d20 msgr2=0x7fe6380591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe638056d20 0x7fe6380591e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fe65c007fd0 tx=0x7fe65c009040).stop 2020-11-20T12:14:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 msgr2=0x7fe6601376d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:15.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe6601376d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fe6600adec0 tx=0x7fe650015000).stop 2020-11-20T12:14:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 shutdown_connections 2020-11-20T12:14:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe638056d20 0x7fe6380591e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe66013f800 0x7fe6600ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660060330 0x7fe660130190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 --2- 172.21.15.44:0/2417607230 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe66013a990 0x7fe6601376d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 >> 172.21.15.44:0/2417607230 conn(0x7fe660062690 msgr2=0x7fe6600672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 shutdown_connections 2020-11-20T12:14:15.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:15.390+0000 7fe6675e3700 1 -- 172.21.15.44:0/2417607230 wait complete. 2020-11-20T12:14:18.406 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.538+0000 7fea772c2700 1 Processor -- start 2020-11-20T12:14:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea772c2700 1 -- start start 2020-11-20T12:14:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea772c2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea700a9130 0x7fea700adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:18.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea772c2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fea7013b770 0x7fea70145420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:18.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea772c2700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea700ab630 con 0x7fea700a9510 2020-11-20T12:14:18.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea772c2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea700ab110 con 0x7fea7013b770 2020-11-20T12:14:18.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea772c2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700ab3b0 con 0x7fea700a9130 2020-11-20T12:14:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea7485d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fea700a9510 0x7fea7013b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44732/0 (socket says 172.21.15.44:44732) 2020-11-20T12:14:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.539+0000 7fea7485d700 1 -- 172.21.15.44:0/3096423645 learned_addr learned my addr 172.21.15.44:0/3096423645 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7585f700 1 --2- 172.21.15.44:0/3096423645 >> v2:172.21.15.79:3300/0 conn(0x7fea7013b770 0x7fea70145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:18.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7505e700 1 --2- 172.21.15.44:0/3096423645 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea700a9130 0x7fea700adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7505e700 1 -- 172.21.15.44:0/3096423645 >> v2:172.21.15.79:3300/0 conn(0x7fea7013b770 msgr2=0x7fea70145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7505e700 1 --2- 172.21.15.44:0/3096423645 >> v2:172.21.15.79:3300/0 conn(0x7fea7013b770 0x7fea70145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7505e700 1 -- 172.21.15.44:0/3096423645 >> v1:172.21.15.44:6789/0 conn(0x7fea700a9510 legacy=0x7fea7013b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7505e700 1 -- 172.21.15.44:0/3096423645 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea700ad7f0 con 0x7fea700a9130 2020-11-20T12:14:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7585f700 1 --2- 172.21.15.44:0/3096423645 >> v2:172.21.15.79:3300/0 conn(0x7fea7013b770 0x7fea70145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea7505e700 1 --2- 172.21.15.44:0/3096423645 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea700a9130 0x7fea700adc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fea6c00b9a0 tx=0x7fea6c006df0).ready entity=mon.2 client_cookie=6262f44d139ff42 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea5ffff700 1 -- 172.21.15.44:0/3096423645 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0097b0 con 0x7fea700a9130 2020-11-20T12:14:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea5ffff700 1 -- 172.21.15.44:0/3096423645 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c009910 con 0x7fea700a9130 2020-11-20T12:14:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.540+0000 7fea5ffff700 1 -- 172.21.15.44:0/3096423645 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c009c90 con 0x7fea700a9130 2020-11-20T12:14:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 -- 172.21.15.44:0/3096423645 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea700a9130 msgr2=0x7fea700adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 --2- 172.21.15.44:0/3096423645 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea700a9130 0x7fea700adc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fea6c00b9a0 tx=0x7fea6c006df0).stop 2020-11-20T12:14:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 -- 172.21.15.44:0/3096423645 shutdown_connections 2020-11-20T12:14:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 --2- 172.21.15.44:0/3096423645 >> v2:172.21.15.79:3300/0 conn(0x7fea7013b770 0x7fea70145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 --2- 172.21.15.44:0/3096423645 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea700a9130 0x7fea700adc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 -- 172.21.15.44:0/3096423645 >> 172.21.15.44:0/3096423645 conn(0x7fea70060170 msgr2=0x7fea70060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 -- 172.21.15.44:0/3096423645 shutdown_connections 2020-11-20T12:14:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.541+0000 7fea772c2700 1 -- 172.21.15.44:0/3096423645 wait complete. 2020-11-20T12:14:18.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.542+0000 7fea772c2700 1 Processor -- start 2020-11-20T12:14:18.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.542+0000 7fea772c2700 1 -- start start 2020-11-20T12:14:18.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea772c2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 0x7fea700b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:18.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea772c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea700a9510 0x7fea70138bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea772c2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea7013b770 0x7fea7012e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea772c2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea70134370 con 0x7fea700a9130 2020-11-20T12:14:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea772c2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea70134610 con 0x7fea700a9510 2020-11-20T12:14:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea772c2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea701348b0 con 0x7fea7013b770 2020-11-20T12:14:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7505e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 0x7fea700b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7505e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 0x7fea700b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40988/0 (socket says 172.21.15.44:40988) 2020-11-20T12:14:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7505e700 1 -- 172.21.15.44:0/1810025991 learned_addr learned my addr 172.21.15.44:0/1810025991 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7485d700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea700a9510 0x7fea70138bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7505e700 1 -- 172.21.15.44:0/1810025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 msgr2=0x7fea700b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:14:18.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7505e700 1 -- 172.21.15.44:0/1810025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 msgr2=0x7fea700b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:14:18.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7585f700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea7013b770 0x7fea7012e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:18.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7505e700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 0x7fea700b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:18.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7505e700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 0x7fea700b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:18.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7585f700 1 -- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea700a9510 msgr2=0x7fea70138bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:18.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7585f700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea700a9510 0x7fea70138bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7585f700 1 -- 172.21.15.44:0/1810025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 msgr2=0x7fea700b18c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:18.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7585f700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 0x7fea700b18c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7585f700 1 -- 172.21.15.44:0/1810025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea700af0b0 con 0x7fea7013b770 2020-11-20T12:14:18.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7485d700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea700a9510 0x7fea70138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:18.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea7585f700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea7013b770 0x7fea7012e9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fea6800ef20 tx=0x7fea680091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:18.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.543+0000 7fea5d7fa700 1 -- 172.21.15.44:0/1810025991 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6800cbc0 con 0x7fea7013b770 2020-11-20T12:14:18.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.544+0000 7fea5d7fa700 1 -- 172.21.15.44:0/1810025991 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6800cd20 con 0x7fea7013b770 2020-11-20T12:14:18.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.544+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7012eea0 con 0x7fea7013b770 2020-11-20T12:14:18.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.544+0000 7fea5d7fa700 1 -- 172.21.15.44:0/1810025991 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68012620 con 0x7fea7013b770 2020-11-20T12:14:18.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.544+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea700af520 con 0x7fea7013b770 2020-11-20T12:14:18.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.545+0000 7fea5d7fa700 1 -- 172.21.15.44:0/1810025991 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fea68022070 con 0x7fea7013b770 2020-11-20T12:14:18.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.545+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea700af2a0 con 0x7fea7013b770 2020-11-20T12:14:18.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.545+0000 7fea5d7fa700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea48056d20 0x7fea480591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:18.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.545+0000 7fea5d7fa700 1 -- 172.21.15.44:0/1810025991 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fea680708b0 con 0x7fea7013b770 2020-11-20T12:14:18.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.545+0000 7fea7505e700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea48056d20 0x7fea480591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:18.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.546+0000 7fea7505e700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea48056d20 0x7fea480591e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fea6c00be60 tx=0x7fea6c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:18.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.549+0000 7fea5d7fa700 1 -- 172.21.15.44:0/1810025991 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fea68045b80 con 0x7fea7013b770 2020-11-20T12:14:18.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.823+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fea700ae180 con 0x7fea7013b770 2020-11-20T12:14:18.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.824+0000 7fea5d7fa700 1 -- 172.21.15.44:0/1810025991 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fea68016080 con 0x7fea7013b770 2020-11-20T12:14:18.829 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":32,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:18.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea48056d20 msgr2=0x7fea480591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:18.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea48056d20 0x7fea480591e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fea6c00be60 tx=0x7fea6c015000).stop 2020-11-20T12:14:18.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea7013b770 msgr2=0x7fea7012e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:18.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea7013b770 0x7fea7012e9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fea6800ef20 tx=0x7fea680091d0).stop 2020-11-20T12:14:18.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 shutdown_connections 2020-11-20T12:14:18.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fea48056d20 0x7fea480591e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fea7013b770 0x7fea7012e9b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea700a9130 0x7fea700b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 --2- 172.21.15.44:0/1810025991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea700a9510 0x7fea70138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 >> 172.21.15.44:0/1810025991 conn(0x7fea70060170 msgr2=0x7fea700acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:18.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 shutdown_connections 2020-11-20T12:14:18.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:18.826+0000 7fea772c2700 1 -- 172.21.15.44:0/1810025991 wait complete. 2020-11-20T12:14:21.842 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:21.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.978+0000 7f364e0b6700 1 Processor -- start 2020-11-20T12:14:21.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.978+0000 7f364e0b6700 1 -- start start 2020-11-20T12:14:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f364e0b6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 0x7f3648066e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f364e0b6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f36480aad50 0x7f36480a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f364e0b6700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480676b0 con 0x7f364813e260 2020-11-20T12:14:21.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f364e0b6700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3648067410 con 0x7f3648061340 2020-11-20T12:14:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f364e0b6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36480a95d0 con 0x7f36480aad50 2020-11-20T12:14:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f36477fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 0x7f3648066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f3646ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f364813e260 0x7f36480a8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44748/0 (socket says 172.21.15.44:44748) 2020-11-20T12:14:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f3646ffd700 1 -- 172.21.15.44:0/2725584367 learned_addr learned my addr 172.21.15.44:0/2725584367 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f36477fe700 1 -- 172.21.15.44:0/2725584367 >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 msgr2=0x7f3648066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:14:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f36477fe700 1 -- 172.21.15.44:0/2725584367 >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 msgr2=0x7f3648066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f36477fe700 1 --2- 172.21.15.44:0/2725584367 >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 0x7f3648066e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f36477fe700 1 --2- 172.21.15.44:0/2725584367 >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 0x7f3648066e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.979+0000 7f3647fff700 1 --2- 172.21.15.44:0/2725584367 >> v2:172.21.15.79:3300/0 conn(0x7f36480aad50 0x7f36480a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:21.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f3647fff700 1 -- 172.21.15.44:0/2725584367 >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 msgr2=0x7f3648066e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:14:21.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f3647fff700 1 --2- 172.21.15.44:0/2725584367 >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 0x7f3648066e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:21.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f3647fff700 1 -- 172.21.15.44:0/2725584367 >> v1:172.21.15.44:6789/0 conn(0x7f364813e260 legacy=0x7f36480a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:21.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f3647fff700 1 -- 172.21.15.44:0/2725584367 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480a9f60 con 0x7f36480aad50 2020-11-20T12:14:21.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f3647fff700 1 --2- 172.21.15.44:0/2725584367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36480aad50 0x7f36480a9a70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f3648067310 tx=0x7f36400091d0).ready entity=mon.1 client_cookie=f43a064dca038933 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:21.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f36467fc700 1 -- 172.21.15.44:0/2725584367 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000bc50 con 0x7f36480aad50 2020-11-20T12:14:21.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f36467fc700 1 -- 172.21.15.44:0/2725584367 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364000bdb0 con 0x7f36480aad50 2020-11-20T12:14:21.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.980+0000 7f36467fc700 1 -- 172.21.15.44:0/2725584367 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640011610 con 0x7f36480aad50 2020-11-20T12:14:21.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 -- 172.21.15.44:0/2725584367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36480aad50 msgr2=0x7f36480a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:21.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 --2- 172.21.15.44:0/2725584367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36480aad50 0x7f36480a9a70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f3648067310 tx=0x7f36400091d0).stop 2020-11-20T12:14:21.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 -- 172.21.15.44:0/2725584367 shutdown_connections 2020-11-20T12:14:21.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 --2- 172.21.15.44:0/2725584367 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36480aad50 0x7f36480a9a70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:21.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 --2- 172.21.15.44:0/2725584367 >> v2:172.21.15.44:3300/0 conn(0x7f3648061340 0x7f3648066e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:21.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 -- 172.21.15.44:0/2725584367 >> 172.21.15.44:0/2725584367 conn(0x7f36480ae170 msgr2=0x7f36480aa170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 -- 172.21.15.44:0/2725584367 shutdown_connections 2020-11-20T12:14:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.981+0000 7f364e0b6700 1 -- 172.21.15.44:0/2725584367 wait complete. 2020-11-20T12:14:21.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 Processor -- start 2020-11-20T12:14:21.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 -- start start 2020-11-20T12:14:21.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3648061340 0x7f364809e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:21.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 0x7f36480a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:21.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f364813e260 0x7f364814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:21.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364809e7b0 con 0x7f36480aad50 2020-11-20T12:14:21.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364809eca0 con 0x7f364813e260 2020-11-20T12:14:21.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f364e0b6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364809ef00 con 0x7f3648061340 2020-11-20T12:14:21.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.983+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 0x7f36480a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:21.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 0x7f36480a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41008/0 (socket says 172.21.15.44:41008) 2020-11-20T12:14:21.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3646ffd700 1 -- 172.21.15.44:0/3237424886 learned_addr learned my addr 172.21.15.44:0/3237424886 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3647fff700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f364813e260 0x7f364814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3646ffd700 1 -- 172.21.15.44:0/3237424886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 msgr2=0x7f36480a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3646ffd700 1 -- 172.21.15.44:0/3237424886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 msgr2=0x7f36480a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f36477fe700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3648061340 0x7f364809e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:21.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3646ffd700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 0x7f36480a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3646ffd700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 0x7f36480a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3647fff700 1 -- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3648061340 msgr2=0x7f364809e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3647fff700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3648061340 0x7f364809e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:21.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3647fff700 1 -- 172.21.15.44:0/3237424886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 msgr2=0x7f36480a0c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:21.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3647fff700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 0x7f36480a0c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:21.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3647fff700 1 -- 172.21.15.44:0/3237424886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364809da00 con 0x7f364813e260 2020-11-20T12:14:21.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f3647fff700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f364813e260 0x7f364814d230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f3648067310 tx=0x7f3640016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f36277fe700 1 -- 172.21.15.44:0/3237424886 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000bc50 con 0x7f364813e260 2020-11-20T12:14:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f36277fe700 1 -- 172.21.15.44:0/3237424886 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640010b90 con 0x7f364813e260 2020-11-20T12:14:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364813b740 con 0x7f364813e260 2020-11-20T12:14:22.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f36277fe700 1 -- 172.21.15.44:0/3237424886 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364001d480 con 0x7f364813e260 2020-11-20T12:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.984+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364809f5e0 con 0x7f364813e260 2020-11-20T12:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.985+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364809f0c0 con 0x7f364813e260 2020-11-20T12:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.986+0000 7f36277fe700 1 -- 172.21.15.44:0/3237424886 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3640024030 con 0x7f364813e260 2020-11-20T12:14:22.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.986+0000 7f36277fe700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3620056d20 0x7f36200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.986+0000 7f36277fe700 1 -- 172.21.15.44:0/3237424886 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3640071ec0 con 0x7f364813e260 2020-11-20T12:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.986+0000 7f36477fe700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3620056d20 0x7f36200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:22.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.987+0000 7f36477fe700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3620056d20 0x7f36200591e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f364809d990 tx=0x7f3638008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:22.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:21.990+0000 7f36277fe700 1 -- 172.21.15.44:0/3237424886 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3640047210 con 0x7f364813e260 2020-11-20T12:14:22.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.255+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f36480ae580 con 0x7f364813e260 2020-11-20T12:14:22.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.256+0000 7f36277fe700 1 -- 172.21.15.44:0/3237424886 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f3640022020 con 0x7f364813e260 2020-11-20T12:14:22.259 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":35,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:22.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3620056d20 msgr2=0x7f36200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:22.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3620056d20 0x7f36200591e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f364809d990 tx=0x7f3638008040).stop 2020-11-20T12:14:22.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f364813e260 msgr2=0x7f364814d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:22.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f364813e260 0x7f364814d230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f3648067310 tx=0x7f3640016000).stop 2020-11-20T12:14:22.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 shutdown_connections 2020-11-20T12:14:22.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3620056d20 0x7f36200591e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:22.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f364813e260 0x7f364814d230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:22.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3648061340 0x7f364809e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:22.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 --2- 172.21.15.44:0/3237424886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36480aad50 0x7f36480a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:22.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 >> 172.21.15.44:0/3237424886 conn(0x7f36480ae170 msgr2=0x7f36480a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:22.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 shutdown_connections 2020-11-20T12:14:22.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:22.258+0000 7f364e0b6700 1 -- 172.21.15.44:0/3237424886 wait complete. 2020-11-20T12:14:25.274 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:26.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.863+0000 7f127b10e700 1 Processor -- start 2020-11-20T12:14:26.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.863+0000 7f127b10e700 1 -- start start 2020-11-20T12:14:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.864+0000 7f127b10e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740a9740 0x7f1274137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:26.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.864+0000 7f127b10e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12740a9000 con 0x7f12740b17c0 2020-11-20T12:14:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.864+0000 7f127b10e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12740a92a0 con 0x7f12740b1ba0 2020-11-20T12:14:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.864+0000 7f127b10e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12740a4c30 con 0x7f12740a9740 2020-11-20T12:14:26.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.864+0000 7f127a10c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f12740b17c0 0x7f127413abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44762/0 (socket says 172.21.15.44:44762) 2020-11-20T12:14:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.864+0000 7f127a10c700 1 -- 172.21.15.44:0/1316620477 learned_addr learned my addr 172.21.15.44:0/1316620477 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:26.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.864+0000 7f127a90d700 1 --2- 172.21.15.44:0/1316620477 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740a9740 0x7f1274137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127910a700 1 -- 172.21.15.44:0/1316620477 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264390421 0 0) 0x7f12740a92a0 con 0x7f12740b1ba0 2020-11-20T12:14:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127910a700 1 -- 172.21.15.44:0/1316620477 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f125c003240 con 0x7f12740b1ba0 2020-11-20T12:14:26.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127a90d700 1 -- 172.21.15.44:0/1316620477 >> v1:172.21.15.79:6789/0 conn(0x7f12740b1ba0 legacy=0x7f12740aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127a90d700 1 -- 172.21.15.44:0/1316620477 >> v1:172.21.15.44:6789/0 conn(0x7f12740b17c0 legacy=0x7f127413abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:26.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127a90d700 1 -- 172.21.15.44:0/1316620477 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274063a30 con 0x7f12740a9740 2020-11-20T12:14:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127a90d700 1 --2- 172.21.15.44:0/1316620477 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740a9740 0x7f1274137ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f126c009cc0 tx=0x7f126c0091d0).ready entity=mon.2 client_cookie=8a58a4cf59f82bef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:26.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127910a700 1 -- 172.21.15.44:0/1316620477 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c00bb60 con 0x7f12740a9740 2020-11-20T12:14:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127910a700 1 -- 172.21.15.44:0/1316620477 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c00bcc0 con 0x7f12740a9740 2020-11-20T12:14:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.865+0000 7f127910a700 1 -- 172.21.15.44:0/1316620477 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c011610 con 0x7f12740a9740 2020-11-20T12:14:26.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.866+0000 7f127b10e700 1 -- 172.21.15.44:0/1316620477 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740a9740 msgr2=0x7f1274137ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.866+0000 7f127b10e700 1 --2- 172.21.15.44:0/1316620477 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740a9740 0x7f1274137ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f126c009cc0 tx=0x7f126c0091d0).stop 2020-11-20T12:14:26.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.866+0000 7f127b10e700 1 -- 172.21.15.44:0/1316620477 shutdown_connections 2020-11-20T12:14:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.866+0000 7f127b10e700 1 --2- 172.21.15.44:0/1316620477 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740a9740 0x7f1274137ef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.866+0000 7f127b10e700 1 -- 172.21.15.44:0/1316620477 >> 172.21.15.44:0/1316620477 conn(0x7f1274060c10 msgr2=0x7f12740b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:26.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.866+0000 7f127b10e700 1 -- 172.21.15.44:0/1316620477 shutdown_connections 2020-11-20T12:14:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.866+0000 7f127b10e700 1 -- 172.21.15.44:0/1316620477 wait complete. 2020-11-20T12:14:26.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 Processor -- start 2020-11-20T12:14:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 -- start start 2020-11-20T12:14:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 0x7f127413dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:26.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12740b17c0 0x7f12740a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740b1ba0 0x7f1274137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:26.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12740a7c50 con 0x7f12740a9740 2020-11-20T12:14:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12740a65b0 con 0x7f12740b17c0 2020-11-20T12:14:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127b10e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12740a6850 con 0x7f12740b1ba0 2020-11-20T12:14:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.867+0000 7f127a10c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 0x7f127413dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a10c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 0x7f127413dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41018/0 (socket says 172.21.15.44:41018) 2020-11-20T12:14:26.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a10c700 1 -- 172.21.15.44:0/3775986990 learned_addr learned my addr 172.21.15.44:0/3775986990 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a10c700 1 -- 172.21.15.44:0/3775986990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 msgr2=0x7f127413dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:14:26.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a10c700 1 -- 172.21.15.44:0/3775986990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 msgr2=0x7f127413dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:14:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a10c700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 0x7f127413dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a10c700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 0x7f127413dbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:26.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a90d700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740b1ba0 0x7f1274137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127990b700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12740b17c0 0x7f12740a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:26.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127990b700 1 -- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740b1ba0 msgr2=0x7f1274137500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127990b700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740b1ba0 0x7f1274137500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127990b700 1 -- 172.21.15.44:0/3775986990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 msgr2=0x7f127413dbb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:26.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127990b700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 0x7f127413dbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127990b700 1 -- 172.21.15.44:0/3775986990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12740a6af0 con 0x7f12740b17c0 2020-11-20T12:14:26.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127a90d700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740b1ba0 0x7f1274137500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f127990b700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12740b17c0 0x7f12740a7760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1264007fd0 tx=0x7f1264007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.868+0000 7f12627fc700 1 -- 172.21.15.44:0/3775986990 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126400e490 con 0x7f12740b17c0 2020-11-20T12:14:26.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.869+0000 7f12627fc700 1 -- 172.21.15.44:0/3775986990 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126400e5f0 con 0x7f12740b17c0 2020-11-20T12:14:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.869+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12740a6030 con 0x7f12740b17c0 2020-11-20T12:14:26.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.869+0000 7f12627fc700 1 -- 172.21.15.44:0/3775986990 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126400e970 con 0x7f12740b17c0 2020-11-20T12:14:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.869+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1274137c30 con 0x7f12740b17c0 2020-11-20T12:14:26.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.870+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1274057d80 con 0x7f12740b17c0 2020-11-20T12:14:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.870+0000 7f12627fc700 1 -- 172.21.15.44:0/3775986990 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1264019030 con 0x7f12740b17c0 2020-11-20T12:14:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.870+0000 7f12627fc700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f124c056d20 0x7f124c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:26.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.870+0000 7f12627fc700 1 -- 172.21.15.44:0/3775986990 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f126406f4a0 con 0x7f12740b17c0 2020-11-20T12:14:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.870+0000 7f127a10c700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f124c056d20 0x7f124c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.871+0000 7f127a10c700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f124c056d20 0x7f124c0591e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f1270007a50 tx=0x7f1270009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:26.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:26.874+0000 7f12627fc700 1 -- 172.21.15.44:0/3775986990 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1264043fd0 con 0x7f12740b17c0 2020-11-20T12:14:27.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.146+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f12740446d0 con 0x7f12740b17c0 2020-11-20T12:14:27.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.147+0000 7f12627fc700 1 -- 172.21.15.44:0/3775986990 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f1264044820 con 0x7f12740b17c0 2020-11-20T12:14:27.150 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":40,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:27.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f124c056d20 msgr2=0x7f124c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:27.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f124c056d20 0x7f124c0591e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f1270007a50 tx=0x7f1270009040).stop 2020-11-20T12:14:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12740b17c0 msgr2=0x7f12740a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12740b17c0 0x7f12740a7760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1264007fd0 tx=0x7f1264007a20).stop 2020-11-20T12:14:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 shutdown_connections 2020-11-20T12:14:27.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f124c056d20 0x7f124c0591e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f12740b1ba0 0x7f1274137500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12740a9740 0x7f127413dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 --2- 172.21.15.44:0/3775986990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12740b17c0 0x7f12740a7760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 >> 172.21.15.44:0/3775986990 conn(0x7f1274060c10 msgr2=0x7f12740ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:27.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 shutdown_connections 2020-11-20T12:14:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:27.149+0000 7f127b10e700 1 -- 172.21.15.44:0/3775986990 wait complete. 2020-11-20T12:14:30.164 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:30.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.300+0000 7fd3ae867700 1 Processor -- start 2020-11-20T12:14:30.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.300+0000 7fd3ae867700 1 -- start start 2020-11-20T12:14:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.300+0000 7fd3ae867700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd3a80a86d0 0x7fd3a80a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.300+0000 7fd3ae867700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a81385f0 con 0x7fd3a80ab0e0 2020-11-20T12:14:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.300+0000 7fd3ae867700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a80aca50 con 0x7fd3a80aad00 2020-11-20T12:14:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.300+0000 7fd3ae867700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a813e250 con 0x7fd3a80a86d0 2020-11-20T12:14:30.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3a77fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd3a80ab0e0 0x7fd3a80ac340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44776/0 (socket says 172.21.15.44:44776) 2020-11-20T12:14:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3a77fe700 1 -- 172.21.15.44:0/2382805521 learned_addr learned my addr 172.21.15.44:0/2382805521 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3ace04700 1 --2- 172.21.15.44:0/2382805521 >> v2:172.21.15.79:3300/0 conn(0x7fd3a80a86d0 0x7fd3a80a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3a6ffd700 1 -- 172.21.15.44:0/2382805521 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229004385 0 0) 0x7fd3a80aca50 con 0x7fd3a80aad00 2020-11-20T12:14:30.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3a6ffd700 1 -- 172.21.15.44:0/2382805521 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd394003240 con 0x7fd3a80aad00 2020-11-20T12:14:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3ace04700 1 -- 172.21.15.44:0/2382805521 >> v1:172.21.15.79:6789/0 conn(0x7fd3a80aad00 legacy=0x7fd3a80abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3ace04700 1 -- 172.21.15.44:0/2382805521 >> v1:172.21.15.44:6789/0 conn(0x7fd3a80ab0e0 legacy=0x7fd3a80ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3ace04700 1 -- 172.21.15.44:0/2382805521 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a813e4f0 con 0x7fd3a80a86d0 2020-11-20T12:14:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.301+0000 7fd3ace04700 1 --2- 172.21.15.44:0/2382805521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a94d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd3a0009c90 tx=0x7fd3a00091d0).ready entity=mon.1 client_cookie=303c56a0d1c62251 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:30.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3a6ffd700 1 -- 172.21.15.44:0/2382805521 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000ba70 con 0x7fd3a80a86d0 2020-11-20T12:14:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3a6ffd700 1 -- 172.21.15.44:0/2382805521 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a000bbd0 con 0x7fd3a80a86d0 2020-11-20T12:14:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3a6ffd700 1 -- 172.21.15.44:0/2382805521 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0011400 con 0x7fd3a80a86d0 2020-11-20T12:14:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3ae867700 1 -- 172.21.15.44:0/2382805521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80a86d0 msgr2=0x7fd3a80a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3ae867700 1 --2- 172.21.15.44:0/2382805521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a94d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd3a0009c90 tx=0x7fd3a00091d0).stop 2020-11-20T12:14:30.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3ae867700 1 -- 172.21.15.44:0/2382805521 shutdown_connections 2020-11-20T12:14:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3ae867700 1 --2- 172.21.15.44:0/2382805521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a94d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3ae867700 1 -- 172.21.15.44:0/2382805521 >> 172.21.15.44:0/2382805521 conn(0x7fd3a8060170 msgr2=0x7fd3a8060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3ae867700 1 -- 172.21.15.44:0/2382805521 shutdown_connections 2020-11-20T12:14:30.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.302+0000 7fd3ae867700 1 -- 172.21.15.44:0/2382805521 wait complete. 2020-11-20T12:14:30.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.303+0000 7fd3ae867700 1 Processor -- start 2020-11-20T12:14:30.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.303+0000 7fd3ae867700 1 -- start start 2020-11-20T12:14:30.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3ae867700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:30.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3ae867700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80aad00 0x7fd3a813a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:30.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3ae867700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd3a80ab0e0 0x7fd3a80a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:30.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3ae867700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a809e8d0 con 0x7fd3a80a86d0 2020-11-20T12:14:30.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3ae867700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80a7410 con 0x7fd3a80aad00 2020-11-20T12:14:30.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3ae867700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a809eb00 con 0x7fd3a80ab0e0 2020-11-20T12:14:30.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:30.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41032/0 (socket says 172.21.15.44:41032) 2020-11-20T12:14:30.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a7fff700 1 -- 172.21.15.44:0/287916908 learned_addr learned my addr 172.21.15.44:0/287916908 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:30.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a77fe700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80aad00 0x7fd3a813a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:30.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3ace04700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd3a80ab0e0 0x7fd3a80a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:30.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a7fff700 1 -- 172.21.15.44:0/287916908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 msgr2=0x7fd3a80a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:14:30.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a7fff700 1 -- 172.21.15.44:0/287916908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 msgr2=0x7fd3a80a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:14:30.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a7fff700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:30.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.304+0000 7fd3a7fff700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a7d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:30.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3a77fe700 1 -- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd3a80ab0e0 msgr2=0x7fd3a80a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:30.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3a77fe700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd3a80ab0e0 0x7fd3a80a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:30.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3a77fe700 1 -- 172.21.15.44:0/287916908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 msgr2=0x7fd3a80a7d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:30.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3a77fe700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a7d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:30.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3a77fe700 1 -- 172.21.15.44:0/287916908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a809eda0 con 0x7fd3a80aad00 2020-11-20T12:14:30.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3ace04700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd3a80ab0e0 0x7fd3a80a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:30.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3a77fe700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80aad00 0x7fd3a813a890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd39c00c740 tx=0x7fd39c007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:30.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd387fff700 1 -- 172.21.15.44:0/287916908 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0072e0 con 0x7fd3a80aad00 2020-11-20T12:14:30.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd387fff700 1 -- 172.21.15.44:0/287916908 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c007440 con 0x7fd3a80aad00 2020-11-20T12:14:30.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd387fff700 1 -- 172.21.15.44:0/287916908 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c005800 con 0x7fd3a80aad00 2020-11-20T12:14:30.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a809f040 con 0x7fd3a80aad00 2020-11-20T12:14:30.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.305+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a80a6cc0 con 0x7fd3a80aad00 2020-11-20T12:14:30.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.306+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a80a65d0 con 0x7fd3a80aad00 2020-11-20T12:14:30.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.306+0000 7fd387fff700 1 -- 172.21.15.44:0/287916908 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd39c010070 con 0x7fd3a80aad00 2020-11-20T12:14:30.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.307+0000 7fd387fff700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd380056d20 0x7fd3800591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:30.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.307+0000 7fd387fff700 1 -- 172.21.15.44:0/287916908 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd39c06f700 con 0x7fd3a80aad00 2020-11-20T12:14:30.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.307+0000 7fd3a7fff700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd380056d20 0x7fd3800591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:30.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.307+0000 7fd3a7fff700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd380056d20 0x7fd3800591e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fd398007fd0 tx=0x7fd398009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:30.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.311+0000 7fd387fff700 1 -- 172.21.15.44:0/287916908 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd39c0449d0 con 0x7fd3a80aad00 2020-11-20T12:14:30.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.643+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd3a8069680 con 0x7fd3a80aad00 2020-11-20T12:14:30.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.644+0000 7fd387fff700 1 -- 172.21.15.44:0/287916908 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fd39c047ff0 con 0x7fd3a80aad00 2020-11-20T12:14:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd380056d20 msgr2=0x7fd3800591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd380056d20 0x7fd3800591e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fd398007fd0 tx=0x7fd398009040).stop 2020-11-20T12:14:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80aad00 msgr2=0x7fd3a813a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:30.975 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80aad00 0x7fd3a813a890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fd39c00c740 tx=0x7fd39c007be0).stop 2020-11-20T12:14:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 shutdown_connections 2020-11-20T12:14:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd380056d20 0x7fd3800591e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd3a80ab0e0 0x7fd3a80a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3a80a86d0 0x7fd3a80a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:30.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 --2- 172.21.15.44:0/287916908 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3a80aad00 0x7fd3a813a890 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 >> 172.21.15.44:0/287916908 conn(0x7fd3a8060170 msgr2=0x7fd3a8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 shutdown_connections 2020-11-20T12:14:30.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:30.646+0000 7fd3ae867700 1 -- 172.21.15.44:0/287916908 wait complete. 2020-11-20T12:14:30.977 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":114,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":43,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:33.979 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:34.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.115+0000 7fcbb40df700 1 Processor -- start 2020-11-20T12:14:34.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.115+0000 7fcbb40df700 1 -- start start 2020-11-20T12:14:34.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.115+0000 7fcbb40df700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcbac0aa020 0x7fcbac0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.115+0000 7fcbb40df700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcbac0aa400 0x7fcbac0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.115+0000 7fcbb40df700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac0a9770 con 0x7fcbac13ac60 2020-11-20T12:14:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.115+0000 7fcbb40df700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0a6da0 con 0x7fcbac0aa400 2020-11-20T12:14:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.115+0000 7fcbb40df700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0a7040 con 0x7fcbac0aa020 2020-11-20T12:14:34.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcbac0aa400 0x7fcbac0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcbac0aa400 0x7fcbac0a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41048/0 (socket says 172.21.15.44:41048) 2020-11-20T12:14:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 -- 172.21.15.44:0/2318196466 learned_addr learned my addr 172.21.15.44:0/2318196466 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb1e7b700 1 --2- 172.21.15.44:0/2318196466 >> v2:172.21.15.79:3300/0 conn(0x7fcbac0aa020 0x7fcbac0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 -- 172.21.15.44:0/2318196466 >> v2:172.21.15.79:3300/0 conn(0x7fcbac0aa020 msgr2=0x7fcbac0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:34.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 --2- 172.21.15.44:0/2318196466 >> v2:172.21.15.79:3300/0 conn(0x7fcbac0aa020 0x7fcbac0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 -- 172.21.15.44:0/2318196466 >> v1:172.21.15.79:6789/0 conn(0x7fcbac13ac60 legacy=0x7fcbac0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 -- 172.21.15.44:0/2318196466 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0a6a90 con 0x7fcbac0aa400 2020-11-20T12:14:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb167a700 1 --2- 172.21.15.44:0/2318196466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa400 0x7fcbac0a4fb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcbac0adec0 tx=0x7fcb9c006eb0).ready entity=mon.0 client_cookie=6e2071906f7f7e7a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb0e79700 1 -- 172.21.15.44:0/2318196466 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c009810 con 0x7fcbac0aa400 2020-11-20T12:14:34.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb0e79700 1 -- 172.21.15.44:0/2318196466 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c009970 con 0x7fcbac0aa400 2020-11-20T12:14:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.116+0000 7fcbb0e79700 1 -- 172.21.15.44:0/2318196466 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c009cf0 con 0x7fcbac0aa400 2020-11-20T12:14:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 -- 172.21.15.44:0/2318196466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa400 msgr2=0x7fcbac0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2318196466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa400 0x7fcbac0a4fb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcbac0adec0 tx=0x7fcb9c006eb0).stop 2020-11-20T12:14:34.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 -- 172.21.15.44:0/2318196466 shutdown_connections 2020-11-20T12:14:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2318196466 >> v2:172.21.15.79:3300/0 conn(0x7fcbac0aa020 0x7fcbac0a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2318196466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa400 0x7fcbac0a4fb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 -- 172.21.15.44:0/2318196466 >> 172.21.15.44:0/2318196466 conn(0x7fcbac062690 msgr2=0x7fcbac0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 -- 172.21.15.44:0/2318196466 shutdown_connections 2020-11-20T12:14:34.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.117+0000 7fcbb40df700 1 -- 172.21.15.44:0/2318196466 wait complete. 2020-11-20T12:14:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.118+0000 7fcbb40df700 1 Processor -- start 2020-11-20T12:14:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.118+0000 7fcbb40df700 1 -- start start 2020-11-20T12:14:34.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb40df700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa020 0x7fcbac066e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb40df700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcbac0aa400 0x7fcbac1470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb40df700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbac13ac60 0x7fcbac149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb40df700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac116bf0 con 0x7fcbac0aa020 2020-11-20T12:14:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb40df700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac067bc0 con 0x7fcbac13ac60 2020-11-20T12:14:34.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb40df700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac067970 con 0x7fcbac0aa400 2020-11-20T12:14:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa020 0x7fcbac066e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb267c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbac13ac60 0x7fcbac149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa020 0x7fcbac066e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41050/0 (socket says 172.21.15.44:41050) 2020-11-20T12:14:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 -- 172.21.15.44:0/2663100942 learned_addr learned my addr 172.21.15.44:0/2663100942 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:34.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb167a700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcbac0aa400 0x7fcbac1470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 -- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcbac0aa400 msgr2=0x7fcbac1470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcbac0aa400 0x7fcbac1470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 -- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbac13ac60 msgr2=0x7fcbac149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbac13ac60 0x7fcbac149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 -- 172.21.15.44:0/2663100942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac067fb0 con 0x7fcbac0aa020 2020-11-20T12:14:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb1e7b700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa020 0x7fcbac066e80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fcba800eba0 tx=0x7fcba8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcb99ffb700 1 -- 172.21.15.44:0/2663100942 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800c8e0 con 0x7fcbac0aa020 2020-11-20T12:14:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcb99ffb700 1 -- 172.21.15.44:0/2663100942 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba800ca40 con 0x7fcbac0aa020 2020-11-20T12:14:34.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac1164b0 con 0x7fcbac0aa020 2020-11-20T12:14:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.119+0000 7fcb99ffb700 1 -- 172.21.15.44:0/2663100942 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800cdc0 con 0x7fcbac0aa020 2020-11-20T12:14:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.120+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac066070 con 0x7fcbac0aa020 2020-11-20T12:14:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.121+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac061650 con 0x7fcbac0aa020 2020-11-20T12:14:34.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.121+0000 7fcb99ffb700 1 -- 172.21.15.44:0/2663100942 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcba8012600 con 0x7fcbac0aa020 2020-11-20T12:14:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.121+0000 7fcb99ffb700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb84056d20 0x7fcb840591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.121+0000 7fcb99ffb700 1 -- 172.21.15.44:0/2663100942 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fcba80704c0 con 0x7fcbac0aa020 2020-11-20T12:14:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.121+0000 7fcbb167a700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb84056d20 0x7fcb840591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.122+0000 7fcbb167a700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb84056d20 0x7fcb840591e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fcb9c006ef0 tx=0x7fcb9c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.125+0000 7fcb99ffb700 1 -- 172.21.15.44:0/2663100942 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcba8045790 con 0x7fcbac0aa020 2020-11-20T12:14:34.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.390+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcbac0446d0 con 0x7fcbac0aa020 2020-11-20T12:14:34.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.391+0000 7fcb99ffb700 1 -- 172.21.15.44:0/2663100942 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fcba8048db0 con 0x7fcbac0aa020 2020-11-20T12:14:34.394 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb84056d20 msgr2=0x7fcb840591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb84056d20 0x7fcb840591e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fcb9c006ef0 tx=0x7fcb9c015000).stop 2020-11-20T12:14:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa020 msgr2=0x7fcbac066e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa020 0x7fcbac066e80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fcba800eba0 tx=0x7fcba8006df0).stop 2020-11-20T12:14:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 shutdown_connections 2020-11-20T12:14:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcb84056d20 0x7fcb840591e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcbac13ac60 0x7fcbac149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcbac0aa020 0x7fcbac066e80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 --2- 172.21.15.44:0/2663100942 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcbac0aa400 0x7fcbac1470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 >> 172.21.15.44:0/2663100942 conn(0x7fcbac062690 msgr2=0x7fcbac0a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 shutdown_connections 2020-11-20T12:14:34.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.392+0000 7fcbb40df700 1 -- 172.21.15.44:0/2663100942 wait complete. 2020-11-20T12:14:34.407 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:14:34.407 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:14:34.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.545+0000 7f4d03484700 1 Processor -- start 2020-11-20T12:14:34.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.545+0000 7f4d03484700 1 -- start start 2020-11-20T12:14:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.546+0000 7f4d03484700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a9880 0x7f4cfc0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.546+0000 7f4d03484700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0a8f80 con 0x7f4cfc0a8af0 2020-11-20T12:14:34.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.546+0000 7f4d03484700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0a8450 con 0x7f4cfc0a6600 2020-11-20T12:14:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.546+0000 7f4d03484700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0a71b0 con 0x7f4cfc0a9880 2020-11-20T12:14:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.546+0000 7f4d02c83700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f4cfc0a8af0 0x7f4cfc0a7d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44810/0 (socket says 172.21.15.44:44810) 2020-11-20T12:14:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.546+0000 7f4d02c83700 1 -- 172.21.15.44:0/1806862336 learned_addr learned my addr 172.21.15.44:0/1806862336 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.546+0000 7f4d02482700 1 --2- 172.21.15.44:0/1806862336 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a9880 0x7f4cfc0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d01480700 1 -- 172.21.15.44:0/1806862336 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746909635 0 0) 0x7f4cfc0a8f80 con 0x7f4cfc0a8af0 2020-11-20T12:14:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d01480700 1 -- 172.21.15.44:0/1806862336 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce4003240 con 0x7f4cfc0a8af0 2020-11-20T12:14:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d01480700 1 -- 172.21.15.44:0/1806862336 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327049735 0 0) 0x7f4cfc0a8450 con 0x7f4cfc0a6600 2020-11-20T12:14:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d02482700 1 -- 172.21.15.44:0/1806862336 >> v1:172.21.15.79:6789/0 conn(0x7f4cfc0a6600 legacy=0x7f4cfc0a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d02482700 1 -- 172.21.15.44:0/1806862336 >> v1:172.21.15.44:6789/0 conn(0x7f4cfc0a8af0 legacy=0x7f4cfc0a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d02482700 1 -- 172.21.15.44:0/1806862336 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0a6b80 con 0x7f4cfc0a9880 2020-11-20T12:14:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d02482700 1 --2- 172.21.15.44:0/1806862336 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a9880 0x7f4cfc0a4ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4cf800b9d0 tx=0x7f4cf8006df0).ready entity=mon.2 client_cookie=bf0d45dc84991a0c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d01480700 1 -- 172.21.15.44:0/1806862336 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8009840 con 0x7f4cfc0a9880 2020-11-20T12:14:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d01480700 1 -- 172.21.15.44:0/1806862336 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf80099a0 con 0x7f4cfc0a9880 2020-11-20T12:14:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d01480700 1 -- 172.21.15.44:0/1806862336 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8009d20 con 0x7f4cfc0a9880 2020-11-20T12:14:34.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d03484700 1 -- 172.21.15.44:0/1806862336 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a9880 msgr2=0x7f4cfc0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.547+0000 7f4d03484700 1 --2- 172.21.15.44:0/1806862336 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a9880 0x7f4cfc0a4ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4cf800b9d0 tx=0x7f4cf8006df0).stop 2020-11-20T12:14:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.548+0000 7f4d03484700 1 -- 172.21.15.44:0/1806862336 shutdown_connections 2020-11-20T12:14:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.548+0000 7f4d03484700 1 --2- 172.21.15.44:0/1806862336 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a9880 0x7f4cfc0a4ef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.548+0000 7f4d03484700 1 -- 172.21.15.44:0/1806862336 >> 172.21.15.44:0/1806862336 conn(0x7f4cfc061b60 msgr2=0x7f4cfc061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:34.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.548+0000 7f4d03484700 1 -- 172.21.15.44:0/1806862336 shutdown_connections 2020-11-20T12:14:34.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.548+0000 7f4d03484700 1 -- 172.21.15.44:0/1806862336 wait complete. 2020-11-20T12:14:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 Processor -- start 2020-11-20T12:14:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 -- start start 2020-11-20T12:14:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc0a6600 0x7f4cfc131790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a8af0 0x7f4cfc135ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc0a9880 0x7f4cfc1363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc1368d0 con 0x7f4cfc0a6600 2020-11-20T12:14:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc1370f0 con 0x7f4cfc0a9880 2020-11-20T12:14:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d03484700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc133f00 con 0x7f4cfc0a8af0 2020-11-20T12:14:34.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d02482700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc0a6600 0x7f4cfc131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d02482700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc0a6600 0x7f4cfc131790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41064/0 (socket says 172.21.15.44:41064) 2020-11-20T12:14:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d02482700 1 -- 172.21.15.44:0/619622950 learned_addr learned my addr 172.21.15.44:0/619622950 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d02c83700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc0a9880 0x7f4cfc1363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.549+0000 7f4d01c81700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a8af0 0x7f4cfc135ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d02482700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a8af0 msgr2=0x7f4cfc135ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d02482700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a8af0 0x7f4cfc135ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d02482700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc0a9880 msgr2=0x7f4cfc1363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d02482700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc0a9880 0x7f4cfc1363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d02482700 1 -- 172.21.15.44:0/619622950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc1341a0 con 0x7f4cfc0a6600 2020-11-20T12:14:34.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d01c81700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cfc0a8af0 0x7f4cfc135ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d02c83700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cfc0a9880 0x7f4cfc1363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d02482700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc0a6600 0x7f4cfc131790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4cf800b920 tx=0x7f4cf8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8011dd0 con 0x7f4cfc0a6600 2020-11-20T12:14:34.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf801e460 con 0x7f4cfc0a6600 2020-11-20T12:14:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801e7e0 con 0x7f4cfc0a6600 2020-11-20T12:14:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc133ba0 con 0x7f4cfc0a6600 2020-11-20T12:14:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.550+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc133710 con 0x7f4cfc0a6600 2020-11-20T12:14:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.551+0000 7f4d03484700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc134490 0x7f4cfc14cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.551+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc133dc0 con 0x7f4cfc134490 2020-11-20T12:14:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.551+0000 7f4d01c81700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc134490 0x7f4cfc14cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4cf8025030 con 0x7f4cfc0a6600 2020-11-20T12:14:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4d01c81700 1 -- 172.21.15.44:0/619622950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cfc057d80 con 0x7f4cfc134490 2020-11-20T12:14:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4d01c81700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc134490 0x7f4cfc14cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4cfc135e80 tx=0x7f4cec009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4cd4056d20 0x7f4cd40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4cf8071860 con 0x7f4cfc0a6600 2020-11-20T12:14:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00dc40 con 0x7f4cfc134490 2020-11-20T12:14:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4cec00dda0 con 0x7f4cfc134490 2020-11-20T12:14:34.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4d02c83700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4cd4056d20 0x7f4cd40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc134490 msgr2=0x7f4cfc14cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc134490 0x7f4cfc14cf40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4cfc135e80 tx=0x7f4cec009040).stop 2020-11-20T12:14:34.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc134490 msgr2=0x7f4cfc14cf40 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:34.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.552+0000 7f4cea7fc700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc134490 0x7f4cfc14cf40 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0x7f4cfc135e80 tx=0x7f4cec009040).stop 2020-11-20T12:14:34.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.553+0000 7f4d02c83700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4cd4056d20 0x7f4cd40591e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f4cf4007b20 tx=0x7f4cf4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.565+0000 7f4d03484700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc12ded0 0x7f4cfc12e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.565+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc061180 con 0x7f4cfc12ded0 2020-11-20T12:14:34.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.565+0000 7f4d01c81700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc12ded0 0x7f4cfc12e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.565+0000 7f4d01c81700 1 -- 172.21.15.44:0/619622950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4cfc0446d0 con 0x7f4cfc12ded0 2020-11-20T12:14:34.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.565+0000 7f4d01c81700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc12ded0 0x7f4cfc12e2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4cfc0ac1e0 tx=0x7f4cec018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.565+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec012690 con 0x7f4cfc12ded0 2020-11-20T12:14:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4268 (secure 0 0 0) 0x7f4cec00da30 con 0x7f4cfc12ded0 2020-11-20T12:14:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc12ded0 msgr2=0x7f4cfc12e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4cea7fc700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc12ded0 0x7f4cfc12e2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f4cfc0ac1e0 tx=0x7f4cec018000).stop 2020-11-20T12:14:34.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4cea7fc700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc12ded0 msgr2=0x7f4cfc12e2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4cea7fc700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc12ded0 0x7f4cfc12e2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4cd4056d20 msgr2=0x7f4cd40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4cd4056d20 0x7f4cd40591e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f4cf4007b20 tx=0x7f4cf4009040).stop 2020-11-20T12:14:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc0a6600 msgr2=0x7f4cfc131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 --2- 172.21.15.44:0/619622950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cfc0a6600 0x7f4cfc131790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4cf800b920 tx=0x7f4cf8015000).stop 2020-11-20T12:14:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d02c83700 1 -- 172.21.15.44:0/619622950 reap_dead start 2020-11-20T12:14:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d02c83700 1 -- 172.21.15.44:0/619622950 reap_dead start 2020-11-20T12:14:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 shutdown_connections 2020-11-20T12:14:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 >> 172.21.15.44:0/619622950 conn(0x7f4cfc061b60 msgr2=0x7f4cfc0ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:34.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 shutdown_connections 2020-11-20T12:14:34.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.566+0000 7f4d03484700 1 -- 172.21.15.44:0/619622950 wait complete. 2020-11-20T12:14:34.574 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:14:34.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:34.574 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:34.575 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:14:34.575 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 118, 2020-11-20T12:14:34.575 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:14:34.575 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:14:34.575 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:14:34.576 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:14:34.576 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.576 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:14:34.576 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:34.576 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:14:34.577 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:14:34.577 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:34.577 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:34.577 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:34.577 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:34.578 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:34.578 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:34.578 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:34.578 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:34.578 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.579 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:14:34.579 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:14:34.579 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:34.579 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:34.579 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:34.580 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:34.581 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.581 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:14:34.581 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:14:34.581 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:14:34.582 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:14:34.582 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:14:34.582 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:14:34.582 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:34.582 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:34.583 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:14:34.583 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:14:34.583 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:14:34.583 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:14:34.583 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:14:34.584 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:34.584 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:14:34.584 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:34.584 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:34.584 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:34.584 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:34.585 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:34.585 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:34.585 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:34.585 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:34.585 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.586 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:14:34.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.586 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:14:34.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.586 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:34.587 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:34.587 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:34.587 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:34.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:34.588 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:14:34.588 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.589 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:34.589 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:14:34.589 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.589 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:14:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:14:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:34.590 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:34.591 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:34.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.591 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:34.591 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:34.591 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:34.592 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:34.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:34.592 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:14:34.592 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.593 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:34.593 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:14:34.593 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.594 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.594 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:14:34.594 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:14:34.595 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:34.595 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:34.595 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:34.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.596 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:34.596 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:34.596 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:34.596 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:34.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:34.597 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:14:34.597 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.598 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:34.598 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:14:34.598 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.598 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.599 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:14:34.599 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:14:34.599 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:34.599 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:34.600 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:34.600 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.600 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.600 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:14:34.601 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:14:34.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.601 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:34.601 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:34.601 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:34.602 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.602 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.602 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:14:34.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.602 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:34.603 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:34.603 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:14:34.603 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.603 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.604 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:14:34.604 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:14:34.605 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:14:34.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.751+0000 7fa8598f2700 1 Processor -- start 2020-11-20T12:14:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.751+0000 7fa8598f2700 1 -- start start 2020-11-20T12:14:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.751+0000 7fa8598f2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa854060330 0x7fa8540a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.751+0000 7fa8598f2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8540a4c30 0x7fa85413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.751+0000 7fa8598f2700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8540a6260 con 0x7fa85413ae80 2020-11-20T12:14:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.751+0000 7fa8598f2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8540a52e0 con 0x7fa854060330 2020-11-20T12:14:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.751+0000 7fa8598f2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540a6490 con 0x7fa8540a4c30 2020-11-20T12:14:34.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa8537fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa85413ae80 0x7fa854136f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44828/0 (socket says 172.21.15.44:44828) 2020-11-20T12:14:34.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa8537fe700 1 -- 172.21.15.44:0/1151020175 learned_addr learned my addr 172.21.15.44:0/1151020175 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:34.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa852ffd700 1 --2- 172.21.15.44:0/1151020175 >> v2:172.21.15.79:3300/0 conn(0x7fa854060330 0x7fa8540a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1151020175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8540a4c30 0x7fa85413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa8527fc700 1 -- 172.21.15.44:0/1151020175 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769908622 0 0) 0x7fa8540a6260 con 0x7fa85413ae80 2020-11-20T12:14:34.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa8527fc700 1 -- 172.21.15.44:0/1151020175 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa834003240 con 0x7fa85413ae80 2020-11-20T12:14:34.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa84bfff700 1 -- 172.21.15.44:0/1151020175 >> v2:172.21.15.79:3300/0 conn(0x7fa854060330 msgr2=0x7fa8540a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:34.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa8527fc700 1 -- 172.21.15.44:0/1151020175 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3582855929 0 0) 0x7fa834003240 con 0x7fa85413ae80 2020-11-20T12:14:34.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1151020175 >> v2:172.21.15.79:3300/0 conn(0x7fa854060330 0x7fa8540a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa84bfff700 1 -- 172.21.15.44:0/1151020175 >> v1:172.21.15.44:6789/0 conn(0x7fa85413ae80 legacy=0x7fa854136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa84bfff700 1 -- 172.21.15.44:0/1151020175 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540a6730 con 0x7fa8540a4c30 2020-11-20T12:14:34.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.752+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1151020175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8540a4c30 0x7fa85413a990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa83c00b9a0 tx=0x7fa83c006df0).ready entity=mon.2 client_cookie=7c8480afcc290ad8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8527fc700 1 -- 172.21.15.44:0/1151020175 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c001860 con 0x7fa8540a4c30 2020-11-20T12:14:34.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8527fc700 1 -- 172.21.15.44:0/1151020175 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c011400 con 0x7fa8540a4c30 2020-11-20T12:14:34.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8527fc700 1 -- 172.21.15.44:0/1151020175 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c011780 con 0x7fa8540a4c30 2020-11-20T12:14:34.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 -- 172.21.15.44:0/1151020175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8540a4c30 msgr2=0x7fa85413a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 --2- 172.21.15.44:0/1151020175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8540a4c30 0x7fa85413a990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa83c00b9a0 tx=0x7fa83c006df0).stop 2020-11-20T12:14:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 -- 172.21.15.44:0/1151020175 shutdown_connections 2020-11-20T12:14:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 --2- 172.21.15.44:0/1151020175 >> v2:172.21.15.79:3300/0 conn(0x7fa854060330 0x7fa8540a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 --2- 172.21.15.44:0/1151020175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8540a4c30 0x7fa85413a990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 -- 172.21.15.44:0/1151020175 >> 172.21.15.44:0/1151020175 conn(0x7fa854062690 msgr2=0x7fa8540a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:34.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 -- 172.21.15.44:0/1151020175 shutdown_connections 2020-11-20T12:14:34.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.753+0000 7fa8598f2700 1 -- 172.21.15.44:0/1151020175 wait complete. 2020-11-20T12:14:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.754+0000 7fa8598f2700 1 Processor -- start 2020-11-20T12:14:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.754+0000 7fa8598f2700 1 -- start start 2020-11-20T12:14:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa8598f2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa854060330 0x7fa854136550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa8598f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8540a4c30 0x7fa8540aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa8598f2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa85413ae80 0x7fa854149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa8598f2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85414d6f0 con 0x7fa854060330 2020-11-20T12:14:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa8598f2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85414d990 con 0x7fa8540a4c30 2020-11-20T12:14:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa8598f2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa854136a40 con 0x7fa85413ae80 2020-11-20T12:14:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa854060330 0x7fa854136550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa854060330 0x7fa854136550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41082/0 (socket says 172.21.15.44:41082) 2020-11-20T12:14:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 -- 172.21.15.44:0/1873035207 learned_addr learned my addr 172.21.15.44:0/1873035207 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8540a4c30 0x7fa8540aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa8537fe700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa85413ae80 0x7fa854149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa85413ae80 msgr2=0x7fa854149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa85413ae80 0x7fa854149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8540a4c30 msgr2=0x7fa8540aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8540a4c30 0x7fa8540aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 -- 172.21.15.44:0/1873035207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540ab150 con 0x7fa854060330 2020-11-20T12:14:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.755+0000 7fa852ffd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa854060330 0x7fa854136550 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa84400eba0 tx=0x7fa844006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.756+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400c8e0 con 0x7fa854060330 2020-11-20T12:14:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.756+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84400ca40 con 0x7fa854060330 2020-11-20T12:14:34.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.756+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85414d370 con 0x7fa854060330 2020-11-20T12:14:34.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.756+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400cdc0 con 0x7fa854060330 2020-11-20T12:14:34.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.756+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85414a2f0 con 0x7fa854060330 2020-11-20T12:14:34.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.757+0000 7fa8598f2700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414dc80 0x7fa85414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.757+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85414e290 con 0x7fa85414dc80 2020-11-20T12:14:34.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.757+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414dc80 0x7fa85414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.757+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa844012600 con 0x7fa854060330 2020-11-20T12:14:34.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.757+0000 7fa84bfff700 1 -- 172.21.15.44:0/1873035207 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa85414d560 con 0x7fa85414dc80 2020-11-20T12:14:34.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.757+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414dc80 0x7fa85414a510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa83c00b9a0 tx=0x7fa83c013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa830056d20 0x7fa8300591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa8440704c0 con 0x7fa854060330 2020-11-20T12:14:34.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c01eaf0 con 0x7fa85414dc80 2020-11-20T12:14:34.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa8537fe700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa830056d20 0x7fa8300591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fa83c01ec50 con 0x7fa85414dc80 2020-11-20T12:14:34.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414dc80 msgr2=0x7fa85414a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414dc80 0x7fa85414a510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa83c00b9a0 tx=0x7fa83c013000).stop 2020-11-20T12:14:34.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414dc80 msgr2=0x7fa85414a510 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:34.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa84affd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414dc80 0x7fa85414a510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:34.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.758+0000 7fa8537fe700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa830056d20 0x7fa8300591e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fa854137fa0 tx=0x7fa84c0019a0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.767+0000 7fa8598f2700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414aa00 0x7fa854138390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.767+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854061650 con 0x7fa85414aa00 2020-11-20T12:14:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.767+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414aa00 0x7fa854138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:34.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.767+0000 7fa84bfff700 1 -- 172.21.15.44:0/1873035207 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa8540446d0 con 0x7fa85414aa00 2020-11-20T12:14:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.767+0000 7fa84bfff700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414aa00 0x7fa854138390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa83c006e30 tx=0x7fa83c021070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c02d9c0 con 0x7fa85414aa00 2020-11-20T12:14:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7fa83c01e860 con 0x7fa85414aa00 2020-11-20T12:14:34.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414aa00 msgr2=0x7fa854138390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa84affd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414aa00 0x7fa854138390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa83c006e30 tx=0x7fa83c021070).stop 2020-11-20T12:14:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa84affd700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414aa00 msgr2=0x7fa854138390 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa84affd700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa85414aa00 0x7fa854138390 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0x7fa83c006e30 tx=0x7fa83c021070).stop 2020-11-20T12:14:34.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa830056d20 msgr2=0x7fa8300591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa830056d20 0x7fa8300591e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fa854137fa0 tx=0x7fa84c0019a0).stop 2020-11-20T12:14:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa854060330 msgr2=0x7fa854136550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 --2- 172.21.15.44:0/1873035207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa854060330 0x7fa854136550 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa84400eba0 tx=0x7fa844006df0).stop 2020-11-20T12:14:34.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8537fe700 1 -- 172.21.15.44:0/1873035207 reap_dead start 2020-11-20T12:14:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8537fe700 1 -- 172.21.15.44:0/1873035207 reap_dead start 2020-11-20T12:14:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 shutdown_connections 2020-11-20T12:14:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 >> 172.21.15.44:0/1873035207 conn(0x7fa854062690 msgr2=0x7fa854132470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 shutdown_connections 2020-11-20T12:14:34.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:34.768+0000 7fa8598f2700 1 -- 172.21.15.44:0/1873035207 wait complete. 2020-11-20T12:14:34.785 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:14:34.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:34.786 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:34.786 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:14:34.786 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 118, 2020-11-20T12:14:34.786 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:14:34.787 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:14:34.787 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:14:34.787 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:14:34.787 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.788 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:14:34.788 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:34.788 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:14:34.788 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:14:34.789 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:34.789 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:34.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:34.790 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:34.790 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:34.790 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:34.790 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:34.791 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:34.791 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.791 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:14:34.791 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:14:34.791 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:34.792 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:34.792 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:34.792 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:34.793 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:34.793 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:34.793 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:34.793 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:34.794 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.794 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:14:34.794 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:14:34.795 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:14:34.795 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:14:34.795 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:14:34.795 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:14:34.795 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:34.796 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:34.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:14:34.796 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:14:34.797 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:14:34.797 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:14:34.797 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:14:34.797 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:34.798 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:14:34.798 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:34.798 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:34.798 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:34.799 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:34.799 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:34.799 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:34.799 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:34.799 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:34.800 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.800 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:14:34.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.800 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:14:34.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:34.801 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:34.801 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:34.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:34.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:34.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:14:34.803 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.804 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:34.804 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:14:34.804 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.804 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.805 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.805 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:14:34.805 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:14:34.805 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:34.806 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:34.806 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:34.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:34.807 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:34.807 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:34.807 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:34.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:34.808 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:14:34.808 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.809 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:34.809 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:14:34.810 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.810 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.810 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.811 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:14:34.811 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:14:34.811 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:34.811 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:34.812 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:34.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.812 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:34.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:34.813 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:34.813 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:34.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.814 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:34.814 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:14:34.814 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:34.815 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:14:34.816 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:34.816 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.816 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.817 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:14:34.817 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:14:34.817 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:34.817 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:34.818 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:34.818 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.818 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.819 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:14:34.819 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:14:34.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.819 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:34.820 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:34.820 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:34.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.820 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.821 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:14:34.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.821 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:34.821 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:34.822 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:14:34.822 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.822 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.822 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:14:34.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.823 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:34.823 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:34.823 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:14:34.824 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.824 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.824 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:14:34.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.825 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:34.825 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:34.825 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:14:34.825 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.826 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:34.826 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:14:34.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:34.826 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:34.827 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:34.827 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:34.827 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:34.827 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:34.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:34.828 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:14:34.828 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:14:34.829 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:14:35.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.007+0000 7fbbd7fff700 1 Processor -- start 2020-11-20T12:14:35.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.008+0000 7fbbd7fff700 1 -- start start 2020-11-20T12:14:35.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.008+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80ad350 0x7fbbd80a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.008+0000 7fbbd7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbbd80a40d0 0x7fbbd81379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.008+0000 7fbbd7fff700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd80a4650 con 0x7fbbd80ad730 2020-11-20T12:14:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.008+0000 7fbbd7fff700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd8135090 con 0x7fbbd80a40d0 2020-11-20T12:14:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.008+0000 7fbbd7fff700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80a63b0 con 0x7fbbd80ad350 2020-11-20T12:14:35.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbbd80a40d0 0x7fbbd81379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.008+0000 7fbbd6ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80ad350 0x7fbbd80a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbbd80a40d0 0x7fbbd81379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33148/0 (socket says 172.21.15.44:33148) 2020-11-20T12:14:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd6ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80ad350 0x7fbbd80a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52532/0 (socket says 172.21.15.44:52532) 2020-11-20T12:14:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 -- 172.21.15.44:0/503102618 learned_addr learned my addr 172.21.15.44:0/503102618 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:35.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd67fc700 1 --1- 172.21.15.44:0/503102618 >> v1:172.21.15.79:6789/0 conn(0x7fbbd80ad730 0x7fbbd80676f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47720/0 (socket says 172.21.15.44:47720) 2020-11-20T12:14:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 -- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80ad350 msgr2=0x7fbbd80a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 --2- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80ad350 0x7fbbd80a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 -- 172.21.15.44:0/503102618 >> v1:172.21.15.79:6789/0 conn(0x7fbbd80ad730 legacy=0x7fbbd80676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 -- 172.21.15.44:0/503102618 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80a48f0 con 0x7fbbd80a40d0 2020-11-20T12:14:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd6ffd700 1 --2- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80ad350 0x7fbbd80a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd77fe700 1 --2- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80a40d0 0x7fbbd81379f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fbbd0009d50 tx=0x7fbbd0009290).ready entity=mon.1 client_cookie=182802c8f4fe0388 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd5ffb700 1 -- 172.21.15.44:0/503102618 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000bbc0 con 0x7fbbd80a40d0 2020-11-20T12:14:35.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd5ffb700 1 -- 172.21.15.44:0/503102618 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd000bd20 con 0x7fbbd80a40d0 2020-11-20T12:14:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.009+0000 7fbbd5ffb700 1 -- 172.21.15.44:0/503102618 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0011610 con 0x7fbbd80a40d0 2020-11-20T12:14:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 -- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80a40d0 msgr2=0x7fbbd81379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80a40d0 0x7fbbd81379f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fbbd0009d50 tx=0x7fbbd0009290).stop 2020-11-20T12:14:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 -- 172.21.15.44:0/503102618 shutdown_connections 2020-11-20T12:14:35.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80a40d0 0x7fbbd81379f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/503102618 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80ad350 0x7fbbd80a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 -- 172.21.15.44:0/503102618 >> 172.21.15.44:0/503102618 conn(0x7fbbd8061b60 msgr2=0x7fbbd8061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 -- 172.21.15.44:0/503102618 shutdown_connections 2020-11-20T12:14:35.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.010+0000 7fbbd7fff700 1 -- 172.21.15.44:0/503102618 wait complete. 2020-11-20T12:14:35.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.011+0000 7fbbd7fff700 1 Processor -- start 2020-11-20T12:14:35.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.011+0000 7fbbd7fff700 1 -- start start 2020-11-20T12:14:35.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80a40d0 0x7fbbd8066da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80ad350 0x7fbbd809f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbd80ad730 0x7fbbd81463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd7fff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8145a90 con 0x7fbbd80ad730 2020-11-20T12:14:35.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd7fff700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8145c80 con 0x7fbbd80ad350 2020-11-20T12:14:35.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd7fff700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8145f20 con 0x7fbbd80a40d0 2020-11-20T12:14:35.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbd80ad730 0x7fbbd81463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.024 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd67fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80ad350 0x7fbbd809f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd6ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80a40d0 0x7fbbd8066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbd80ad730 0x7fbbd81463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41104/0 (socket says 172.21.15.44:41104) 2020-11-20T12:14:35.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 -- 172.21.15.44:0/2561046506 learned_addr learned my addr 172.21.15.44:0/2561046506 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:35.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80a40d0 msgr2=0x7fbbd8066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.025 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd80a40d0 0x7fbbd8066da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80ad350 msgr2=0x7fbbd809f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbbd80ad350 0x7fbbd809f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.012+0000 7fbbd77fe700 1 -- 172.21.15.44:0/2561046506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80a35b0 con 0x7fbbd80ad730 2020-11-20T12:14:35.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.013+0000 7fbbd77fe700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbd80ad730 0x7fbbd81463c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbbd0009fd0 tx=0x7fbbd0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.013+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0011e70 con 0x7fbbd80ad730 2020-11-20T12:14:35.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.013+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd000be90 con 0x7fbbd80ad730 2020-11-20T12:14:35.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.013+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0010cd0 con 0x7fbbd80ad730 2020-11-20T12:14:35.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.013+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd8065d40 con 0x7fbbd80ad730 2020-11-20T12:14:35.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.013+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd809d470 con 0x7fbbd80ad730 2020-11-20T12:14:35.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.014+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd809da20 0x7fbbd809de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.014+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8061180 con 0x7fbbd809da20 2020-11-20T12:14:35.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.014+0000 7fbbd6ffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd809da20 0x7fbbd809de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.014+0000 7fbbd6ffd700 1 -- 172.21.15.44:0/2561046506 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbd80a53f0 con 0x7fbbd809da20 2020-11-20T12:14:35.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.014+0000 7fbbd6ffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd809da20 0x7fbbd809de00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fbbc8011e10 tx=0x7fbbc8006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.014+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbbd0017070 con 0x7fbbd80ad730 2020-11-20T12:14:35.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbb4056cf0 0x7fbbb40591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbbd0071e40 con 0x7fbbd80ad730 2020-11-20T12:14:35.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbd67fc700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbb4056cf0 0x7fbbb40591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8018070 con 0x7fbbd809da20 2020-11-20T12:14:35.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fbbc800fec0 con 0x7fbbd809da20 2020-11-20T12:14:35.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd809da20 msgr2=0x7fbbd809de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd809da20 0x7fbbd809de00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fbbc8011e10 tx=0x7fbbc8006df0).stop 2020-11-20T12:14:35.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd809da20 msgr2=0x7fbbd809de00 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:35.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbceffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd809da20 0x7fbbd809de00 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0x7fbbc8011e10 tx=0x7fbbc8006df0).stop 2020-11-20T12:14:35.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.015+0000 7fbbd67fc700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbb4056cf0 0x7fbbb40591b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fbbd809f280 tx=0x7fbbc0000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.023+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd8149060 0x7fbbd8149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.023+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8143510 con 0x7fbbd8149060 2020-11-20T12:14:35.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.024+0000 7fbbd6ffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd8149060 0x7fbbd8149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.024+0000 7fbbd6ffd700 1 -- 172.21.15.44:0/2561046506 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbbd81460e0 con 0x7fbbd8149060 2020-11-20T12:14:35.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.024+0000 7fbbd6ffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd8149060 0x7fbbd8149470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbbc8011e10 tx=0x7fbbc801a000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.024+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8013b40 con 0x7fbbd8149060 2020-11-20T12:14:35.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7fbbc8027070 con 0x7fbbd8149060 2020-11-20T12:14:35.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd8149060 msgr2=0x7fbbd8149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbceffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd8149060 0x7fbbd8149470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbbc8011e10 tx=0x7fbbc801a000).stop 2020-11-20T12:14:35.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbceffd700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd8149060 msgr2=0x7fbbd8149470 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:35.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbceffd700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbbd8149060 0x7fbbd8149470 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbb4056cf0 msgr2=0x7fbbb40591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbbb4056cf0 0x7fbbb40591b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fbbd809f280 tx=0x7fbbc0000b80).stop 2020-11-20T12:14:35.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbd80ad730 msgr2=0x7fbbd81463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd77fe700 1 -- 172.21.15.44:0/2561046506 reap_dead start 2020-11-20T12:14:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbd80ad730 0x7fbbd81463c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbbd0009fd0 tx=0x7fbbd0016000).stop 2020-11-20T12:14:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 shutdown_connections 2020-11-20T12:14:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 --2- 172.21.15.44:0/2561046506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbd80ad730 0x7fbbd81463c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 >> 172.21.15.44:0/2561046506 conn(0x7fbbd8061b60 msgr2=0x7fbbd805ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:35.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 shutdown_connections 2020-11-20T12:14:35.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.025+0000 7fbbd7fff700 1 -- 172.21.15.44:0/2561046506 wait complete. 2020-11-20T12:14:35.037 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:14:35.037 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:35.037 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:35.038 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:14:35.038 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 118, 2020-11-20T12:14:35.038 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:14:35.038 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:14:35.038 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:14:35.039 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:14:35.039 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:35.039 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:14:35.039 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:35.040 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:14:35.040 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:14:35.040 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:35.040 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:35.040 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:35.041 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:35.041 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:35.041 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:35.041 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:35.041 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:35.042 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:35.042 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:14:35.042 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:14:35.042 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:35.042 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:35.043 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:35.043 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:35.043 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:35.043 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:35.043 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:35.044 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:35.044 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:35.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.044 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:14:35.044 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:14:35.045 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:14:35.045 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:14:35.045 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:14:35.045 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:14:35.045 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:35.046 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:35.046 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:14:35.046 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:14:35.046 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:14:35.046 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:14:35.047 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:14:35.047 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:35.047 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:14:35.047 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:35.047 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:35.048 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:35.048 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:35.048 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:35.048 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:35.048 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:35.049 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:35.049 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:35.049 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:14:35.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.049 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:14:35.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.050 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:35.050 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:35.050 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:35.050 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:35.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.051 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:35.051 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:14:35.051 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:35.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:35.052 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:14:35.052 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:35.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.053 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:35.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.053 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:14:35.053 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:14:35.053 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:35.054 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:35.054 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:35.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.054 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:35.056 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:14:35.056 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:35.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:35.057 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:14:35.057 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:35.057 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.057 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:35.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.058 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:14:35.058 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:14:35.058 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:35.058 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:35.058 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:35.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.059 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:35.059 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:35.059 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:35.060 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:35.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:35.060 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:14:35.060 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:35.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:35.061 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:14:35.061 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:35.062 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.062 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:35.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.062 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:14:35.062 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:14:35.063 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:35.063 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:35.063 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:35.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.063 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:35.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.064 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:14:35.064 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:14:35.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.064 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:35.065 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:35.065 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:35.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.065 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:35.065 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:14:35.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.066 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:35.066 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:35.066 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:35.066 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.067 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:35.067 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:14:35.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.067 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:35.067 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:35.068 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:14:35.068 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.068 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:35.068 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:14:35.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:35.069 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:35.069 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:35.069 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:35.069 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:35.069 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:35.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:35.070 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:14:35.070 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:14:35.071 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:14:35.071 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:14:35.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.207+0000 7fda5618a700 1 Processor -- start 2020-11-20T12:14:35.211 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.207+0000 7fda5618a700 1 -- start start 2020-11-20T12:14:35.213 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda5618a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fda50068f60 0x7fda500a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda5618a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fda5013e040 0x7fda50060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda5618a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda500a8870 con 0x7fda50060660 2020-11-20T12:14:35.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda5618a700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda500a9f00 con 0x7fda5013e040 2020-11-20T12:14:35.214 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda5618a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda500a8aa0 con 0x7fda50068f60 2020-11-20T12:14:35.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda4effd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fda5013e040 0x7fda50060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda4ffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fda50060660 0x7fda501376f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44864/0 (socket says 172.21.15.44:44864) 2020-11-20T12:14:35.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda4ffff700 1 -- 172.21.15.44:0/1681075630 learned_addr learned my addr 172.21.15.44:0/1681075630 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:35.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda4effd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fda5013e040 0x7fda50060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41112/0 (socket says 172.21.15.44:41112) 2020-11-20T12:14:35.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda4f7fe700 1 --2- 172.21.15.44:0/1681075630 >> v2:172.21.15.79:3300/0 conn(0x7fda50068f60 0x7fda500a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.208+0000 7fda4e7fc700 1 -- 172.21.15.44:0/1681075630 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750079224 0 0) 0x7fda500a8870 con 0x7fda50060660 2020-11-20T12:14:35.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4e7fc700 1 -- 172.21.15.44:0/1681075630 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda3c003240 con 0x7fda50060660 2020-11-20T12:14:35.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4effd700 1 -- 172.21.15.44:0/1681075630 >> v2:172.21.15.79:3300/0 conn(0x7fda50068f60 msgr2=0x7fda500a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:35.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4effd700 1 --2- 172.21.15.44:0/1681075630 >> v2:172.21.15.79:3300/0 conn(0x7fda50068f60 0x7fda500a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4effd700 1 -- 172.21.15.44:0/1681075630 >> v1:172.21.15.44:6789/0 conn(0x7fda50060660 legacy=0x7fda501376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4effd700 1 -- 172.21.15.44:0/1681075630 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500a8d40 con 0x7fda5013e040 2020-11-20T12:14:35.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4f7fe700 1 --2- 172.21.15.44:0/1681075630 >> v2:172.21.15.79:3300/0 conn(0x7fda50068f60 0x7fda500a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:35.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4effd700 1 --2- 172.21.15.44:0/1681075630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda50060170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fda500ab1e0 tx=0x7fda44006df0).ready entity=mon.0 client_cookie=7645be62e097732e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4e7fc700 1 -- 172.21.15.44:0/1681075630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440096d0 con 0x7fda5013e040 2020-11-20T12:14:35.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4e7fc700 1 -- 172.21.15.44:0/1681075630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44009830 con 0x7fda5013e040 2020-11-20T12:14:35.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda4e7fc700 1 -- 172.21.15.44:0/1681075630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44009bb0 con 0x7fda5013e040 2020-11-20T12:14:35.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda5618a700 1 -- 172.21.15.44:0/1681075630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 msgr2=0x7fda50060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda5618a700 1 --2- 172.21.15.44:0/1681075630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda50060170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fda500ab1e0 tx=0x7fda44006df0).stop 2020-11-20T12:14:35.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda5618a700 1 -- 172.21.15.44:0/1681075630 shutdown_connections 2020-11-20T12:14:35.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda5618a700 1 --2- 172.21.15.44:0/1681075630 >> v2:172.21.15.79:3300/0 conn(0x7fda50068f60 0x7fda500a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.209+0000 7fda5618a700 1 --2- 172.21.15.44:0/1681075630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda50060170 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.210+0000 7fda5618a700 1 -- 172.21.15.44:0/1681075630 >> 172.21.15.44:0/1681075630 conn(0x7fda500ad6f0 msgr2=0x7fda500adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:35.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.210+0000 7fda5618a700 1 -- 172.21.15.44:0/1681075630 shutdown_connections 2020-11-20T12:14:35.219 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.210+0000 7fda5618a700 1 -- 172.21.15.44:0/1681075630 wait complete. 2020-11-20T12:14:35.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.211+0000 7fda5618a700 1 Processor -- start 2020-11-20T12:14:35.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.211+0000 7fda5618a700 1 -- start start 2020-11-20T12:14:35.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.211+0000 7fda5618a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda50060660 0x7fda500acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda5618a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda50068f60 0x7fda50149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda5618a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda500a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda5618a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda500ac5b0 con 0x7fda5013e040 2020-11-20T12:14:35.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda5618a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda500ac850 con 0x7fda50068f60 2020-11-20T12:14:35.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda5618a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda500acaf0 con 0x7fda50060660 2020-11-20T12:14:35.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda500a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda500a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41122/0 (socket says 172.21.15.44:41122) 2020-11-20T12:14:35.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 -- 172.21.15.44:0/1088208430 learned_addr learned my addr 172.21.15.44:0/1088208430 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:35.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4effd700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda50068f60 0x7fda50149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4f7fe700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda50060660 0x7fda500acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 -- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda50060660 msgr2=0x7fda500acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda50060660 0x7fda500acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 -- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda50068f60 msgr2=0x7fda50149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda50068f60 0x7fda50149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 -- 172.21.15.44:0/1088208430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5014a880 con 0x7fda5013e040 2020-11-20T12:14:35.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4effd700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda50068f60 0x7fda50149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:14:35.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda4ffff700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda500a5a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fda48001420 tx=0x7fda48007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda2f7fe700 1 -- 172.21.15.44:0/1088208430 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48005b80 con 0x7fda5013e040 2020-11-20T12:14:35.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda2f7fe700 1 -- 172.21.15.44:0/1088208430 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda48005ce0 con 0x7fda5013e040 2020-11-20T12:14:35.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda500abc30 con 0x7fda5013e040 2020-11-20T12:14:35.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda2f7fe700 1 -- 172.21.15.44:0/1088208430 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48007320 con 0x7fda5013e040 2020-11-20T12:14:35.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.212+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda500ac1a0 con 0x7fda5013e040 2020-11-20T12:14:35.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.213+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda500ac420 con 0x7fda5013e040 2020-11-20T12:14:35.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.214+0000 7fda2f7fe700 1 -- 172.21.15.44:0/1088208430 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fda48010030 con 0x7fda5013e040 2020-11-20T12:14:35.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.214+0000 7fda2f7fe700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda28056d20 0x7fda280591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:35.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.215+0000 7fda2f7fe700 1 -- 172.21.15.44:0/1088208430 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fda48070780 con 0x7fda5013e040 2020-11-20T12:14:35.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.215+0000 7fda4f7fe700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda28056d20 0x7fda280591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:35.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.215+0000 7fda4f7fe700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda28056d20 0x7fda280591e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fda40006fd0 tx=0x7fda40008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:35.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.218+0000 7fda2f7fe700 1 -- 172.21.15.44:0/1088208430 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fda48045ad0 con 0x7fda5013e040 2020-11-20T12:14:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.485+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fda500accb0 con 0x7fda5013e040 2020-11-20T12:14:35.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.485+0000 7fda2f7fe700 1 -- 172.21.15.44:0/1088208430 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fda480490f0 con 0x7fda5013e040 2020-11-20T12:14:35.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda28056d20 msgr2=0x7fda280591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda28056d20 0x7fda280591e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fda40006fd0 tx=0x7fda40008040).stop 2020-11-20T12:14:35.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 msgr2=0x7fda500a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:35.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda500a5a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fda48001420 tx=0x7fda48007e20).stop 2020-11-20T12:14:35.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 shutdown_connections 2020-11-20T12:14:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fda28056d20 0x7fda280591e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda5013e040 0x7fda500a5a10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fda50060660 0x7fda500acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 --2- 172.21.15.44:0/1088208430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda50068f60 0x7fda50149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:35.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 >> 172.21.15.44:0/1088208430 conn(0x7fda500ad6f0 msgr2=0x7fda500a8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 shutdown_connections 2020-11-20T12:14:35.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:35.488+0000 7fda5618a700 1 -- 172.21.15.44:0/1088208430 wait complete. 2020-11-20T12:14:35.507 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:14:37.878 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:14:37.881 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:14:45.508 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:14:45.509 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:45.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.646+0000 7ff3c93c0700 1 Processor -- start 2020-11-20T12:14:45.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.646+0000 7ff3c93c0700 1 -- start start 2020-11-20T12:14:45.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c93c0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413a990 0x7ff3c413adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:45.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c93c0700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40a6260 con 0x7ff3c4060330 2020-11-20T12:14:45.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c93c0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40a6490 con 0x7ff3c413f800 2020-11-20T12:14:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c93c0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40a52e0 con 0x7ff3c413a990 2020-11-20T12:14:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c2ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff3c4060330 0x7ff3c40a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44874/0 (socket says 172.21.15.44:44874) 2020-11-20T12:14:45.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c2ffd700 1 -- 172.21.15.44:0/2417169346 learned_addr learned my addr 172.21.15.44:0/2417169346 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:45.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3bbfff700 1 --2- 172.21.15.44:0/2417169346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413a990 0x7ff3c413adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:45.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930404734 0 0) 0x7ff3c40a6260 con 0x7ff3c4060330 2020-11-20T12:14:45.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a4003240 con 0x7ff3c4060330 2020-11-20T12:14:45.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.647+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106571655 0 0) 0x7ff3c40a6490 con 0x7ff3c413f800 2020-11-20T12:14:45.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c40a6260 con 0x7ff3c413f800 2020-11-20T12:14:45.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3bbfff700 1 -- 172.21.15.44:0/2417169346 >> v1:172.21.15.79:6789/0 conn(0x7ff3c413f800 legacy=0x7ff3c4141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:45.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1318965733 0 0) 0x7ff3a4003240 con 0x7ff3c4060330 2020-11-20T12:14:45.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3bbfff700 1 -- 172.21.15.44:0/2417169346 >> v1:172.21.15.44:6789/0 conn(0x7ff3c4060330 legacy=0x7ff3c40a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:45.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3bbfff700 1 -- 172.21.15.44:0/2417169346 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40a6730 con 0x7ff3c413a990 2020-11-20T12:14:45.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3bbfff700 1 --2- 172.21.15.44:0/2417169346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413a990 0x7ff3c413adf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff3ac00b9d0 tx=0x7ff3ac006df0).ready entity=mon.2 client_cookie=4e482bef8f5998a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:45.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac014070 con 0x7ff3c413a990 2020-11-20T12:14:45.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac00e460 con 0x7ff3c413a990 2020-11-20T12:14:45.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.648+0000 7ff3c27fc700 1 -- 172.21.15.44:0/2417169346 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40014e0 con 0x7ff3c413a990 2020-11-20T12:14:45.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.649+0000 7ff3c93c0700 1 -- 172.21.15.44:0/2417169346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413a990 msgr2=0x7ff3c413adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:45.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.649+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/2417169346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413a990 0x7ff3c413adf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff3ac00b9d0 tx=0x7ff3ac006df0).stop 2020-11-20T12:14:45.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.649+0000 7ff3c93c0700 1 -- 172.21.15.44:0/2417169346 shutdown_connections 2020-11-20T12:14:45.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.649+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/2417169346 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413a990 0x7ff3c413adf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:45.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.649+0000 7ff3c93c0700 1 -- 172.21.15.44:0/2417169346 >> 172.21.15.44:0/2417169346 conn(0x7ff3c4062690 msgr2=0x7ff3c40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:45.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.649+0000 7ff3c93c0700 1 -- 172.21.15.44:0/2417169346 shutdown_connections 2020-11-20T12:14:45.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.649+0000 7ff3c93c0700 1 -- 172.21.15.44:0/2417169346 wait complete. 2020-11-20T12:14:45.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 Processor -- start 2020-11-20T12:14:45.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 -- start start 2020-11-20T12:14:45.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4060330 0x7ff3c4136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:45.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c413a990 0x7ff3c4135120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:45.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413f800 0x7ff3c4066c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:45.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c4135f90 con 0x7ff3c413a990 2020-11-20T12:14:45.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c4135610 con 0x7ff3c4060330 2020-11-20T12:14:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3c93c0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41375c0 con 0x7ff3c413f800 2020-11-20T12:14:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.650+0000 7ff3bbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c413a990 0x7ff3c4135120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c413a990 0x7ff3c4135120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41134/0 (socket says 172.21.15.44:41134) 2020-11-20T12:14:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 -- 172.21.15.44:0/322569061 learned_addr learned my addr 172.21.15.44:0/322569061 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:45.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3c37fe700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413f800 0x7ff3c4066c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3c2ffd700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4060330 0x7ff3c4136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 -- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413f800 msgr2=0x7ff3c4066c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413f800 0x7ff3c4066c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 -- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4060330 msgr2=0x7ff3c4136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:45.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4060330 0x7ff3c4136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:45.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 -- 172.21.15.44:0/322569061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4137860 con 0x7ff3c413a990 2020-11-20T12:14:45.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3c37fe700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413f800 0x7ff3c4066c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:14:45.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3c2ffd700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4060330 0x7ff3c4136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:14:45.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3bbfff700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c413a990 0x7ff3c4135120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff3ac00b150 tx=0x7ff3ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:45.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3baffd700 1 -- 172.21.15.44:0/322569061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac010b50 con 0x7ff3c413a990 2020-11-20T12:14:45.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3baffd700 1 -- 172.21.15.44:0/322569061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac020070 con 0x7ff3c413a990 2020-11-20T12:14:45.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c4137b00 con 0x7ff3c413a990 2020-11-20T12:14:45.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3baffd700 1 -- 172.21.15.44:0/322569061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac010cd0 con 0x7ff3c413a990 2020-11-20T12:14:45.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.651+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c4067140 con 0x7ff3c413a990 2020-11-20T12:14:45.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.652+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c4137420 con 0x7ff3c413a990 2020-11-20T12:14:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.652+0000 7ff3baffd700 1 -- 172.21.15.44:0/322569061 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff3ac02c070 con 0x7ff3c413a990 2020-11-20T12:14:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.653+0000 7ff3baffd700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff3a0056d20 0x7ff3a00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.653+0000 7ff3baffd700 1 -- 172.21.15.44:0/322569061 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff3ac071900 con 0x7ff3c413a990 2020-11-20T12:14:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.653+0000 7ff3c2ffd700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff3a0056d20 0x7ff3a00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.654+0000 7ff3c2ffd700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff3a0056d20 0x7ff3a00591e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7ff3b4000fb0 tx=0x7ff3b4001dd0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:45.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.658+0000 7ff3baffd700 1 -- 172.21.15.44:0/322569061 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff3c4137420 con 0x7ff3c413a990 2020-11-20T12:14:45.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.947+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff3c4137420 con 0x7ff3c413a990 2020-11-20T12:14:45.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.947+0000 7ff3baffd700 1 -- 172.21.15.44:0/322569061 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7ff3c4137420 con 0x7ff3c413a990 2020-11-20T12:14:45.952 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:45.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff3a0056d20 msgr2=0x7ff3a00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff3a0056d20 0x7ff3a00591e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7ff3b4000fb0 tx=0x7ff3b4001dd0).stop 2020-11-20T12:14:45.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c413a990 msgr2=0x7ff3c4135120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c413a990 0x7ff3c4135120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff3ac00b150 tx=0x7ff3ac016000).stop 2020-11-20T12:14:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 shutdown_connections 2020-11-20T12:14:45.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff3a0056d20 0x7ff3a00591e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:45.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff3c413f800 0x7ff3c4066c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:45.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c4060330 0x7ff3c4136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:45.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 --2- 172.21.15.44:0/322569061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3c413a990 0x7ff3c4135120 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:45.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 >> 172.21.15.44:0/322569061 conn(0x7ff3c4062690 msgr2=0x7ff3c40a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:45.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 shutdown_connections 2020-11-20T12:14:45.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:45.949+0000 7ff3c93c0700 1 -- 172.21.15.44:0/322569061 wait complete. 2020-11-20T12:14:45.965 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:14:45.966 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:14:45.967 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:14:46.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.109+0000 7f3fa80ed700 1 Processor -- start 2020-11-20T12:14:46.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.109+0000 7f3fa80ed700 1 -- start start 2020-11-20T12:14:46.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa80ed700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013a990 0x7f3fa013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa80ed700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa00a6260 con 0x7f3fa0060330 2020-11-20T12:14:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa80ed700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa00a6490 con 0x7f3fa013f800 2020-11-20T12:14:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa80ed700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00a52e0 con 0x7f3fa013a990 2020-11-20T12:14:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa5e89700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f3fa0060330 0x7f3fa00a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44892/0 (socket says 172.21.15.44:44892) 2020-11-20T12:14:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa5e89700 1 -- 172.21.15.44:0/1855585852 learned_addr learned my addr 172.21.15.44:0/1855585852 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa5688700 1 --2- 172.21.15.44:0/1855585852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013a990 0x7f3fa013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099688337 0 0) 0x7f3fa00a6260 con 0x7f3fa0060330 2020-11-20T12:14:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.110+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f88003240 con 0x7f3fa0060330 2020-11-20T12:14:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293187011 0 0) 0x7f3fa00a6490 con 0x7f3fa013f800 2020-11-20T12:14:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa00a6260 con 0x7f3fa013f800 2020-11-20T12:14:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa5688700 1 -- 172.21.15.44:0/1855585852 >> v1:172.21.15.79:6789/0 conn(0x7f3fa013f800 legacy=0x7f3fa0141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa5688700 1 -- 172.21.15.44:0/1855585852 >> v1:172.21.15.44:6789/0 conn(0x7f3fa0060330 legacy=0x7f3fa00a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 97846097 0 0) 0x7f3f88003240 con 0x7f3fa0060330 2020-11-20T12:14:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa5688700 1 -- 172.21.15.44:0/1855585852 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00a6730 con 0x7f3fa013a990 2020-11-20T12:14:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa5688700 1 --2- 172.21.15.44:0/1855585852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013a990 0x7f3fa013adf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3f9000b9d0 tx=0x7f3f90006df0).ready entity=mon.2 client_cookie=4c9e5d432f831d8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c0014e0 con 0x7f3fa013a990 2020-11-20T12:14:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9000f560 con 0x7f3fa013a990 2020-11-20T12:14:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.111+0000 7f3fa4e87700 1 -- 172.21.15.44:0/1855585852 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000f8e0 con 0x7f3fa013a990 2020-11-20T12:14:46.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.112+0000 7f3fa80ed700 1 -- 172.21.15.44:0/1855585852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013a990 msgr2=0x7f3fa013adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.112+0000 7f3fa80ed700 1 --2- 172.21.15.44:0/1855585852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013a990 0x7f3fa013adf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3f9000b9d0 tx=0x7f3f90006df0).stop 2020-11-20T12:14:46.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.112+0000 7f3fa80ed700 1 -- 172.21.15.44:0/1855585852 shutdown_connections 2020-11-20T12:14:46.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.112+0000 7f3fa80ed700 1 --2- 172.21.15.44:0/1855585852 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013a990 0x7f3fa013adf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.112+0000 7f3fa80ed700 1 -- 172.21.15.44:0/1855585852 >> 172.21.15.44:0/1855585852 conn(0x7f3fa0062690 msgr2=0x7f3fa00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:46.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.112+0000 7f3fa80ed700 1 -- 172.21.15.44:0/1855585852 shutdown_connections 2020-11-20T12:14:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.112+0000 7f3fa80ed700 1 -- 172.21.15.44:0/1855585852 wait complete. 2020-11-20T12:14:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 Processor -- start 2020-11-20T12:14:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 -- start start 2020-11-20T12:14:46.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa0060330 0x7f3fa00ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa013a990 0x7f3fa00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013f800 0x7f3fa00a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00ad0c0 con 0x7f3fa013a990 2020-11-20T12:14:46.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00ad3d0 con 0x7f3fa0060330 2020-11-20T12:14:46.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa80ed700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00aad00 con 0x7f3fa013f800 2020-11-20T12:14:46.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa5688700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa013a990 0x7f3fa00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa5688700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa013a990 0x7f3fa00ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41148/0 (socket says 172.21.15.44:41148) 2020-11-20T12:14:46.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa5688700 1 -- 172.21.15.44:0/583617310 learned_addr learned my addr 172.21.15.44:0/583617310 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:46.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa5e89700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa0060330 0x7f3fa00ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.113+0000 7f3fa668a700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013f800 0x7f3fa00a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa5688700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013f800 msgr2=0x7f3fa00a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa5688700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3fa013f800 0x7f3fa00a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa5688700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa0060330 msgr2=0x7f3fa00ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa5688700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3fa0060330 0x7f3fa00ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa5688700 1 -- 172.21.15.44:0/583617310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00aafa0 con 0x7f3fa013a990 2020-11-20T12:14:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa5688700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa013a990 0x7f3fa00ac890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3fa00aa8b0 tx=0x7f3f90017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90010ae0 con 0x7f3fa013a990 2020-11-20T12:14:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90010c40 con 0x7f3fa013a990 2020-11-20T12:14:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa00ab440 con 0x7f3fa013a990 2020-11-20T12:14:46.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90018400 con 0x7f3fa013a990 2020-11-20T12:14:46.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.114+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa00a1880 con 0x7f3fa013a990 2020-11-20T12:14:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.115+0000 7f3fa80ed700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa00a28c0 0x7f3fa01312f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.115+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00acf40 con 0x7f3fa00a28c0 2020-11-20T12:14:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.115+0000 7f3fa5e89700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa00a28c0 0x7f3fa01312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.115+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3f9002b070 con 0x7f3fa013a990 2020-11-20T12:14:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3fa5e89700 1 -- 172.21.15.44:0/583617310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fa0061650 con 0x7f3fa00a28c0 2020-11-20T12:14:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3f78056d70 0x7f3f78059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3f90071160 con 0x7f3fa013a990 2020-11-20T12:14:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3fa5e89700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa00a28c0 0x7f3fa01312f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3f9c0011a0 tx=0x7f3f9c0011f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00d970 con 0x7f3fa00a28c0 2020-11-20T12:14:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3fa668a700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3f78056d70 0x7f3f78059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f3f9c00dad0 con 0x7f3fa00a28c0 2020-11-20T12:14:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa00a28c0 msgr2=0x7f3fa01312f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa00a28c0 0x7f3fa01312f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3f9c0011a0 tx=0x7f3f9c0011f0).stop 2020-11-20T12:14:46.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa00a28c0 msgr2=0x7f3fa01312f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:46.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.116+0000 7f3f8dffb700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa00a28c0 0x7f3fa01312f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.117+0000 7f3fa668a700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3f78056d70 0x7f3f78059230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f3fa00a21a0 tx=0x7f3f98007d10).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.126+0000 7f3fa80ed700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa01317e0 0x7f3fa014a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.126+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa0131bf0 con 0x7f3fa01317e0 2020-11-20T12:14:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.126+0000 7f3fa5e89700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa01317e0 0x7f3fa014a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.126+0000 7f3fa5e89700 1 -- 172.21.15.44:0/583617310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3fa00446d0 con 0x7f3fa01317e0 2020-11-20T12:14:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.126+0000 7f3fa5e89700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa01317e0 0x7f3fa014a1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f3f9c0011a0 tx=0x7f3f9c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.126+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c01e040 con 0x7f3fa01317e0 2020-11-20T12:14:46.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4269 (secure 0 0 0) 0x7f3f9c00d760 con 0x7f3fa01317e0 2020-11-20T12:14:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa01317e0 msgr2=0x7f3fa014a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3f8dffb700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa01317e0 0x7f3fa014a1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f3f9c0011a0 tx=0x7f3f9c016000).stop 2020-11-20T12:14:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3f8dffb700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa01317e0 msgr2=0x7f3fa014a1a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:46.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3f8dffb700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa01317e0 0x7f3fa014a1a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3f78056d70 msgr2=0x7f3f78059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3f78056d70 0x7f3f78059230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f3fa00a21a0 tx=0x7f3f98007d10).stop 2020-11-20T12:14:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa013a990 msgr2=0x7f3fa00ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 --2- 172.21.15.44:0/583617310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3fa013a990 0x7f3fa00ac890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3fa00aa8b0 tx=0x7f3f90017000).stop 2020-11-20T12:14:46.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa668a700 1 -- 172.21.15.44:0/583617310 reap_dead start 2020-11-20T12:14:46.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa668a700 1 -- 172.21.15.44:0/583617310 reap_dead start 2020-11-20T12:14:46.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 shutdown_connections 2020-11-20T12:14:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 >> 172.21.15.44:0/583617310 conn(0x7f3fa0062690 msgr2=0x7f3fa00a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 shutdown_connections 2020-11-20T12:14:46.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.127+0000 7f3fa80ed700 1 -- 172.21.15.44:0/583617310 wait complete. 2020-11-20T12:14:46.139 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:14:46.140 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:46.140 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:46.140 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:14:46.140 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 118, 2020-11-20T12:14:46.140 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:14:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:14:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:14:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:14:46.141 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.142 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:14:46.142 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:46.142 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:14:46.142 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:14:46.142 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.143 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.143 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.143 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.144 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.144 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.144 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.144 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:14:46.145 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:14:46.145 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.145 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.145 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.146 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.146 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.146 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.146 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.147 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:14:46.147 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:14:46.147 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:14:46.147 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:14:46.148 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:14:46.148 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:14:46.148 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:46.148 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:46.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:14:46.149 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:14:46.149 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:14:46.149 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:14:46.149 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:14:46.149 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:46.150 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:14:46.150 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.150 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.151 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.151 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.151 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.151 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.152 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:14:46.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.152 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:14:46.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.152 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:46.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:46.153 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:14:46.154 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:14:46.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.157 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:14:46.158 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:14:46.159 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.159 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.159 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.161 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:46.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:46.162 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:14:46.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:14:46.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:14:46.166 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:14:46.166 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.167 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.167 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.167 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:46.168 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:46.168 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:14:46.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:14:46.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:14:46.171 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:14:46.171 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.172 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.172 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.173 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:14:46.173 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:14:46.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.174 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.174 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.174 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:46.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.175 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:14:46.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.177 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.177 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.177 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:14:46.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.178 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.179 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:14:46.179 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:14:46.182 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:14:46.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.321+0000 7f0aa2f12700 1 Processor -- start 2020-11-20T12:14:46.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.321+0000 7f0aa2f12700 1 -- start start 2020-11-20T12:14:46.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2f12700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0a9c0ad440 0x7f0a9c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2f12700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a9c0a9dd0 0x7f0a9c142350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2f12700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0a70f0 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2f12700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0a5880 con 0x7f0a9c0a9dd0 2020-11-20T12:14:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2f12700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0a4170 con 0x7f0a9c0ad440 2020-11-20T12:14:46.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2711700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a9c0a9dd0 0x7f0a9c142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2711700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a9c0a9dd0 0x7f0a9c142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41162/0 (socket says 172.21.15.44:41162) 2020-11-20T12:14:46.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.322+0000 7f0aa2711700 1 -- 172.21.15.44:0/992084411 learned_addr learned my addr 172.21.15.44:0/992084411 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:46.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa1f10700 1 --2- 172.21.15.44:0/992084411 >> v2:172.21.15.79:3300/0 conn(0x7f0a9c0ad440 0x7f0a9c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa2711700 1 -- 172.21.15.44:0/992084411 >> v2:172.21.15.79:3300/0 conn(0x7f0a9c0ad440 msgr2=0x7f0a9c0a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:46.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa2711700 1 --2- 172.21.15.44:0/992084411 >> v2:172.21.15.79:3300/0 conn(0x7f0a9c0ad440 0x7f0a9c0a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa2711700 1 -- 172.21.15.44:0/992084411 >> v1:172.21.15.79:6789/0 conn(0x7f0a9c0a4b00 legacy=0x7f0a9c0a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa2711700 1 -- 172.21.15.44:0/992084411 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0ad820 con 0x7f0a9c0a9dd0 2020-11-20T12:14:46.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa2711700 1 --2- 172.21.15.44:0/992084411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a9dd0 0x7f0a9c142350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0a9c062720 tx=0x7f0a94009290).ready entity=mon.0 client_cookie=c93b9034a605eb94 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa0f0e700 1 -- 172.21.15.44:0/992084411 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400bc50 con 0x7f0a9c0a9dd0 2020-11-20T12:14:46.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa0f0e700 1 -- 172.21.15.44:0/992084411 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9400bdb0 con 0x7f0a9c0a9dd0 2020-11-20T12:14:46.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.323+0000 7f0aa0f0e700 1 -- 172.21.15.44:0/992084411 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94011610 con 0x7f0a9c0a9dd0 2020-11-20T12:14:46.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 -- 172.21.15.44:0/992084411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a9dd0 msgr2=0x7f0a9c142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 --2- 172.21.15.44:0/992084411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a9dd0 0x7f0a9c142350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0a9c062720 tx=0x7f0a94009290).stop 2020-11-20T12:14:46.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 -- 172.21.15.44:0/992084411 shutdown_connections 2020-11-20T12:14:46.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 --2- 172.21.15.44:0/992084411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a9dd0 0x7f0a9c142350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 --2- 172.21.15.44:0/992084411 >> v2:172.21.15.79:3300/0 conn(0x7f0a9c0ad440 0x7f0a9c0a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 -- 172.21.15.44:0/992084411 >> 172.21.15.44:0/992084411 conn(0x7f0a9c061b60 msgr2=0x7f0a9c061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:46.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 -- 172.21.15.44:0/992084411 shutdown_connections 2020-11-20T12:14:46.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.324+0000 7f0aa2f12700 1 -- 172.21.15.44:0/992084411 wait complete. 2020-11-20T12:14:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.325+0000 7f0aa2f12700 1 Processor -- start 2020-11-20T12:14:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.325+0000 7f0aa2f12700 1 -- start start 2020-11-20T12:14:46.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa2f12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a4b00 0x7f0a9c1418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa2f12700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a9c0a9dd0 0x7f0a9c065af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa2f12700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ad440 0x7f0a9c066780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa2f12700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c065750 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa2f12700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c135440 con 0x7f0a9c0ad440 2020-11-20T12:14:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa2f12700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c066550 con 0x7f0a9c0a9dd0 2020-11-20T12:14:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a4b00 0x7f0a9c1418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a4b00 0x7f0a9c1418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41164/0 (socket says 172.21.15.44:41164) 2020-11-20T12:14:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 -- 172.21.15.44:0/3140204236 learned_addr learned my addr 172.21.15.44:0/3140204236 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:46.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa2711700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ad440 0x7f0a9c066780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa170f700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a9c0a9dd0 0x7f0a9c065af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a9c0a9dd0 msgr2=0x7f0a9c065af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a9c0a9dd0 0x7f0a9c065af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ad440 msgr2=0x7f0a9c066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ad440 0x7f0a9c066780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.326+0000 7f0aa1f10700 1 -- 172.21.15.44:0/3140204236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c065fe0 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.327+0000 7f0aa1f10700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a4b00 0x7f0a9c1418e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0a9800ec00 tx=0x7f0a98006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.327+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800b8d0 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.327+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9800ba30 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.327+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c066270 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.327+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800bdb0 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.327+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c13a0c0 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.328+0000 7f0aa2f12700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0aba70 0x7f0a9c0abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.328+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c061180 con 0x7f0a9c0aba70 2020-11-20T12:14:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.328+0000 7f0aa170f700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0aba70 0x7f0a9c0abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.328+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0a98012600 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.328+0000 7f0aa170f700 1 -- 172.21.15.44:0/3140204236 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a9c0aa4d0 con 0x7f0a9c0aba70 2020-11-20T12:14:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0aa170f700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0aba70 0x7f0a9c0abe50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0a9c13ad70 tx=0x7f0a8c007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0a74056d20 0x7f0a740591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0a980703c0 con 0x7f0a9c0a4b00 2020-11-20T12:14:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00d6e0 con 0x7f0a9c0aba70 2020-11-20T12:14:46.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0aa2711700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0a74056d20 0x7f0a740591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f0a8c00d840 con 0x7f0a9c0aba70 2020-11-20T12:14:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0aba70 msgr2=0x7f0a9c0abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0aba70 0x7f0a9c0abe50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f0a9c13ad70 tx=0x7f0a8c007b90).stop 2020-11-20T12:14:46.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0aba70 msgr2=0x7f0a9c0abe50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:46.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0a89ffb700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0aba70 0x7f0a9c0abe50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.329+0000 7f0aa2711700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0a74056d20 0x7f0a740591e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f0a9c1414e0 tx=0x7f0a94016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.338+0000 7f0aa2f12700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ac340 0x7f0a9c14da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.338+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c057d80 con 0x7f0a9c0ac340 2020-11-20T12:14:46.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0aa170f700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ac340 0x7f0a9c14da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0aa170f700 1 -- 172.21.15.44:0/3140204236 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0a9c0446d0 con 0x7f0a9c0ac340 2020-11-20T12:14:46.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0aa170f700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ac340 0x7f0a9c14da40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0a8c007570 tx=0x7f0a8c018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c014c90 con 0x7f0a9c0ac340 2020-11-20T12:14:46.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f0a8c014df0 con 0x7f0a9c0ac340 2020-11-20T12:14:46.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ac340 msgr2=0x7f0a9c14da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0a89ffb700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ac340 0x7f0a9c14da40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0a8c007570 tx=0x7f0a8c018000).stop 2020-11-20T12:14:46.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0a89ffb700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ac340 msgr2=0x7f0a9c14da40 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:46.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.339+0000 7f0a89ffb700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a9c0ac340 0x7f0a9c14da40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0a74056d20 msgr2=0x7f0a740591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0a74056d20 0x7f0a740591e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f0a9c1414e0 tx=0x7f0a94016000).stop 2020-11-20T12:14:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a4b00 msgr2=0x7f0a9c1418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 --2- 172.21.15.44:0/3140204236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a9c0a4b00 0x7f0a9c1418e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0a9800ec00 tx=0x7f0a98006df0).stop 2020-11-20T12:14:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2711700 1 -- 172.21.15.44:0/3140204236 reap_dead start 2020-11-20T12:14:46.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2711700 1 -- 172.21.15.44:0/3140204236 reap_dead start 2020-11-20T12:14:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 shutdown_connections 2020-11-20T12:14:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 >> 172.21.15.44:0/3140204236 conn(0x7f0a9c061b60 msgr2=0x7f0a9c061f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 shutdown_connections 2020-11-20T12:14:46.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.340+0000 7f0aa2f12700 1 -- 172.21.15.44:0/3140204236 wait complete. 2020-11-20T12:14:46.352 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:14:46.353 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:46.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:46.353 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:14:46.354 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 118, 2020-11-20T12:14:46.354 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:14:46.354 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:14:46.354 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:14:46.354 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:14:46.355 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.355 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:14:46.355 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:46.355 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:14:46.356 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:14:46.356 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.356 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.356 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.356 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.357 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.357 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.357 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.357 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.357 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.358 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:14:46.358 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:14:46.358 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.358 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.359 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.359 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.359 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.359 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.359 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.360 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.360 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.360 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:14:46.360 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:14:46.361 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:14:46.361 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:14:46.361 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:14:46.361 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:14:46.362 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:46.362 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:46.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:14:46.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:14:46.362 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:14:46.363 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:14:46.363 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:14:46.363 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:46.363 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:14:46.363 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.364 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.364 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.364 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.364 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.365 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.365 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.365 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.365 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.365 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:14:46.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.366 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:14:46.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:46.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:46.367 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.367 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:14:46.368 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.368 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:14:46.369 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.369 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.369 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.369 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:14:46.370 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:14:46.370 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.370 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.370 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.371 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:46.371 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:46.372 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.372 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.372 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:14:46.373 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.374 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:14:46.374 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.374 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.375 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:14:46.375 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:14:46.375 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.375 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.376 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:46.376 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:46.377 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.377 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.377 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.378 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:14:46.378 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.378 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.379 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:14:46.379 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.379 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.380 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:14:46.380 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:14:46.380 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.380 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.380 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.381 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.381 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.381 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:14:46.381 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:14:46.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.382 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.382 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.382 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:46.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.383 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.383 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:14:46.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.383 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.384 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.384 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:14:46.384 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.384 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.384 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:14:46.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.385 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.385 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.385 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:14:46.385 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.386 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.386 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:14:46.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.386 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.386 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.387 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:14:46.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.387 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.387 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:14:46.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.388 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.388 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.388 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:46.388 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.388 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.389 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:14:46.389 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:14:46.390 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:14:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.533+0000 7f20e85e6700 1 Processor -- start 2020-11-20T12:14:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.533+0000 7f20e85e6700 1 -- start start 2020-11-20T12:14:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e85e6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa020 0x7f20e00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e85e6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f20e00aa400 0x7f20e00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e85e6700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00a9770 con 0x7f20e013ac60 2020-11-20T12:14:46.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e85e6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e00a6da0 con 0x7f20e00aa400 2020-11-20T12:14:46.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e85e6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00a7040 con 0x7f20e00aa020 2020-11-20T12:14:46.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e6b83700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f20e013ac60 0x7f20e00a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:44928/0 (socket says 172.21.15.44:44928) 2020-11-20T12:14:46.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e6b83700 1 -- 172.21.15.44:0/4152309553 learned_addr learned my addr 172.21.15.44:0/4152309553 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:46.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5b81700 1 --2- 172.21.15.44:0/4152309553 >> v2:172.21.15.79:3300/0 conn(0x7f20e00aa400 0x7f20e00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e6382700 1 --2- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa020 0x7f20e00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5380700 1 -- 172.21.15.44:0/4152309553 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238157155 0 0) 0x7f20d8001450 con 0x7f20e013ac60 2020-11-20T12:14:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5380700 1 -- 172.21.15.44:0/4152309553 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c8003240 con 0x7f20e013ac60 2020-11-20T12:14:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5b81700 1 -- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa020 msgr2=0x7f20e00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:46.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5b81700 1 --2- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa020 0x7f20e00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5b81700 1 -- 172.21.15.44:0/4152309553 >> v1:172.21.15.44:6789/0 conn(0x7f20e013ac60 legacy=0x7f20e00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5b81700 1 -- 172.21.15.44:0/4152309553 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00a6a90 con 0x7f20e00aa400 2020-11-20T12:14:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e6382700 1 --2- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa020 0x7f20e00a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.534+0000 7f20e5b81700 1 --2- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e00aa400 0x7f20e00a4fb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f20d000b9a0 tx=0x7f20d0006df0).ready entity=mon.1 client_cookie=6c60a034c5eb677 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e5380700 1 -- 172.21.15.44:0/4152309553 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0009680 con 0x7f20e00aa400 2020-11-20T12:14:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e5380700 1 -- 172.21.15.44:0/4152309553 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d00097e0 con 0x7f20e00aa400 2020-11-20T12:14:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e5380700 1 -- 172.21.15.44:0/4152309553 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0009b60 con 0x7f20e00aa400 2020-11-20T12:14:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 -- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e00aa400 msgr2=0x7f20e00a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 --2- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e00aa400 0x7f20e00a4fb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f20d000b9a0 tx=0x7f20d0006df0).stop 2020-11-20T12:14:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 -- 172.21.15.44:0/4152309553 shutdown_connections 2020-11-20T12:14:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 --2- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa020 0x7f20e00a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 --2- 172.21.15.44:0/4152309553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e00aa400 0x7f20e00a4fb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 -- 172.21.15.44:0/4152309553 >> 172.21.15.44:0/4152309553 conn(0x7f20e0062690 msgr2=0x7f20e00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:46.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 -- 172.21.15.44:0/4152309553 shutdown_connections 2020-11-20T12:14:46.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.535+0000 7f20e85e6700 1 -- 172.21.15.44:0/4152309553 wait complete. 2020-11-20T12:14:46.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.536+0000 7f20e85e6700 1 Processor -- start 2020-11-20T12:14:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.536+0000 7f20e85e6700 1 -- start start 2020-11-20T12:14:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e85e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00aa020 0x7f20e0116670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e85e6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa400 0x7f20e0149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e85e6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e013ac60 0x7f20e0147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e85e6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00670b0 con 0x7f20e00aa020 2020-11-20T12:14:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e85e6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e0067350 con 0x7f20e013ac60 2020-11-20T12:14:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e85e6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00675f0 con 0x7f20e00aa400 2020-11-20T12:14:46.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00aa020 0x7f20e0116670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00aa020 0x7f20e0116670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41182/0 (socket says 172.21.15.44:41182) 2020-11-20T12:14:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 -- 172.21.15.44:0/1295441210 learned_addr learned my addr 172.21.15.44:0/1295441210 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6b83700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e013ac60 0x7f20e0147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e5b81700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa400 0x7f20e0149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa400 msgr2=0x7f20e0149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e00aa400 0x7f20e0149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e013ac60 msgr2=0x7f20e0147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20e013ac60 0x7f20e0147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.537+0000 7f20e6382700 1 -- 172.21.15.44:0/1295441210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0114f70 con 0x7f20e00aa020 2020-11-20T12:14:46.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.538+0000 7f20e6382700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00aa020 0x7f20e0116670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f20dc00ed20 tx=0x7f20dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.538+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00caa0 con 0x7f20e00aa020 2020-11-20T12:14:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.538+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc00cc00 con 0x7f20e00aa020 2020-11-20T12:14:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.538+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e0068080 con 0x7f20e00aa020 2020-11-20T12:14:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.538+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0123f0 con 0x7f20e00aa020 2020-11-20T12:14:46.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.538+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e00a3f30 con 0x7f20e00aa020 2020-11-20T12:14:46.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.539+0000 7f20e85e6700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e01474f0 0x7f20e00677b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.539+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00a4120 con 0x7f20e01474f0 2020-11-20T12:14:46.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.539+0000 7f20e5b81700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e01474f0 0x7f20e00677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.539+0000 7f20e5b81700 1 -- 172.21.15.44:0/1295441210 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20e014a650 con 0x7f20e01474f0 2020-11-20T12:14:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.539+0000 7f20e5b81700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e01474f0 0x7f20e00677b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f20e00a4990 tx=0x7f20d0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.539+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f20dc0127a0 con 0x7f20e00aa020 2020-11-20T12:14:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20b8056cc0 0x7f20b8059180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f20dc070670 con 0x7f20e00aa020 2020-11-20T12:14:46.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20e6b83700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20b8056cc0 0x7f20b8059180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0023030 con 0x7f20e01474f0 2020-11-20T12:14:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f20d0011a70 con 0x7f20e01474f0 2020-11-20T12:14:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e01474f0 msgr2=0x7f20e00677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e01474f0 0x7f20e00677b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f20e00a4990 tx=0x7f20d0015000).stop 2020-11-20T12:14:46.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e01474f0 msgr2=0x7f20e00677b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:46.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20ce7fc700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e01474f0 0x7f20e00677b0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0x7f20e00a4990 tx=0x7f20d0015000).stop 2020-11-20T12:14:46.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.540+0000 7f20e6b83700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20b8056cc0 0x7f20b8059180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f20d8005920 tx=0x7f20d8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.548+0000 7f20e85e6700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e014d570 0x7f20e014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.548+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00671f0 con 0x7f20e014d570 2020-11-20T12:14:46.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.548+0000 7f20e5b81700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e014d570 0x7f20e014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20e5b81700 1 -- 172.21.15.44:0/1295441210 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f20e0066f30 con 0x7f20e014d570 2020-11-20T12:14:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20e5b81700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e014d570 0x7f20e014d950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f20d0006e30 tx=0x7f20d0021070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0033060 con 0x7f20e014d570 2020-11-20T12:14:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7f20d0011860 con 0x7f20e014d570 2020-11-20T12:14:46.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e014d570 msgr2=0x7f20e014d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20ce7fc700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e014d570 0x7f20e014d950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f20d0006e30 tx=0x7f20d0021070).stop 2020-11-20T12:14:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20ce7fc700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e014d570 msgr2=0x7f20e014d950 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:14:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.549+0000 7f20ce7fc700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20e014d570 0x7f20e014d950 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0x7f20d0006e30 tx=0x7f20d0021070).stop 2020-11-20T12:14:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20b8056cc0 msgr2=0x7f20b8059180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20b8056cc0 0x7f20b8059180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f20d8005920 tx=0x7f20d8009040).stop 2020-11-20T12:14:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00aa020 msgr2=0x7f20e0116670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00aa020 0x7f20e0116670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f20dc00ed20 tx=0x7f20dc006df0).stop 2020-11-20T12:14:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e6b83700 1 -- 172.21.15.44:0/1295441210 reap_dead start 2020-11-20T12:14:46.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 shutdown_connections 2020-11-20T12:14:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 --2- 172.21.15.44:0/1295441210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20e00aa020 0x7f20e0116670 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 >> 172.21.15.44:0/1295441210 conn(0x7f20e0062690 msgr2=0x7f20e0135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 shutdown_connections 2020-11-20T12:14:46.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.550+0000 7f20e85e6700 1 -- 172.21.15.44:0/1295441210 wait complete. 2020-11-20T12:14:46.562 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:14:46.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:46.563 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:46.563 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:14:46.563 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 118, 2020-11-20T12:14:46.564 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:14:46.564 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:14:46.564 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:14:46.564 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:14:46.564 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.565 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:14:46.565 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:46.565 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:14:46.565 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:14:46.565 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.566 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.566 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.566 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.566 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.567 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.567 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.567 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.567 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:14:46.567 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:14:46.568 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.568 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.568 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.568 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.568 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.569 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.570 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:14:46.570 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:14:46.570 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:14:46.570 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:14:46.571 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:14:46.571 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:14:46.571 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:46.571 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:14:46.571 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:14:46.572 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:14:46.572 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:14:46.572 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:14:46.572 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:14:46.572 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:14:46.573 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:14:46.573 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:14:46.573 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:14:46.573 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:14:46.573 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:14:46.574 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:14:46.574 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:14:46.574 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:14:46.574 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:14:46.574 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.575 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:14:46.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.575 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:14:46.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.575 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:14:46.576 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:14:46.576 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.576 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.576 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.577 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:14:46.577 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.577 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.578 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:14:46.578 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.578 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.579 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:14:46.579 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:14:46.579 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.579 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.580 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.580 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:14:46.580 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:14:46.581 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.581 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.581 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:14:46.582 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.582 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:14:46.583 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.583 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:14:46.584 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:14:46.584 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.584 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.584 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.585 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:14:46.585 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:14:46.585 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:14:46.585 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:14:46.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:14:46.586 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:14:46.586 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:14:46.587 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:14:46.587 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:14:46.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.588 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.588 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:14:46.588 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:14:46.588 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:14:46.589 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:14:46.589 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:14:46.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.589 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.590 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:14:46.590 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:14:46.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.590 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.591 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.591 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:46.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.591 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.591 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:14:46.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.592 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.592 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.592 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:46.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.593 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.593 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:14:46.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.593 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.593 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.594 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:14:46.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.594 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:14:46.594 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:14:46.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:14:46.595 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:14:46.595 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:14:46.595 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:14:46.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:14:46.596 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:14:46.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:14:46.596 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:14:46.596 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:14:46.598 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:14:46.598 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:14:46.598 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:14:46.598 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:14:46.599 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:14:46.599 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:14:46.608 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:14:46.608 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:14:46.609 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:46.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.780+0000 7f442b175700 1 Processor -- start 2020-11-20T12:14:46.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.780+0000 7f442b175700 1 -- start start 2020-11-20T12:14:46.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.781+0000 7f442b175700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f44240a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.781+0000 7f442b175700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f442413ac60 0x7f44240a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.781+0000 7f442b175700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240a9770 con 0x7f44240aa400 2020-11-20T12:14:46.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.781+0000 7f442b175700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44240a56c0 con 0x7f442413ac60 2020-11-20T12:14:46.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.781+0000 7f442b175700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44240a6da0 con 0x7f44240aa020 2020-11-20T12:14:46.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f44240a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4429712700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f442413ac60 0x7f44240a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f44240a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52632/0 (socket says 172.21.15.44:52632) 2020-11-20T12:14:46.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4429712700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f442413ac60 0x7f44240a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33248/0 (socket says 172.21.15.44:33248) 2020-11-20T12:14:46.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 -- 172.21.15.44:0/554289753 learned_addr learned my addr 172.21.15.44:0/554289753 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:46.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4423fff700 1 --1- 172.21.15.44:0/554289753 >> v1:172.21.15.79:6789/0 conn(0x7f44240aa400 0x7f44240a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47818/0 (socket says 172.21.15.44:47818) 2020-11-20T12:14:46.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 -- 172.21.15.44:0/554289753 >> v2:172.21.15.79:3300/0 conn(0x7f442413ac60 msgr2=0x7f44240a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:14:46.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 --2- 172.21.15.44:0/554289753 >> v2:172.21.15.79:3300/0 conn(0x7f442413ac60 0x7f44240a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 -- 172.21.15.44:0/554289753 >> v1:172.21.15.79:6789/0 conn(0x7f44240aa400 legacy=0x7f44240a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 -- 172.21.15.44:0/554289753 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240a7040 con 0x7f44240aa020 2020-11-20T12:14:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f4428f11700 1 --2- 172.21.15.44:0/554289753 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f44240a9280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f4418006fd0 tx=0x7f4418006eb0).ready entity=mon.2 client_cookie=f55eb495e3ccc32f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f44237fe700 1 -- 172.21.15.44:0/554289753 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418009810 con 0x7f44240aa020 2020-11-20T12:14:46.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f44237fe700 1 -- 172.21.15.44:0/554289753 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418009970 con 0x7f44240aa020 2020-11-20T12:14:46.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.782+0000 7f44237fe700 1 -- 172.21.15.44:0/554289753 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418009cf0 con 0x7f44240aa020 2020-11-20T12:14:46.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 -- 172.21.15.44:0/554289753 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 msgr2=0x7f44240a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 --2- 172.21.15.44:0/554289753 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f44240a9280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f4418006fd0 tx=0x7f4418006eb0).stop 2020-11-20T12:14:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 -- 172.21.15.44:0/554289753 shutdown_connections 2020-11-20T12:14:46.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 --2- 172.21.15.44:0/554289753 >> v2:172.21.15.79:3300/0 conn(0x7f442413ac60 0x7f44240a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 --2- 172.21.15.44:0/554289753 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f44240a9280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 -- 172.21.15.44:0/554289753 >> 172.21.15.44:0/554289753 conn(0x7f4424062690 msgr2=0x7f44240a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:46.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 -- 172.21.15.44:0/554289753 shutdown_connections 2020-11-20T12:14:46.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.783+0000 7f442b175700 1 -- 172.21.15.44:0/554289753 wait complete. 2020-11-20T12:14:46.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.784+0000 7f442b175700 1 Processor -- start 2020-11-20T12:14:46.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.784+0000 7f442b175700 1 -- start start 2020-11-20T12:14:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f442b175700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f4424060170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f442b175700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44240aa400 0x7f44240654a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f442b175700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442413ac60 0x7f44241313f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f442b175700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4424134060 con 0x7f442413ac60 2020-11-20T12:14:46.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f442b175700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44240ab100 con 0x7f44240aa400 2020-11-20T12:14:46.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f442b175700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44240ab3a0 con 0x7f44240aa020 2020-11-20T12:14:46.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4429712700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442413ac60 msgr2=0x7f44241313f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4429712700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442413ac60 0x7f44241313f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4428f11700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f4424060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4423fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44240aa400 0x7f44240654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4423fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44240aa400 0x7f44240654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33252/0 (socket says 172.21.15.44:33252) 2020-11-20T12:14:46.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4423fff700 1 -- 172.21.15.44:0/1774886385 learned_addr learned my addr 172.21.15.44:0/1774886385 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4428f11700 1 -- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44240aa400 msgr2=0x7f44240654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4428f11700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44240aa400 0x7f44240654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:46.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4428f11700 1 -- 172.21.15.44:0/1774886385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442413ac60 msgr2=0x7f44241313f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4428f11700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442413ac60 0x7f44241313f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4428f11700 1 -- 172.21.15.44:0/1774886385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424060b70 con 0x7f44240aa020 2020-11-20T12:14:46.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.785+0000 7f4423fff700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44240aa400 0x7f44240654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.786+0000 7f4428f11700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f4424060170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4418006fd0 tx=0x7f4418015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.786+0000 7f4420ff9700 1 -- 172.21.15.44:0/1774886385 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418011d50 con 0x7f44240aa020 2020-11-20T12:14:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.786+0000 7f4420ff9700 1 -- 172.21.15.44:0/1774886385 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441801e460 con 0x7f44240aa020 2020-11-20T12:14:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.786+0000 7f4420ff9700 1 -- 172.21.15.44:0/1774886385 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441801e7e0 con 0x7f44240aa020 2020-11-20T12:14:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.786+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44240aadf0 con 0x7f44240aa020 2020-11-20T12:14:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.786+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4424065990 con 0x7f44240aa020 2020-11-20T12:14:46.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.787+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44240ab560 con 0x7f44240aa020 2020-11-20T12:14:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.787+0000 7f4420ff9700 1 -- 172.21.15.44:0/1774886385 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f441801a040 con 0x7f44240aa020 2020-11-20T12:14:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.788+0000 7f4420ff9700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f43fc056cd0 0x7f43fc059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:46.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.788+0000 7f4420ff9700 1 -- 172.21.15.44:0/1774886385 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4418071380 con 0x7f44240aa020 2020-11-20T12:14:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.788+0000 7f4423fff700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f43fc056cd0 0x7f43fc059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.788+0000 7f4423fff700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f43fc056cd0 0x7f43fc059190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f4424067d40 tx=0x7f4410007b40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:46.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:46.792+0000 7f4420ff9700 1 -- 172.21.15.44:0/1774886385 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4418046700 con 0x7f44240aa020 2020-11-20T12:14:47.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.066+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4424061650 con 0x7f44240aa020 2020-11-20T12:14:47.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.066+0000 7f4420ff9700 1 -- 172.21.15.44:0/1774886385 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f4418023080 con 0x7f44240aa020 2020-11-20T12:14:47.070 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:47.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f43fc056cd0 msgr2=0x7f43fc059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:47.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f43fc056cd0 0x7f43fc059190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f4424067d40 tx=0x7f4410007b40).stop 2020-11-20T12:14:47.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 msgr2=0x7f4424060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:47.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f4424060170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4418006fd0 tx=0x7f4418015000).stop 2020-11-20T12:14:47.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 shutdown_connections 2020-11-20T12:14:47.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f43fc056cd0 0x7f43fc059190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:47.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f442413ac60 0x7f44241313f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:47.073 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f44240aa020 0x7f4424060170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 --2- 172.21.15.44:0/1774886385 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f44240aa400 0x7f44240654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.068+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 >> 172.21.15.44:0/1774886385 conn(0x7f4424062690 msgr2=0x7f44241391f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.069+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 shutdown_connections 2020-11-20T12:14:47.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:47.069+0000 7f442b175700 1 -- 172.21.15.44:0/1774886385 wait complete. 2020-11-20T12:14:50.084 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:50.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.222+0000 7f74eb13c700 1 Processor -- start 2020-11-20T12:14:50.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.222+0000 7f74eb13c700 1 -- start start 2020-11-20T12:14:50.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74eb13c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa020 0x7f74e40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:50.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74eb13c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f74e413ac60 0x7f74e40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:50.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74eb13c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e40a9770 con 0x7f74e40aa400 2020-11-20T12:14:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74eb13c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e40a56c0 con 0x7f74e413ac60 2020-11-20T12:14:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74eb13c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e40a6da0 con 0x7f74e40aa020 2020-11-20T12:14:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e96d9700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f74e413ac60 msgr2=0x7f74e40a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e96d9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f74e413ac60 0x7f74e40a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:50.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e8ed8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa020 0x7f74e40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e3fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f74e40aa400 0x7f74e40a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47832/0 (socket says 172.21.15.44:47832) 2020-11-20T12:14:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e3fff700 1 -- 172.21.15.44:0/4292634252 learned_addr learned my addr 172.21.15.44:0/4292634252 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:14:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e8ed8700 1 -- 172.21.15.44:0/4292634252 >> v2:172.21.15.44:3300/0 conn(0x7f74e413ac60 msgr2=0x7f74e40a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:14:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e8ed8700 1 --2- 172.21.15.44:0/4292634252 >> v2:172.21.15.44:3300/0 conn(0x7f74e413ac60 0x7f74e40a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:50.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e37fe700 1 -- 172.21.15.44:0/4292634252 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246466327 0 0) 0x7f74d00010d0 con 0x7f74e40aa400 2020-11-20T12:14:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e8ed8700 1 -- 172.21.15.44:0/4292634252 >> v1:172.21.15.79:6789/0 conn(0x7f74e40aa400 legacy=0x7f74e40a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.223+0000 7f74e8ed8700 1 -- 172.21.15.44:0/4292634252 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e40a7040 con 0x7f74e40aa020 2020-11-20T12:14:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74e8ed8700 1 --2- 172.21.15.44:0/4292634252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa020 0x7f74e40a9280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f74d800b9a0 tx=0x7f74d8006df0).ready entity=mon.2 client_cookie=23b8b0a5d5c86847 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:50.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74e37fe700 1 -- 172.21.15.44:0/4292634252 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80097b0 con 0x7f74e40aa020 2020-11-20T12:14:50.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74e37fe700 1 -- 172.21.15.44:0/4292634252 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d8009910 con 0x7f74e40aa020 2020-11-20T12:14:50.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74e37fe700 1 -- 172.21.15.44:0/4292634252 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8009c90 con 0x7f74e40aa020 2020-11-20T12:14:50.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 -- 172.21.15.44:0/4292634252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa020 msgr2=0x7f74e40a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:50.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 --2- 172.21.15.44:0/4292634252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa020 0x7f74e40a9280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f74d800b9a0 tx=0x7f74d8006df0).stop 2020-11-20T12:14:50.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 -- 172.21.15.44:0/4292634252 shutdown_connections 2020-11-20T12:14:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 --2- 172.21.15.44:0/4292634252 >> v2:172.21.15.44:3300/0 conn(0x7f74e413ac60 0x7f74e40a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 --2- 172.21.15.44:0/4292634252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa020 0x7f74e40a9280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 -- 172.21.15.44:0/4292634252 >> 172.21.15.44:0/4292634252 conn(0x7f74e4062690 msgr2=0x7f74e40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:50.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 -- 172.21.15.44:0/4292634252 shutdown_connections 2020-11-20T12:14:50.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.224+0000 7f74eb13c700 1 -- 172.21.15.44:0/4292634252 wait complete. 2020-11-20T12:14:50.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.225+0000 7f74eb13c700 1 Processor -- start 2020-11-20T12:14:50.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.225+0000 7f74eb13c700 1 -- start start 2020-11-20T12:14:50.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74eb13c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74e40aa020 0x7f74e4133010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:50.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74eb13c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa400 0x7f74e4067a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:50.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74eb13c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74e413ac60 0x7f74e4146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:50.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74eb13c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4133a50 con 0x7f74e413ac60 2020-11-20T12:14:50.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74eb13c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4066870 con 0x7f74e40aa020 2020-11-20T12:14:50.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74eb13c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4066b10 con 0x7f74e40aa400 2020-11-20T12:14:50.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74e96d9700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74e413ac60 msgr2=0x7f74e4146c10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:50.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74e96d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74e413ac60 0x7f74e4146c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:50.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74e8ed8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74e40aa020 0x7f74e4133010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:50.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74e3fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa400 0x7f74e4067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:50.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74e8ed8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74e40aa020 0x7f74e4133010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33264/0 (socket says 172.21.15.44:33264) 2020-11-20T12:14:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.226+0000 7f74e8ed8700 1 -- 172.21.15.44:0/1581056742 learned_addr learned my addr 172.21.15.44:0/1581056742 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e3fff700 1 -- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74e40aa020 msgr2=0x7f74e4133010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e3fff700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74e40aa020 0x7f74e4133010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:50.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e3fff700 1 -- 172.21.15.44:0/1581056742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74e413ac60 msgr2=0x7f74e4146c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:50.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e3fff700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74e413ac60 0x7f74e4146c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:50.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e3fff700 1 -- 172.21.15.44:0/1581056742 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e4066db0 con 0x7f74e40aa400 2020-11-20T12:14:50.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e8ed8700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74e40aa020 0x7f74e4133010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:14:50.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e3fff700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa400 0x7f74e4067a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f74e4063320 tx=0x7f74d00078a0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:50.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e0ff9700 1 -- 172.21.15.44:0/1581056742 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d00072e0 con 0x7f74e40aa400 2020-11-20T12:14:50.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e0ff9700 1 -- 172.21.15.44:0/1581056742 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d0007440 con 0x7f74e40aa400 2020-11-20T12:14:50.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e4067250 con 0x7f74e40aa400 2020-11-20T12:14:50.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74e0ff9700 1 -- 172.21.15.44:0/1581056742 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0005800 con 0x7f74e40aa400 2020-11-20T12:14:50.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.227+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e4147100 con 0x7f74e40aa400 2020-11-20T12:14:50.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.228+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e4066f40 con 0x7f74e40aa400 2020-11-20T12:14:50.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.229+0000 7f74e0ff9700 1 -- 172.21.15.44:0/1581056742 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f74d0010070 con 0x7f74e40aa400 2020-11-20T12:14:50.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.229+0000 7f74e0ff9700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f74bc056cd0 0x7f74bc059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:50.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.229+0000 7f74e0ff9700 1 -- 172.21.15.44:0/1581056742 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f74d006f550 con 0x7f74e40aa400 2020-11-20T12:14:50.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.229+0000 7f74e8ed8700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f74bc056cd0 0x7f74bc059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:50.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.230+0000 7f74e8ed8700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f74bc056cd0 0x7f74bc059190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f74d8006a90 tx=0x7f74d8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:50.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.233+0000 7f74e0ff9700 1 -- 172.21.15.44:0/1581056742 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f74d0044820 con 0x7f74e40aa400 2020-11-20T12:14:50.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.502+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f74e40a9750 con 0x7f74e40aa400 2020-11-20T12:14:50.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.503+0000 7f74e0ff9700 1 -- 172.21.15.44:0/1581056742 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f74d0047e40 con 0x7f74e40aa400 2020-11-20T12:14:50.506 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:50.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f74bc056cd0 msgr2=0x7f74bc059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:50.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f74bc056cd0 0x7f74bc059190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f74d8006a90 tx=0x7f74d8015000).stop 2020-11-20T12:14:50.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa400 msgr2=0x7f74e4067a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:50.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa400 0x7f74e4067a80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f74e4063320 tx=0x7f74d00078a0).stop 2020-11-20T12:14:50.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 shutdown_connections 2020-11-20T12:14:50.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f74bc056cd0 0x7f74bc059190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:50.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74e413ac60 0x7f74e4146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:50.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74e40aa020 0x7f74e4133010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:50.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.504+0000 7f74eb13c700 1 --2- 172.21.15.44:0/1581056742 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74e40aa400 0x7f74e4067a80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:50.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.505+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 >> 172.21.15.44:0/1581056742 conn(0x7f74e4062690 msgr2=0x7f74e4139180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:50.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.505+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 shutdown_connections 2020-11-20T12:14:50.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:50.505+0000 7f74eb13c700 1 -- 172.21.15.44:0/1581056742 wait complete. 2020-11-20T12:14:53.522 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:53.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.662+0000 7efc6d214700 1 Processor -- start 2020-11-20T12:14:53.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.662+0000 7efc6d214700 1 -- start start 2020-11-20T12:14:53.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.662+0000 7efc6d214700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc680a1a40 0x7efc680aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:53.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc6d214700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efc680666d0 0x7efc68066b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:53.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc6d214700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc68067470 con 0x7efc68137bb0 2020-11-20T12:14:53.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc6d214700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc680a9b30 con 0x7efc680666d0 2020-11-20T12:14:53.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc6d214700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc680ab240 con 0x7efc680a1a40 2020-11-20T12:14:53.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc6759e700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7efc68137bb0 legacy=0x7efc68138040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:14:53.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc5ffff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7efc680666d0 msgr2=0x7efc68066b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:53.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc5ffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efc680666d0 0x7efc68066b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:53.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc680a1a40 0x7efc680aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:53.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc680a1a40 0x7efc680aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33272/0 (socket says 172.21.15.44:33272) 2020-11-20T12:14:53.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 -- 172.21.15.44:0/3965711326 learned_addr learned my addr 172.21.15.44:0/3965711326 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:53.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 -- 172.21.15.44:0/3965711326 >> v2:172.21.15.44:3300/0 conn(0x7efc680666d0 msgr2=0x7efc68066b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:14:53.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 --2- 172.21.15.44:0/3965711326 >> v2:172.21.15.44:3300/0 conn(0x7efc680666d0 0x7efc68066b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:53.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 -- 172.21.15.44:0/3965711326 >> v1:172.21.15.44:6789/0 conn(0x7efc68137bb0 legacy=0x7efc68138040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:14:53.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 -- 172.21.15.44:0/3965711326 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68067710 con 0x7efc680a1a40 2020-11-20T12:14:53.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc66d9d700 1 --2- 172.21.15.44:0/3965711326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680a1a40 0x7efc680aad50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efc680aef40 tx=0x7efc58006eb0).ready entity=mon.1 client_cookie=8c2ca0075acc73ff server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:53.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.663+0000 7efc6659c700 1 -- 172.21.15.44:0/3965711326 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58009890 con 0x7efc680a1a40 2020-11-20T12:14:53.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6659c700 1 -- 172.21.15.44:0/3965711326 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc580099f0 con 0x7efc680a1a40 2020-11-20T12:14:53.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6659c700 1 -- 172.21.15.44:0/3965711326 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58009d70 con 0x7efc680a1a40 2020-11-20T12:14:53.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 -- 172.21.15.44:0/3965711326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680a1a40 msgr2=0x7efc680aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 --2- 172.21.15.44:0/3965711326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680a1a40 0x7efc680aad50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efc680aef40 tx=0x7efc58006eb0).stop 2020-11-20T12:14:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 -- 172.21.15.44:0/3965711326 shutdown_connections 2020-11-20T12:14:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 --2- 172.21.15.44:0/3965711326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680a1a40 0x7efc680aad50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 --2- 172.21.15.44:0/3965711326 >> v2:172.21.15.44:3300/0 conn(0x7efc680666d0 0x7efc68066b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 -- 172.21.15.44:0/3965711326 >> 172.21.15.44:0/3965711326 conn(0x7efc680af030 msgr2=0x7efc680af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 -- 172.21.15.44:0/3965711326 shutdown_connections 2020-11-20T12:14:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.664+0000 7efc6d214700 1 -- 172.21.15.44:0/3965711326 wait complete. 2020-11-20T12:14:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.665+0000 7efc6d214700 1 Processor -- start 2020-11-20T12:14:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.665+0000 7efc6d214700 1 -- start start 2020-11-20T12:14:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6d214700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680666d0 0x7efc680ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6d214700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc680a1a40 0x7efc6813d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6d214700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68137bb0 0x7efc6814ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6d214700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680ace40 con 0x7efc68137bb0 2020-11-20T12:14:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6d214700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680aeba0 con 0x7efc680666d0 2020-11-20T12:14:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6d214700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6814a4f0 con 0x7efc680a1a40 2020-11-20T12:14:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6759e700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68137bb0 msgr2=0x7efc6814ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc6759e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68137bb0 0x7efc6814ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680666d0 0x7efc680ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680666d0 0x7efc680ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33278/0 (socket says 172.21.15.44:33278) 2020-11-20T12:14:53.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 -- 172.21.15.44:0/1939314976 learned_addr learned my addr 172.21.15.44:0/1939314976 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:53.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc5ffff700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc680a1a40 0x7efc6813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:53.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 -- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc680a1a40 msgr2=0x7efc6813d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:53.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc680a1a40 0x7efc6813d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:53.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 -- 172.21.15.44:0/1939314976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68137bb0 msgr2=0x7efc6814ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:14:53.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68137bb0 0x7efc6814ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:53.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 -- 172.21.15.44:0/1939314976 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6814a790 con 0x7efc680666d0 2020-11-20T12:14:53.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc66d9d700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680666d0 0x7efc680ae500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7efc58006ef0 tx=0x7efc58015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:53.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.666+0000 7efc5effd700 1 -- 172.21.15.44:0/1939314976 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5801e460 con 0x7efc680666d0 2020-11-20T12:14:53.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.667+0000 7efc5effd700 1 -- 172.21.15.44:0/1939314976 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5801e5c0 con 0x7efc680666d0 2020-11-20T12:14:53.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.667+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc68062420 con 0x7efc680666d0 2020-11-20T12:14:53.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.667+0000 7efc5effd700 1 -- 172.21.15.44:0/1939314976 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5801e980 con 0x7efc680666d0 2020-11-20T12:14:53.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.667+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6814b360 con 0x7efc680666d0 2020-11-20T12:14:53.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.668+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc680ae9f0 con 0x7efc680666d0 2020-11-20T12:14:53.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.668+0000 7efc5effd700 1 -- 172.21.15.44:0/1939314976 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7efc58025030 con 0x7efc680666d0 2020-11-20T12:14:53.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.668+0000 7efc5effd700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc44056cd0 0x7efc44059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:53.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.668+0000 7efc5effd700 1 -- 172.21.15.44:0/1939314976 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7efc58071da0 con 0x7efc680666d0 2020-11-20T12:14:53.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.668+0000 7efc5ffff700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc44056cd0 0x7efc44059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:53.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.669+0000 7efc5ffff700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc44056cd0 0x7efc44059190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7efc680ac6c0 tx=0x7efc50008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:14:53.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.672+0000 7efc5effd700 1 -- 172.21.15.44:0/1939314976 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efc58047070 con 0x7efc680666d0 2020-11-20T12:14:53.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.959+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efc6805c2a0 con 0x7efc680666d0 2020-11-20T12:14:53.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.959+0000 7efc5effd700 1 -- 172.21.15.44:0/1939314976 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7efc58023070 con 0x7efc680666d0 2020-11-20T12:14:53.963 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:14:53.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc44056cd0 msgr2=0x7efc44059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:53.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc44056cd0 0x7efc44059190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7efc680ac6c0 tx=0x7efc50008040).stop 2020-11-20T12:14:53.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680666d0 msgr2=0x7efc680ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:14:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680666d0 0x7efc680ae500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7efc58006ef0 tx=0x7efc58015000).stop 2020-11-20T12:14:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 shutdown_connections 2020-11-20T12:14:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc44056cd0 0x7efc44059190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc68137bb0 0x7efc6814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:14:53.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc680666d0 0x7efc680ae500 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 --2- 172.21.15.44:0/1939314976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc680a1a40 0x7efc6813d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:14:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 >> 172.21.15.44:0/1939314976 conn(0x7efc680af030 msgr2=0x7efc680af410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:14:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 shutdown_connections 2020-11-20T12:14:53.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:53.961+0000 7efc6d214700 1 -- 172.21.15.44:0/1939314976 wait complete. 2020-11-20T12:14:56.980 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:14:57.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.110+0000 7f9ed21fe700 1 Processor -- start 2020-11-20T12:14:57.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.110+0000 7f9ed21fe700 1 -- start start 2020-11-20T12:14:57.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ed21fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 0x7f9ecc0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:57.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ed21fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:57.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ed21fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:14:57.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ed21fe700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc1385f0 con 0x7f9ecc0aad00 2020-11-20T12:14:57.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ed21fe700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc13e250 con 0x7f9ecc0a86d0 2020-11-20T12:14:57.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ed21fe700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc13e4f0 con 0x7f9ecc0ab0e0 2020-11-20T12:14:57.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ecb7fe700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 msgr2=0x7f9ecc0a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:57.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ecb7fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 0x7f9ecc0a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:57.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ecbfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:57.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ecaffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:57.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.111+0000 7f9ecbfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33290/0 (socket says 172.21.15.44:33290) 2020-11-20T12:14:57.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 learned_addr learned my addr 172.21.15.44:0/1553159426 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:14:57.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:14:57.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:14:57.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:57.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:57.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:57.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:57.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:57.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.112+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:14:57.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.312+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 msgr2=0x7f9ecc0a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:57.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.312+0000 7f9ecb7fe700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 0x7f9ecc0a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:14:57.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:57.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:57.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:14:57.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:57.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:57.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:14:57.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:14:57.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:57.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:57.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.313+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:14:57.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.713+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 msgr2=0x7f9ecc0a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.713+0000 7f9ecb7fe700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 0x7f9ecc0a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:14:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:14:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:14:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:14:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:57.714+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:14:58.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.514+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 msgr2=0x7f9ecc0a94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:14:58.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.514+0000 7f9ecb7fe700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 0x7f9ecc0a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:14:58.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:58.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:14:58.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:14:58.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecaffd700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:58.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:58.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:14:58.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:14:58.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:14:58.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:14:58.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:14:58.515+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:15:00.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 msgr2=0x7f9ecc13ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:00.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ecc0ab0e0 0x7f9ecc13ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:00.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 msgr2=0x7f9ecc0abc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.79:3300/0 conn(0x7f9ecc0a86d0 0x7f9ecc0abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 -- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 msgr2=0x7f9ecc0a94d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 --2- 172.21.15.44:0/1553159426 >> v2:172.21.15.44:3300/0 conn(0x7f9ecc0aad00 0x7f9ecc0a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 -- 172.21.15.44:0/1553159426 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc0ac120 con 0x7f9eac003360 2020-11-20T12:15:00.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ec97fa700 1 -- 172.21.15.44:0/1553159426 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac00be10 con 0x7f9eac0070a0 2020-11-20T12:15:00.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.111+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1553159426 >> v1:172.21.15.44:6789/0 conn(0x7f9eac003360 legacy=0x7f9eac0057e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:15:00.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.312+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1553159426 >> v1:172.21.15.44:6789/0 conn(0x7f9eac003360 legacy=0x7f9eac0057e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:15:00.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.713+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1553159426 >> v1:172.21.15.44:6789/0 conn(0x7f9eac003360 legacy=0x7f9eac0057e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:15:00.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.814+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146239014 0 0) 0x7f9eac00be10 con 0x7f9eac0070a0 2020-11-20T12:15:00.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.814+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb8003240 con 0x7f9eac0070a0 2020-11-20T12:15:00.821 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3222921727 0 0) 0x7f9eb8003240 con 0x7f9eac0070a0 2020-11-20T12:15:00.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 >> v1:172.21.15.44:6789/0 conn(0x7f9eac003360 legacy=0x7f9eac0057e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:00.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac00d010 con 0x7f9eac0070a0 2020-11-20T12:15:00.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f9ec000d3b0 con 0x7f9eac0070a0 2020-11-20T12:15:00.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 >> v1:172.21.15.79:6789/0 conn(0x7f9eac0070a0 legacy=0x7f9eac009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 reap_dead start 2020-11-20T12:15:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003730 0x7f9eb8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8004020 0x7f9eb8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:00.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9eb800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:00.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb8005210 con 0x7f9eb8003730 2020-11-20T12:15:00.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb8005380 con 0x7f9eb8004ba0 2020-11-20T12:15:00.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb800bc70 con 0x7f9eb8004020 2020-11-20T12:15:00.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003730 msgr2=0x7f9eb8003b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:15:00.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecb7fe700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003730 0x7f9eb8003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:00.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9eb800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:00.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8004020 0x7f9eb8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:00.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8004020 msgr2=0x7f9eb8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:00.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8004020 0x7f9eb8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:00.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003730 msgr2=0x7f9eb8003b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:00.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003730 0x7f9eb8003b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:00.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecbfff700 1 -- 172.21.15.44:0/1553159426 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb8003240 con 0x7f9eb8004ba0 2020-11-20T12:15:00.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.817+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8004020 0x7f9eb8004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:00.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.818+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9eb800b730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9ecc0ad1a0 tx=0x7f9ec4008040).ready entity=mon.1 client_cookie=96c76a3d210452d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:00.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.819+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4014a80 con 0x7f9eb8004ba0 2020-11-20T12:15:00.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.819+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec400c040 con 0x7f9eb8004ba0 2020-11-20T12:15:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.819+0000 7f9eca7fc700 1 -- 172.21.15.44:0/1553159426 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4014e10 con 0x7f9eb8004ba0 2020-11-20T12:15:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 msgr2=0x7f9eb800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9eb800b730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9ecc0ad1a0 tx=0x7f9ec4008040).stop 2020-11-20T12:15:00.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1553159426 shutdown_connections 2020-11-20T12:15:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9eb800b730 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8004020 0x7f9eb8004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1553159426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8003730 0x7f9eb8003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1553159426 >> 172.21.15.44:0/1553159426 conn(0x7f9ecc060170 msgr2=0x7f9ecc060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:00.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1553159426 shutdown_connections 2020-11-20T12:15:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.820+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1553159426 wait complete. 2020-11-20T12:15:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.821+0000 7f9ed21fe700 1 Processor -- start 2020-11-20T12:15:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.821+0000 7f9ed21fe700 1 -- start start 2020-11-20T12:15:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ed21fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8003730 0x7f9ecc0a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:00.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ed21fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8004020 0x7f9ecc13afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ed21fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9ecc13a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ed21fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc13b4c0 con 0x7f9eb8004020 2020-11-20T12:15:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ed21fe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc13b750 con 0x7f9eb8004ba0 2020-11-20T12:15:00.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ed21fe700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc13b9f0 con 0x7f9eb8003730 2020-11-20T12:15:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecaffd700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8004020 msgr2=0x7f9ecc13afd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:15:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecaffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8004020 0x7f9ecc13afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecb7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8003730 0x7f9ecc0a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9ecc13a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:00.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecb7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8003730 0x7f9ecc0a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52712/0 (socket says 172.21.15.44:52712) 2020-11-20T12:15:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1415582379 learned_addr learned my addr 172.21.15.44:0/1415582379 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 msgr2=0x7f9ecc13a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.822+0000 7f9ecb7fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9ecc13a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:00.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1415582379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8004020 msgr2=0x7f9ecc13afd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9ecb7fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8004020 0x7f9ecc13afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9ecb7fe700 1 -- 172.21.15.44:0/1415582379 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc09ec50 con 0x7f9eb8003730 2020-11-20T12:15:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9ecbfff700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9ecc13a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9ecb7fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8003730 0x7f9ecc0a6cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9ebc003ba0 tx=0x7f9ebc009440).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:00.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9eb37fe700 1 -- 172.21.15.44:0/1415582379 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00cba0 con 0x7f9eb8003730 2020-11-20T12:15:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9eb37fe700 1 -- 172.21.15.44:0/1415582379 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc00a040 con 0x7f9eb8003730 2020-11-20T12:15:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc09ef00 con 0x7f9eb8003730 2020-11-20T12:15:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9eb37fe700 1 -- 172.21.15.44:0/1415582379 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc006170 con 0x7f9eb8003730 2020-11-20T12:15:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.823+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc13a8f0 con 0x7f9eb8003730 2020-11-20T12:15:00.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.824+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc13bbb0 con 0x7f9eb8003730 2020-11-20T12:15:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.824+0000 7f9eb37fe700 1 -- 172.21.15.44:0/1415582379 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9ebc0065a0 con 0x7f9eb8003730 2020-11-20T12:15:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.825+0000 7f9eb37fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ea0056d00 0x7f9ea00591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.825+0000 7f9eb37fe700 1 -- 172.21.15.44:0/1415582379 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9ebc0700c0 con 0x7f9eb8003730 2020-11-20T12:15:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.825+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ea0056d00 0x7f9ea00591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:00.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.825+0000 7f9ecaffd700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ea0056d00 0x7f9ea00591c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f9ec0006ba0 tx=0x7f9ec0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:00.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:00.829+0000 7f9eb37fe700 1 -- 172.21.15.44:0/1415582379 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9ebc045390 con 0x7f9eb8003730 2020-11-20T12:15:01.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.085+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9ecc069680 con 0x7f9eb8003730 2020-11-20T12:15:01.089 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.085+0000 7f9eb37fe700 1 -- 172.21.15.44:0/1415582379 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f9ebc0489b0 con 0x7f9eb8003730 2020-11-20T12:15:01.089 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":120,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:01.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ea0056d00 msgr2=0x7f9ea00591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ea0056d00 0x7f9ea00591c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f9ec0006ba0 tx=0x7f9ec0008040).stop 2020-11-20T12:15:01.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8003730 msgr2=0x7f9ecc0a6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8003730 0x7f9ecc0a6cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9ebc003ba0 tx=0x7f9ebc009440).stop 2020-11-20T12:15:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 shutdown_connections 2020-11-20T12:15:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ea0056d00 0x7f9ea00591c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9eb8004ba0 0x7f9ecc13a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9eb8003730 0x7f9ecc0a6cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 --2- 172.21.15.44:0/1415582379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9eb8004020 0x7f9ecc13afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 >> 172.21.15.44:0/1415582379 conn(0x7f9ecc060170 msgr2=0x7f9ecc060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 shutdown_connections 2020-11-20T12:15:01.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.088+0000 7f9ed21fe700 1 -- 172.21.15.44:0/1415582379 wait complete. 2020-11-20T12:15:01.103 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:15:01.103 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:15:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.237+0000 7f4cc6691700 1 Processor -- start 2020-11-20T12:15:01.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cc6691700 1 -- start start 2020-11-20T12:15:01.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cc6691700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4cc013a990 0x7f4cc013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cc6691700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4cc013f800 0x7f4cc0141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cc6691700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc00a6260 con 0x7f4cc0060330 2020-11-20T12:15:01.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cc6691700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc00a52e0 con 0x7f4cc013a990 2020-11-20T12:15:01.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cc6691700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc00a6490 con 0x7f4cc013f800 2020-11-20T12:15:01.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cbffff700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f4cc0060330 legacy=0x7f4cc00a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:15:01.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cb7fff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f4cc013a990 msgr2=0x7f4cc013ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:15:01.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.238+0000 7f4cb7fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4cc013a990 0x7f4cc013ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:01.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4cc013f800 0x7f4cc0141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4cc013f800 0x7f4cc0141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33336/0 (socket says 172.21.15.44:33336) 2020-11-20T12:15:01.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 -- 172.21.15.44:0/3652280113 learned_addr learned my addr 172.21.15.44:0/3652280113 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:01.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 -- 172.21.15.44:0/3652280113 >> v2:172.21.15.44:3300/0 conn(0x7f4cc013a990 msgr2=0x7f4cc013ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:01.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 --2- 172.21.15.44:0/3652280113 >> v2:172.21.15.44:3300/0 conn(0x7f4cc013a990 0x7f4cc013ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:01.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 -- 172.21.15.44:0/3652280113 >> v1:172.21.15.44:6789/0 conn(0x7f4cc0060330 legacy=0x7f4cc00a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:01.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 -- 172.21.15.44:0/3652280113 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00a6730 con 0x7f4cc013f800 2020-11-20T12:15:01.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cc4c2e700 1 --2- 172.21.15.44:0/3652280113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013f800 0x7f4cc0141bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f4cc00a5a90 tx=0x7f4cb8009290).ready entity=mon.1 client_cookie=eb97efcb59a4371f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cbf7fe700 1 -- 172.21.15.44:0/3652280113 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb800bb80 con 0x7f4cc013f800 2020-11-20T12:15:01.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cbf7fe700 1 -- 172.21.15.44:0/3652280113 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb800bce0 con 0x7f4cc013f800 2020-11-20T12:15:01.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.239+0000 7f4cbf7fe700 1 -- 172.21.15.44:0/3652280113 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8011610 con 0x7f4cc013f800 2020-11-20T12:15:01.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 -- 172.21.15.44:0/3652280113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013f800 msgr2=0x7f4cc0141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 --2- 172.21.15.44:0/3652280113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013f800 0x7f4cc0141bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f4cc00a5a90 tx=0x7f4cb8009290).stop 2020-11-20T12:15:01.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 -- 172.21.15.44:0/3652280113 shutdown_connections 2020-11-20T12:15:01.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 --2- 172.21.15.44:0/3652280113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013f800 0x7f4cc0141bf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 --2- 172.21.15.44:0/3652280113 >> v2:172.21.15.44:3300/0 conn(0x7f4cc013a990 0x7f4cc013ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:01.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 -- 172.21.15.44:0/3652280113 >> 172.21.15.44:0/3652280113 conn(0x7f4cc0062690 msgr2=0x7f4cc00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:01.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 -- 172.21.15.44:0/3652280113 shutdown_connections 2020-11-20T12:15:01.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.240+0000 7f4cc6691700 1 -- 172.21.15.44:0/3652280113 wait complete. 2020-11-20T12:15:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.241+0000 7f4cc6691700 1 Processor -- start 2020-11-20T12:15:01.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.241+0000 7f4cc6691700 1 -- start start 2020-11-20T12:15:01.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cc6691700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc0060330 0x7f4cc0141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cc6691700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013a990 0x7f4cc012f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cc6691700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cc013f800 0x7f4cc012f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cc6691700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc0136000 con 0x7f4cc0060330 2020-11-20T12:15:01.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cc6691700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01362a0 con 0x7f4cc013a990 2020-11-20T12:15:01.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cc6691700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc0136540 con 0x7f4cc013f800 2020-11-20T12:15:01.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cbffff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc0060330 msgr2=0x7f4cc0141080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:15:01.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cbffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc0060330 0x7f4cc0141080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:01.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013a990 0x7f4cc012f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013a990 0x7f4cc012f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33340/0 (socket says 172.21.15.44:33340) 2020-11-20T12:15:01.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 -- 172.21.15.44:0/3237237276 learned_addr learned my addr 172.21.15.44:0/3237237276 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:01.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cc4c2e700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cc013f800 0x7f4cc012f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cc013f800 msgr2=0x7f4cc012f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4cc013f800 0x7f4cc012f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc0060330 msgr2=0x7f4cc0141080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:01.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cc0060330 0x7f4cc0141080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 -- 172.21.15.44:0/3237237276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc0137850 con 0x7f4cc013a990 2020-11-20T12:15:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.242+0000 7f4cb7fff700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013a990 0x7f4cc012f170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f4ca8009710 tx=0x7f4ca8006e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.243+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8015070 con 0x7f4cc013a990 2020-11-20T12:15:01.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.243+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca8006170 con 0x7f4cc013a990 2020-11-20T12:15:01.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.243+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc0140020 con 0x7f4cc013a990 2020-11-20T12:15:01.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.243+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80064f0 con 0x7f4cc013a990 2020-11-20T12:15:01.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.243+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc01369e0 con 0x7f4cc013a990 2020-11-20T12:15:01.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cc6691700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc0136c30 0x7f4cc0137010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc0136140 con 0x7f4cc0136c30 2020-11-20T12:15:01.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cbffff700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc0136c30 0x7f4cc0137010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4ca8003af0 con 0x7f4cc013a990 2020-11-20T12:15:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cbffff700 1 -- 172.21.15.44:0/3237237276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cc0141d90 con 0x7f4cc0136c30 2020-11-20T12:15:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cbffff700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc0136c30 0x7f4cc0137010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f4cb00036a0 tx=0x7f4cb0006e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cbcff9700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c9c056cd0 0x7f4c9c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4ca8071ad0 con 0x7f4cc013a990 2020-11-20T12:15:01.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cc4c2e700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c9c056cd0 0x7f4c9c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.244+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00050d0 con 0x7f4cc0136c30 2020-11-20T12:15:01.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.245+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4cb0005230 con 0x7f4cc0136c30 2020-11-20T12:15:01.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.245+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc0136c30 msgr2=0x7f4cc0137010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.245+0000 7f4cbcff9700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc0136c30 0x7f4cc0137010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f4cb00036a0 tx=0x7f4cb0006e60).stop 2020-11-20T12:15:01.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.245+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc0136c30 msgr2=0x7f4cc0137010 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:01.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.245+0000 7f4cbcff9700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc0136c30 0x7f4cc0137010 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0x7f4cb00036a0 tx=0x7f4cb0006e60).stop 2020-11-20T12:15:01.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.245+0000 7f4cc4c2e700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c9c056cd0 0x7f4c9c059190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f4cb80092d0 tx=0x7f4cb8016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.257+0000 7f4cc6691700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc014a340 0x7f4cc014a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.257+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01363e0 con 0x7f4cc014a340 2020-11-20T12:15:01.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.257+0000 7f4cbffff700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc014a340 0x7f4cc014a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.257+0000 7f4cbffff700 1 -- 172.21.15.44:0/3237237276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4cc0135e80 con 0x7f4cc014a340 2020-11-20T12:15:01.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.257+0000 7f4cbffff700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc014a340 0x7f4cc014a750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f4cb0009fd0 tx=0x7f4cb0013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000b4b0 con 0x7f4cc014a340 2020-11-20T12:15:01.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4767 (secure 0 0 0) 0x7f4cb00036d0 con 0x7f4cc014a340 2020-11-20T12:15:01.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc014a340 msgr2=0x7f4cc014a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cbcff9700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc014a340 0x7f4cc014a750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f4cb0009fd0 tx=0x7f4cb0013000).stop 2020-11-20T12:15:01.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cbcff9700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc014a340 msgr2=0x7f4cc014a750 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:01.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cbcff9700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc014a340 0x7f4cc014a750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c9c056cd0 msgr2=0x7f4c9c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c9c056cd0 0x7f4c9c059190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f4cb80092d0 tx=0x7f4cb8016000).stop 2020-11-20T12:15:01.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013a990 msgr2=0x7f4cc012f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 --2- 172.21.15.44:0/3237237276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cc013a990 0x7f4cc012f170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f4ca8009710 tx=0x7f4ca8006e20).stop 2020-11-20T12:15:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc4c2e700 1 -- 172.21.15.44:0/3237237276 reap_dead start 2020-11-20T12:15:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc4c2e700 1 -- 172.21.15.44:0/3237237276 reap_dead start 2020-11-20T12:15:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 shutdown_connections 2020-11-20T12:15:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 >> 172.21.15.44:0/3237237276 conn(0x7f4cc0062690 msgr2=0x7f4cc0138390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 shutdown_connections 2020-11-20T12:15:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.258+0000 7f4cc6691700 1 -- 172.21.15.44:0/3237237276 wait complete. 2020-11-20T12:15:01.277 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:01.277 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:01.277 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:01.278 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:15:01.278 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 120, 2020-11-20T12:15:01.279 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:01.279 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:01.279 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:01.280 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.280 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:15:01.280 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:01.280 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:01.280 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:01.281 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:01.281 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:01.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:01.281 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:01.281 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.282 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:01.283 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:01.283 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:01.283 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:01.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:01.284 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:01.284 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:01.284 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:01.284 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:01.284 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:01.285 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.285 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:01.285 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:01.285 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:01.286 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:01.286 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:01.286 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:01.286 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:01.286 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:01.287 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:01.287 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:01.287 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:01.287 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:01.288 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:01.288 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:01.288 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:01.288 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:01.288 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:01.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:01.289 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:01.289 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:01.289 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:01.289 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:01.290 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:01.290 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.290 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:01.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:01.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.291 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:01.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:01.291 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:01.291 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:01.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:01.292 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:01.292 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:01.293 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:01.293 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.294 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.294 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.294 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:01.294 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:01.295 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:01.295 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:01.295 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:01.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.296 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:01.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:01.296 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:01.296 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:01.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:01.297 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:01.297 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:01.298 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:01.298 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.299 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.299 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:01.299 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:01.299 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:01.300 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:01.300 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:01.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.300 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:01.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:01.301 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:01.301 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:01.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:01.302 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:01.302 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:01.303 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:01.303 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.303 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.303 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.304 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:01.304 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:01.304 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:01.304 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:01.305 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:01.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.305 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.305 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:01.306 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:01.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.306 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.306 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.306 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:01.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.307 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.307 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:15:01.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.307 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.308 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.308 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:15:01.308 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.308 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:15:01.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.309 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.309 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.309 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:15:01.309 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.310 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.310 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:01.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.310 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.311 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.311 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:15:01.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.311 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.311 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:15:01.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.312 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.312 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.312 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:01.312 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.313 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.313 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:01.313 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:01.314 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:15:01.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.479+0000 7fe28df63700 1 Processor -- start 2020-11-20T12:15:01.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.479+0000 7fe28df63700 1 -- start start 2020-11-20T12:15:01.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.479+0000 7fe28df63700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288061340 0x7fe288068130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.479+0000 7fe28df63700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe2880a4e10 0x7fe28813ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.479+0000 7fe28df63700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe2880a74d0 0x7fe288137a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe28df63700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2880a5900 con 0x7fe2880a74d0 2020-11-20T12:15:01.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe28df63700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2880a6ba0 con 0x7fe2880a4e10 2020-11-20T12:15:01.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe28df63700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe288068620 con 0x7fe288061340 2020-11-20T12:15:01.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe287fff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fe2880a74d0 msgr2=0x7fe288137a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:15:01.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe287fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe2880a74d0 0x7fe288137a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:01.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288061340 0x7fe288068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe286ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe2880a4e10 0x7fe28813ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288061340 0x7fe288068130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52738/0 (socket says 172.21.15.44:52738) 2020-11-20T12:15:01.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe286ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe2880a4e10 0x7fe28813ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33352/0 (socket says 172.21.15.44:33352) 2020-11-20T12:15:01.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 -- 172.21.15.44:0/2674961607 learned_addr learned my addr 172.21.15.44:0/2674961607 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:01.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 -- 172.21.15.44:0/2674961607 >> v2:172.21.15.79:3300/0 conn(0x7fe2880a4e10 msgr2=0x7fe28813ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:01.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 --2- 172.21.15.44:0/2674961607 >> v2:172.21.15.79:3300/0 conn(0x7fe2880a4e10 0x7fe28813ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 -- 172.21.15.44:0/2674961607 >> v2:172.21.15.44:3300/0 conn(0x7fe2880a74d0 msgr2=0x7fe288137a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:01.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 --2- 172.21.15.44:0/2674961607 >> v2:172.21.15.44:3300/0 conn(0x7fe2880a74d0 0x7fe288137a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:01.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 -- 172.21.15.44:0/2674961607 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880a6d90 con 0x7fe288061340 2020-11-20T12:15:01.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.480+0000 7fe2877fe700 1 --2- 172.21.15.44:0/2674961607 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288061340 0x7fe288068130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe288064b90 tx=0x7fe278006eb0).ready entity=mon.2 client_cookie=322742d1968b3bcd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe2867fc700 1 -- 172.21.15.44:0/2674961607 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278009810 con 0x7fe288061340 2020-11-20T12:15:01.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe2867fc700 1 -- 172.21.15.44:0/2674961607 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe278009970 con 0x7fe288061340 2020-11-20T12:15:01.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe2867fc700 1 -- 172.21.15.44:0/2674961607 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278009cf0 con 0x7fe288061340 2020-11-20T12:15:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 -- 172.21.15.44:0/2674961607 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288061340 msgr2=0x7fe288068130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 --2- 172.21.15.44:0/2674961607 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288061340 0x7fe288068130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe288064b90 tx=0x7fe278006eb0).stop 2020-11-20T12:15:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 -- 172.21.15.44:0/2674961607 shutdown_connections 2020-11-20T12:15:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 --2- 172.21.15.44:0/2674961607 >> v2:172.21.15.44:3300/0 conn(0x7fe2880a74d0 0x7fe288137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:01.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 --2- 172.21.15.44:0/2674961607 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288061340 0x7fe288068130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 --2- 172.21.15.44:0/2674961607 >> v2:172.21.15.79:3300/0 conn(0x7fe2880a4e10 0x7fe28813ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 -- 172.21.15.44:0/2674961607 >> 172.21.15.44:0/2674961607 conn(0x7fe2880ad6e0 msgr2=0x7fe2880adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:01.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 -- 172.21.15.44:0/2674961607 shutdown_connections 2020-11-20T12:15:01.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.481+0000 7fe28df63700 1 -- 172.21.15.44:0/2674961607 wait complete. 2020-11-20T12:15:01.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.482+0000 7fe28df63700 1 Processor -- start 2020-11-20T12:15:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.482+0000 7fe28df63700 1 -- start start 2020-11-20T12:15:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe28df63700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288061340 0x7fe288066a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe28df63700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880a4e10 0x7fe2880a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe28df63700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2880a74d0 0x7fe28813a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe28df63700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880672e0 con 0x7fe2880a74d0 2020-11-20T12:15:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe28df63700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe288067580 con 0x7fe288061340 2020-11-20T12:15:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe28df63700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe288066fd0 con 0x7fe2880a4e10 2020-11-20T12:15:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe287fff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2880a74d0 msgr2=0x7fe28813a190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:15:01.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe287fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2880a74d0 0x7fe28813a190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe2877fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288061340 0x7fe288066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe2877fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288061340 0x7fe288066a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33356/0 (socket says 172.21.15.44:33356) 2020-11-20T12:15:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe286ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880a4e10 0x7fe2880a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.483+0000 7fe2877fe700 1 -- 172.21.15.44:0/1823890051 learned_addr learned my addr 172.21.15.44:0/1823890051 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe286ffd700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288061340 msgr2=0x7fe288066a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe286ffd700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288061340 0x7fe288066a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe286ffd700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2880a74d0 msgr2=0x7fe28813a190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe286ffd700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2880a74d0 0x7fe28813a190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:01.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe286ffd700 1 -- 172.21.15.44:0/1823890051 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288066570 con 0x7fe2880a4e10 2020-11-20T12:15:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe2877fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288061340 0x7fe288066a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe286ffd700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880a4e10 0x7fe2880a42d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe27c006fd0 tx=0x7fe27c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00c970 con 0x7fe2880a4e10 2020-11-20T12:15:01.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c00cad0 con 0x7fe2880a4e10 2020-11-20T12:15:01.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe288138800 con 0x7fe2880a4e10 2020-11-20T12:15:01.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00ce50 con 0x7fe2880a4e10 2020-11-20T12:15:01.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.484+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2880a3490 con 0x7fe2880a4e10 2020-11-20T12:15:01.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.485+0000 7fe28df63700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880619a0 0x7fe288134960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.485+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe288067190 con 0x7fe2880619a0 2020-11-20T12:15:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.485+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe27c01a030 con 0x7fe2880a4e10 2020-11-20T12:15:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.485+0000 7fe2877fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880619a0 0x7fe288134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.485+0000 7fe2677fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe260056cd0 0x7fe260059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2877fe700 1 -- 172.21.15.44:0/1823890051 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2880a36e0 con 0x7fe2880619a0 2020-11-20T12:15:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe27c06fc60 con 0x7fe2880a4e10 2020-11-20T12:15:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe287fff700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe260056cd0 0x7fe260059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2877fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880619a0 0x7fe288134960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe27800b1b0 tx=0x7fe278015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27801f720 con 0x7fe2880619a0 2020-11-20T12:15:01.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe287fff700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe260056cd0 0x7fe260059190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe280006fd0 tx=0x7fe280006d90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fe27801e420 con 0x7fe2880619a0 2020-11-20T12:15:01.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880619a0 msgr2=0x7fe288134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2677fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880619a0 0x7fe288134960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe27800b1b0 tx=0x7fe278015000).stop 2020-11-20T12:15:01.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880619a0 msgr2=0x7fe288134960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:01.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.486+0000 7fe2677fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880619a0 0x7fe288134960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.494+0000 7fe28df63700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288134e50 0x7fe28814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.494+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880a5fb0 con 0x7fe288134e50 2020-11-20T12:15:01.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.494+0000 7fe2877fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288134e50 0x7fe28814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2877fe700 1 -- 172.21.15.44:0/1823890051 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe288068460 con 0x7fe288134e50 2020-11-20T12:15:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2877fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288134e50 0x7fe28814d6f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fe278006d10 tx=0x7fe278021070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278029440 con 0x7fe288134e50 2020-11-20T12:15:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4595 (secure 0 0 0) 0x7fe27801f510 con 0x7fe288134e50 2020-11-20T12:15:01.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288134e50 msgr2=0x7fe28814d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2677fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288134e50 0x7fe28814d6f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fe278006d10 tx=0x7fe278021070).stop 2020-11-20T12:15:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2677fe700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288134e50 msgr2=0x7fe28814d6f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.495+0000 7fe2677fe700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe288134e50 0x7fe28814d6f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:01.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe260056cd0 msgr2=0x7fe260059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe260056cd0 0x7fe260059190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fe280006fd0 tx=0x7fe280006d90).stop 2020-11-20T12:15:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880a4e10 msgr2=0x7fe2880a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:01.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 --2- 172.21.15.44:0/1823890051 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2880a4e10 0x7fe2880a42d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe27c006fd0 tx=0x7fe27c006df0).stop 2020-11-20T12:15:01.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe287fff700 1 -- 172.21.15.44:0/1823890051 reap_dead start 2020-11-20T12:15:01.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe287fff700 1 -- 172.21.15.44:0/1823890051 reap_dead start 2020-11-20T12:15:01.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 shutdown_connections 2020-11-20T12:15:01.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 >> 172.21.15.44:0/1823890051 conn(0x7fe2880ad6e0 msgr2=0x7fe288060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:01.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 shutdown_connections 2020-11-20T12:15:01.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:01.496+0000 7fe28df63700 1 -- 172.21.15.44:0/1823890051 wait complete. 2020-11-20T12:15:01.511 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:01.511 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:01.511 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:01.512 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:15:01.512 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 120, 2020-11-20T12:15:01.512 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:01.512 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:01.512 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:01.513 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.513 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:15:01.513 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:01.513 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:01.514 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:01.514 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:01.514 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:01.514 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:01.514 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:01.515 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:01.515 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:01.515 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:01.515 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:01.515 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.516 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:01.516 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:01.516 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:01.516 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:01.516 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:01.517 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:01.518 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:01.518 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.519 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:01.519 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:01.519 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:01.519 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:01.520 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:01.520 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:01.520 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:01.520 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:01.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:01.521 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:01.521 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:01.521 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:01.521 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:01.521 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:01.522 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:01.522 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:01.522 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:01.522 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:01.522 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:01.523 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:01.523 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:01.523 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:01.523 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:01.523 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.524 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:01.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.524 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:01.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.525 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:01.525 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:01.525 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:01.526 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:01.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:01.527 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:01.527 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.528 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:01.528 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:01.529 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.529 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.529 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.530 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:01.530 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:01.530 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:01.531 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:01.531 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:01.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.532 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:01.532 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:01.532 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:01.532 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:01.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.533 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:01.533 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:01.533 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:01.534 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:01.534 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.535 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.535 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:01.535 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:01.535 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:01.536 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:01.536 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:01.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.536 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:01.537 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:01.537 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:01.537 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:01.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:01.538 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:01.538 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:01.539 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:01.539 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:01.539 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.539 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.540 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:01.540 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:01.540 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:01.540 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:01.541 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:01.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.541 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.541 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:01.542 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:01.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.542 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.542 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.542 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:01.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.543 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.543 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:15:01.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.544 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.544 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.544 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:15:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:15:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:01.548 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:01.550 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:15:07.947 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:07.970 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:21.551 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:15:21.552 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:15:21.554 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:15:21.555 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:15:21.561 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:15:21.562 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:15:21.562 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:21.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.710+0000 7fea03226700 1 Processor -- start 2020-11-20T12:15:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.710+0000 7fea03226700 1 -- start start 2020-11-20T12:15:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea03226700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe9fc0a9230 0x7fe9fc13fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea03226700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc0a87c0 0x7fe9fc0ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:21.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea03226700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc13f1f0 con 0x7fe9fc142640 2020-11-20T12:15:21.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea03226700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9fc0ab790 con 0x7fe9fc0a9230 2020-11-20T12:15:21.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea03226700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc13f490 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea00fc2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe9fc0a9230 0x7fe9fc13fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc0a87c0 0x7fe9fc0ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc0a87c0 0x7fe9fc0ab2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:52762/0 (socket says 172.21.15.44:52762) 2020-11-20T12:15:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea00fc2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe9fc0a9230 0x7fe9fc13fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33374/0 (socket says 172.21.15.44:33374) 2020-11-20T12:15:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fea017c3700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe9fc142640 0x7fe9fc142ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:47952/0 (socket says 172.21.15.44:47952) 2020-11-20T12:15:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.711+0000 7fe9fbfff700 1 -- 172.21.15.44:0/2294814855 learned_addr learned my addr 172.21.15.44:0/2294814855 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fbfff700 1 -- 172.21.15.44:0/2294814855 >> v2:172.21.15.79:3300/0 conn(0x7fe9fc0a9230 msgr2=0x7fe9fc13fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fbfff700 1 --2- 172.21.15.44:0/2294814855 >> v2:172.21.15.79:3300/0 conn(0x7fe9fc0a9230 0x7fe9fc13fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fbfff700 1 -- 172.21.15.44:0/2294814855 >> v1:172.21.15.79:6789/0 conn(0x7fe9fc142640 legacy=0x7fe9fc142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fb7fe700 1 -- 172.21.15.44:0/2294814855 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562886391 0 0) 0x7fe9fc13f1f0 con 0x7fe9fc142640 2020-11-20T12:15:21.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fbfff700 1 -- 172.21.15.44:0/2294814855 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc13f730 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fea00fc2700 1 --2- 172.21.15.44:0/2294814855 >> v2:172.21.15.79:3300/0 conn(0x7fe9fc0a9230 0x7fe9fc13fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:21.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fbfff700 1 --2- 172.21.15.44:0/2294814855 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc0a87c0 0x7fe9fc0ab2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe9e8006fd0 tx=0x7fe9e8006eb0).ready entity=mon.2 client_cookie=1f88b8f094292aaa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fb7fe700 1 -- 172.21.15.44:0/2294814855 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8009810 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fb7fe700 1 -- 172.21.15.44:0/2294814855 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e8009970 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fe9fb7fe700 1 -- 172.21.15.44:0/2294814855 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8009cf0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fea03226700 1 -- 172.21.15.44:0/2294814855 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc0a87c0 msgr2=0x7fe9fc0ab2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fea03226700 1 --2- 172.21.15.44:0/2294814855 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc0a87c0 0x7fe9fc0ab2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe9e8006fd0 tx=0x7fe9e8006eb0).stop 2020-11-20T12:15:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fea03226700 1 -- 172.21.15.44:0/2294814855 shutdown_connections 2020-11-20T12:15:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fea03226700 1 --2- 172.21.15.44:0/2294814855 >> v2:172.21.15.79:3300/0 conn(0x7fe9fc0a9230 0x7fe9fc13fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fea03226700 1 --2- 172.21.15.44:0/2294814855 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc0a87c0 0x7fe9fc0ab2a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.712+0000 7fea03226700 1 -- 172.21.15.44:0/2294814855 >> 172.21.15.44:0/2294814855 conn(0x7fe9fc0621c0 msgr2=0x7fe9fc060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.713+0000 7fea03226700 1 -- 172.21.15.44:0/2294814855 shutdown_connections 2020-11-20T12:15:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.713+0000 7fea03226700 1 -- 172.21.15.44:0/2294814855 wait complete. 2020-11-20T12:15:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 Processor -- start 2020-11-20T12:15:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 -- start start 2020-11-20T12:15:21.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc0a87c0 0x7fe9fc143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:21.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 0x7fe9fc066fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:21.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc142640 0x7fe9fc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:21.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc065c20 con 0x7fe9fc0a9230 2020-11-20T12:15:21.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc13cde0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fea03226700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc143f30 con 0x7fe9fc142640 2020-11-20T12:15:21.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 0x7fe9fc066fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fe9fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 0x7fe9fc066fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41332/0 (socket says 172.21.15.44:41332) 2020-11-20T12:15:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.714+0000 7fe9fbfff700 1 -- 172.21.15.44:0/2021613252 learned_addr learned my addr 172.21.15.44:0/2021613252 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea00fc2700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc0a87c0 0x7fe9fc143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fe9fbfff700 1 -- 172.21.15.44:0/2021613252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 msgr2=0x7fe9fc066fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:15:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fe9fbfff700 1 -- 172.21.15.44:0/2021613252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 msgr2=0x7fe9fc066fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:15:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fe9fbfff700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 0x7fe9fc066fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fe9fbfff700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 0x7fe9fc066fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea017c3700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc142640 0x7fe9fc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea00fc2700 1 -- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc142640 msgr2=0x7fe9fc0665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:21.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea00fc2700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc142640 0x7fe9fc0665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea00fc2700 1 -- 172.21.15.44:0/2021613252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 msgr2=0x7fe9fc066fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:21.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea00fc2700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 0x7fe9fc066fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea00fc2700 1 -- 172.21.15.44:0/2021613252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc066120 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea00fc2700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc0a87c0 0x7fe9fc143890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe9f000eca0 tx=0x7fe9f0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:21.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fe9f8ff9700 1 -- 172.21.15.44:0/2021613252 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000cb50 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fe9f8ff9700 1 -- 172.21.15.44:0/2021613252 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f000ccb0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc066ab0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fe9f8ff9700 1 -- 172.21.15.44:0/2021613252 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00123f0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.715+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc14e460 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.717+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc14e6b0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.717+0000 7fe9f8ff9700 1 -- 172.21.15.44:0/2021613252 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe9f00127e0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.717+0000 7fe9f8ff9700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe9d4056d20 0x7fe9d40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.717+0000 7fe9f8ff9700 1 -- 172.21.15.44:0/2021613252 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe9f0070710 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.717+0000 7fe9fbfff700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe9d4056d20 0x7fe9d40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.718+0000 7fe9fbfff700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe9d4056d20 0x7fe9d40591e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe9fc0afcd0 tx=0x7fe9e8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:21.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.721+0000 7fe9f8ff9700 1 -- 172.21.15.44:0/2021613252 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe9fc14e6b0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.982+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe9fc14e6b0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.983+0000 7fe9f8ff9700 1 -- 172.21.15.44:0/2021613252 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fe9fc14e6b0 con 0x7fe9fc0a87c0 2020-11-20T12:15:21.986 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":120,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:21.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe9d4056d20 msgr2=0x7fe9d40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:21.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe9d4056d20 0x7fe9d40591e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fe9fc0afcd0 tx=0x7fe9e8015000).stop 2020-11-20T12:15:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc0a87c0 msgr2=0x7fe9fc143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc0a87c0 0x7fe9fc143890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe9f000eca0 tx=0x7fe9f0006df0).stop 2020-11-20T12:15:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 shutdown_connections 2020-11-20T12:15:21.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe9d4056d20 0x7fe9d40591e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe9fc142640 0x7fe9fc0665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc0a87c0 0x7fe9fc143890 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 --2- 172.21.15.44:0/2021613252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9fc0a9230 0x7fe9fc066fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:21.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.984+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 >> 172.21.15.44:0/2021613252 conn(0x7fe9fc0621c0 msgr2=0x7fe9fc060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:21.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.985+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 shutdown_connections 2020-11-20T12:15:21.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:21.985+0000 7fea03226700 1 -- 172.21.15.44:0/2021613252 wait complete. 2020-11-20T12:15:25.001 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:25.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.135+0000 7ff529247700 1 Processor -- start 2020-11-20T12:15:25.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.135+0000 7ff529247700 1 -- start start 2020-11-20T12:15:25.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff529247700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff524142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:25.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff529247700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5240a6260 con 0x7ff524060330 2020-11-20T12:15:25.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff529247700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5240a6490 con 0x7ff52413a990 2020-11-20T12:15:25.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff529247700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240a52e0 con 0x7ff524140810 2020-11-20T12:15:25.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff522d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff524060330 0x7ff5240a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45102/0 (socket says 172.21.15.44:45102) 2020-11-20T12:15:25.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff522d9d700 1 -- 172.21.15.44:0/1900394891 learned_addr learned my addr 172.21.15.44:0/1900394891 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:25.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff52359e700 1 --2- 172.21.15.44:0/1900394891 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff524142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:25.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff521d9b700 1 -- 172.21.15.44:0/1900394891 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871897270 0 0) 0x7ff5240a6490 con 0x7ff52413a990 2020-11-20T12:15:25.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff521d9b700 1 -- 172.21.15.44:0/1900394891 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff510003240 con 0x7ff52413a990 2020-11-20T12:15:25.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff52359e700 1 -- 172.21.15.44:0/1900394891 >> v1:172.21.15.79:6789/0 conn(0x7ff52413a990 legacy=0x7ff52413ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:25.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.136+0000 7ff52359e700 1 -- 172.21.15.44:0/1900394891 >> v1:172.21.15.44:6789/0 conn(0x7ff524060330 legacy=0x7ff5240a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:25.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff52359e700 1 -- 172.21.15.44:0/1900394891 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5240a6730 con 0x7ff524140810 2020-11-20T12:15:25.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff52359e700 1 --2- 172.21.15.44:0/1900394891 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff524142c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff51c00ae60 tx=0x7ff51c00a370).ready entity=mon.2 client_cookie=e8c2c49b72ec2857 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:25.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff521d9b700 1 -- 172.21.15.44:0/1900394891 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00cd90 con 0x7ff524140810 2020-11-20T12:15:25.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff521d9b700 1 -- 172.21.15.44:0/1900394891 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c010410 con 0x7ff524140810 2020-11-20T12:15:25.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff521d9b700 1 -- 172.21.15.44:0/1900394891 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c010790 con 0x7ff524140810 2020-11-20T12:15:25.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff529247700 1 -- 172.21.15.44:0/1900394891 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 msgr2=0x7ff524142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:25.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff529247700 1 --2- 172.21.15.44:0/1900394891 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff524142c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7ff51c00ae60 tx=0x7ff51c00a370).stop 2020-11-20T12:15:25.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff529247700 1 -- 172.21.15.44:0/1900394891 shutdown_connections 2020-11-20T12:15:25.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff529247700 1 --2- 172.21.15.44:0/1900394891 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff524142c00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:25.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff529247700 1 -- 172.21.15.44:0/1900394891 >> 172.21.15.44:0/1900394891 conn(0x7ff524062690 msgr2=0x7ff5240a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:25.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff529247700 1 -- 172.21.15.44:0/1900394891 shutdown_connections 2020-11-20T12:15:25.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.137+0000 7ff529247700 1 -- 172.21.15.44:0/1900394891 wait complete. 2020-11-20T12:15:25.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 Processor -- start 2020-11-20T12:15:25.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 -- start start 2020-11-20T12:15:25.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 0x7ff5240ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:25.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff52413a990 0x7ff5241319b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:25.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff52412fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:25.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240aac60 con 0x7ff524060330 2020-11-20T12:15:25.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff524133b30 con 0x7ff52413a990 2020-11-20T12:15:25.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff529247700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff524131510 con 0x7ff524140810 2020-11-20T12:15:25.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.139+0000 7ff522d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 0x7ff5240ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:25.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff522d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 0x7ff5240ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41360/0 (socket says 172.21.15.44:41360) 2020-11-20T12:15:25.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff522d9d700 1 -- 172.21.15.44:0/3532683809 learned_addr learned my addr 172.21.15.44:0/3532683809 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:25.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff522d9d700 1 -- 172.21.15.44:0/3532683809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 msgr2=0x7ff5240ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:15:25.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff522d9d700 1 -- 172.21.15.44:0/3532683809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 msgr2=0x7ff5240ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:15:25.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff522d9d700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 0x7ff5240ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:25.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff522d9d700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 0x7ff5240ad6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:25.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52259c700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff52413a990 0x7ff5241319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:25.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52359e700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff52412fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:25.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52259c700 1 -- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 msgr2=0x7ff52412fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:25.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52259c700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff52412fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:25.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52259c700 1 -- 172.21.15.44:0/3532683809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 msgr2=0x7ff5240ad6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:25.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52259c700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 0x7ff5240ad6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:25.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52259c700 1 -- 172.21.15.44:0/3532683809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524133f70 con 0x7ff52413a990 2020-11-20T12:15:25.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52359e700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff52412fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:25.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff52259c700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff52413a990 0x7ff5241319b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff51800c6e0 tx=0x7ff518007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:25.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff50affd700 1 -- 172.21.15.44:0/3532683809 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800e490 con 0x7ff52413a990 2020-11-20T12:15:25.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff50affd700 1 -- 172.21.15.44:0/3532683809 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51800e5f0 con 0x7ff52413a990 2020-11-20T12:15:25.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5240ac950 con 0x7ff52413a990 2020-11-20T12:15:25.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff50affd700 1 -- 172.21.15.44:0/3532683809 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800e970 con 0x7ff52413a990 2020-11-20T12:15:25.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.140+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff524131030 con 0x7ff52413a990 2020-11-20T12:15:25.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.141+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff524133cf0 con 0x7ff52413a990 2020-11-20T12:15:25.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.142+0000 7ff50affd700 1 -- 172.21.15.44:0/3532683809 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff518019030 con 0x7ff52413a990 2020-11-20T12:15:25.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.142+0000 7ff50affd700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff500056d20 0x7ff5000591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:25.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.142+0000 7ff50affd700 1 -- 172.21.15.44:0/3532683809 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff51806f490 con 0x7ff52413a990 2020-11-20T12:15:25.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.142+0000 7ff522d9d700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff500056d20 0x7ff5000591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:25.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.143+0000 7ff522d9d700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff500056d20 0x7ff5000591e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7ff514007a50 tx=0x7ff514009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:25.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.146+0000 7ff50affd700 1 -- 172.21.15.44:0/3532683809 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff518043fd0 con 0x7ff52413a990 2020-11-20T12:15:25.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.414+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff524061650 con 0x7ff52413a990 2020-11-20T12:15:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.414+0000 7ff50affd700 1 -- 172.21.15.44:0/3532683809 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7ff518044810 con 0x7ff52413a990 2020-11-20T12:15:25.418 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":120,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:25.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff500056d20 msgr2=0x7ff5000591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:25.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff500056d20 0x7ff5000591e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7ff514007a50 tx=0x7ff514009040).stop 2020-11-20T12:15:25.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff52413a990 msgr2=0x7ff5241319b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:25.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff52413a990 0x7ff5241319b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff51800c6e0 tx=0x7ff518007a20).stop 2020-11-20T12:15:25.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 shutdown_connections 2020-11-20T12:15:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff500056d20 0x7ff5000591e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff524140810 0x7ff52412fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff524060330 0x7ff5240ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 --2- 172.21.15.44:0/3532683809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff52413a990 0x7ff5241319b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:25.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 >> 172.21.15.44:0/3532683809 conn(0x7ff524062690 msgr2=0x7ff524067570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:25.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 shutdown_connections 2020-11-20T12:15:25.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:25.416+0000 7ff529247700 1 -- 172.21.15.44:0/3532683809 wait complete. 2020-11-20T12:15:28.433 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:28.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.570+0000 7fb50c192700 1 Processor -- start 2020-11-20T12:15:28.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.571+0000 7fb50c192700 1 -- start start 2020-11-20T12:15:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.571+0000 7fb50c192700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 0x7fb5040a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.571+0000 7fb50c192700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb5040a4c30 0x7fb50413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.571+0000 7fb50c192700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5040a6260 con 0x7fb50413ae80 2020-11-20T12:15:28.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.571+0000 7fb50c192700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5040a52e0 con 0x7fb504060330 2020-11-20T12:15:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.571+0000 7fb50c192700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5040a6490 con 0x7fb5040a4c30 2020-11-20T12:15:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.571+0000 7fb509f2e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 0x7fb5040a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb509f2e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 0x7fb5040a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41372/0 (socket says 172.21.15.44:41372) 2020-11-20T12:15:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb509f2e700 1 -- 172.21.15.44:0/3217809138 learned_addr learned my addr 172.21.15.44:0/3217809138 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:28.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb509f2e700 1 -- 172.21.15.44:0/3217809138 >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 msgr2=0x7fb5040a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:15:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb509f2e700 1 -- 172.21.15.44:0/3217809138 >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 msgr2=0x7fb5040a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb509f2e700 1 --2- 172.21.15.44:0/3217809138 >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 0x7fb5040a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb50972d700 1 --2- 172.21.15.44:0/3217809138 >> v2:172.21.15.79:3300/0 conn(0x7fb5040a4c30 0x7fb50413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb509f2e700 1 --2- 172.21.15.44:0/3217809138 >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 0x7fb5040a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb50972d700 1 -- 172.21.15.44:0/3217809138 >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 msgr2=0x7fb5040a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:28.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb50972d700 1 --2- 172.21.15.44:0/3217809138 >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 0x7fb5040a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb50972d700 1 -- 172.21.15.44:0/3217809138 >> v1:172.21.15.79:6789/0 conn(0x7fb50413ae80 legacy=0x7fb504136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:28.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb50972d700 1 -- 172.21.15.44:0/3217809138 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040a6730 con 0x7fb5040a4c30 2020-11-20T12:15:28.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb508f2c700 1 -- 172.21.15.44:0/3217809138 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648214405 0 0) 0x7fb5040a6260 con 0x7fb50413ae80 2020-11-20T12:15:28.580 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb50972d700 1 --2- 172.21.15.44:0/3217809138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb50413a990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fb5040a3a60 tx=0x7fb4f4006df0).ready entity=mon.1 client_cookie=ef766c11518bf953 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:28.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb508f2c700 1 -- 172.21.15.44:0/3217809138 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40097b0 con 0x7fb5040a4c30 2020-11-20T12:15:28.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb508f2c700 1 -- 172.21.15.44:0/3217809138 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4009910 con 0x7fb5040a4c30 2020-11-20T12:15:28.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.572+0000 7fb508f2c700 1 -- 172.21.15.44:0/3217809138 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4009c90 con 0x7fb5040a4c30 2020-11-20T12:15:28.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 -- 172.21.15.44:0/3217809138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 msgr2=0x7fb50413a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:28.581 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 --2- 172.21.15.44:0/3217809138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb50413a990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fb5040a3a60 tx=0x7fb4f4006df0).stop 2020-11-20T12:15:28.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 -- 172.21.15.44:0/3217809138 shutdown_connections 2020-11-20T12:15:28.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 --2- 172.21.15.44:0/3217809138 >> v2:172.21.15.44:3300/0 conn(0x7fb504060330 0x7fb5040a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 --2- 172.21.15.44:0/3217809138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb50413a990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 -- 172.21.15.44:0/3217809138 >> 172.21.15.44:0/3217809138 conn(0x7fb504062690 msgr2=0x7fb5040a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:28.582 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 -- 172.21.15.44:0/3217809138 shutdown_connections 2020-11-20T12:15:28.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.573+0000 7fb50c192700 1 -- 172.21.15.44:0/3217809138 wait complete. 2020-11-20T12:15:28.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.574+0000 7fb50c192700 1 Processor -- start 2020-11-20T12:15:28.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.574+0000 7fb50c192700 1 -- start start 2020-11-20T12:15:28.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50c192700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb504060330 0x7fb504134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:28.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50c192700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb5040acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:28.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50c192700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 0x7fb504136190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:28.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50c192700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb504135940 con 0x7fb50413ae80 2020-11-20T12:15:28.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50c192700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50409d510 con 0x7fb5040a4c30 2020-11-20T12:15:28.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50c192700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50409d7b0 con 0x7fb504060330 2020-11-20T12:15:28.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50a72f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 0x7fb504136190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:28.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50a72f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 0x7fb504136190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41382/0 (socket says 172.21.15.44:41382) 2020-11-20T12:15:28.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50a72f700 1 -- 172.21.15.44:0/2668297678 learned_addr learned my addr 172.21.15.44:0/2668297678 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:28.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50972d700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb5040acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:28.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb509f2e700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb504060330 0x7fb504134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:28.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50a72f700 1 -- 172.21.15.44:0/2668297678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 msgr2=0x7fb504136190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:15:28.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50a72f700 1 -- 172.21.15.44:0/2668297678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 msgr2=0x7fb504136190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:15:28.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50a72f700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 0x7fb504136190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:28.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50a72f700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 0x7fb504136190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:28.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50972d700 1 -- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb504060330 msgr2=0x7fb504134ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:28.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50972d700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb504060330 0x7fb504134ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50972d700 1 -- 172.21.15.44:0/2668297678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 msgr2=0x7fb504136190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:28.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50972d700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 0x7fb504136190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb50972d700 1 -- 172.21.15.44:0/2668297678 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040a34e0 con 0x7fb5040a4c30 2020-11-20T12:15:28.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.575+0000 7fb509f2e700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb504060330 0x7fb504134ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:28.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.576+0000 7fb50972d700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb5040acfc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb504061250 tx=0x7fb4f4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:28.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.576+0000 7fb4f1ffb700 1 -- 172.21.15.44:0/2668297678 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f401e460 con 0x7fb5040a4c30 2020-11-20T12:15:28.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.576+0000 7fb4f1ffb700 1 -- 172.21.15.44:0/2668297678 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f401e5c0 con 0x7fb5040a4c30 2020-11-20T12:15:28.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.576+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb504137ef0 con 0x7fb5040a4c30 2020-11-20T12:15:28.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.576+0000 7fb4f1ffb700 1 -- 172.21.15.44:0/2668297678 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f401e940 con 0x7fb5040a4c30 2020-11-20T12:15:28.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.576+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb504136680 con 0x7fb5040a4c30 2020-11-20T12:15:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.577+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5040a3670 con 0x7fb5040a4c30 2020-11-20T12:15:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.578+0000 7fb4f1ffb700 1 -- 172.21.15.44:0/2668297678 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb4f401a040 con 0x7fb5040a4c30 2020-11-20T12:15:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.578+0000 7fb4f1ffb700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb4dc056d20 0x7fb4dc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.578+0000 7fb4f1ffb700 1 -- 172.21.15.44:0/2668297678 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb4f4071ec0 con 0x7fb5040a4c30 2020-11-20T12:15:28.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.578+0000 7fb509f2e700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb4dc056d20 0x7fb4dc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:28.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.579+0000 7fb509f2e700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb4dc056d20 0x7fb4dc0591e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fb5041448e0 tx=0x7fb500006f90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:28.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.582+0000 7fb4f1ffb700 1 -- 172.21.15.44:0/2668297678 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb5040a3670 con 0x7fb5040a4c30 2020-11-20T12:15:28.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.852+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb5040a3670 con 0x7fb5040a4c30 2020-11-20T12:15:28.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.853+0000 7fb4f1ffb700 1 -- 172.21.15.44:0/2668297678 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fb5040a3670 con 0x7fb5040a4c30 2020-11-20T12:15:28.856 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":120,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.854+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb4dc056d20 msgr2=0x7fb4dc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.854+0000 7fb50c192700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb4dc056d20 0x7fb4dc0591e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fb5041448e0 tx=0x7fb500006f90).stop 2020-11-20T12:15:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.854+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 msgr2=0x7fb5040acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:28.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.854+0000 7fb50c192700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb5040acfc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb504061250 tx=0x7fb4f4015000).stop 2020-11-20T12:15:28.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 shutdown_connections 2020-11-20T12:15:28.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb4dc056d20 0x7fb4dc0591e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50413ae80 0x7fb504136190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb504060330 0x7fb504134ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 --2- 172.21.15.44:0/2668297678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5040a4c30 0x7fb5040acfc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 >> 172.21.15.44:0/2668297678 conn(0x7fb504062690 msgr2=0x7fb50405ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 shutdown_connections 2020-11-20T12:15:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:28.855+0000 7fb50c192700 1 -- 172.21.15.44:0/2668297678 wait complete. 2020-11-20T12:15:31.870 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.748+0000 7f2100cb8700 1 Processor -- start 2020-11-20T12:15:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.748+0000 7f2100cb8700 1 -- start start 2020-11-20T12:15:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.748+0000 7f2100cb8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad730 0x7f20fc0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.748+0000 7f2100cb8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f20fc0a40d0 0x7f20fc1379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.748+0000 7f2100cb8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0a4650 con 0x7f20fc0ad350 2020-11-20T12:15:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.748+0000 7f2100cb8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc135090 con 0x7f20fc0a40d0 2020-11-20T12:15:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.748+0000 7f2100cb8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0a63b0 con 0x7f20fc0ad730 2020-11-20T12:15:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20faffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad730 0x7f20fc0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20fb7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f20fc0ad350 0x7f20fc0676f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:48010/0 (socket says 172.21.15.44:48010) 2020-11-20T12:15:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20fb7fe700 1 -- 172.21.15.44:0/1157076848 learned_addr learned my addr 172.21.15.44:0/1157076848 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20fbfff700 1 --2- 172.21.15.44:0/1157076848 >> v2:172.21.15.79:3300/0 conn(0x7f20fc0a40d0 0x7f20fc1379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20faffd700 1 -- 172.21.15.44:0/1157076848 >> v2:172.21.15.79:3300/0 conn(0x7f20fc0a40d0 msgr2=0x7f20fc1379f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20faffd700 1 --2- 172.21.15.44:0/1157076848 >> v2:172.21.15.79:3300/0 conn(0x7f20fc0a40d0 0x7f20fc1379f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20faffd700 1 -- 172.21.15.44:0/1157076848 >> v1:172.21.15.79:6789/0 conn(0x7f20fc0ad350 legacy=0x7f20fc0676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20faffd700 1 -- 172.21.15.44:0/1157076848 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc0a48f0 con 0x7f20fc0ad730 2020-11-20T12:15:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20faffd700 1 --2- 172.21.15.44:0/1157076848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad730 0x7f20fc0a5680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f20f000b9a0 tx=0x7f20f0006df0).ready entity=mon.2 client_cookie=375500b77df0d799 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20fa7fc700 1 -- 172.21.15.44:0/1157076848 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00097b0 con 0x7f20fc0ad730 2020-11-20T12:15:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.749+0000 7f20fa7fc700 1 -- 172.21.15.44:0/1157076848 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f0009910 con 0x7f20fc0ad730 2020-11-20T12:15:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f20fa7fc700 1 -- 172.21.15.44:0/1157076848 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0009c90 con 0x7f20fc0ad730 2020-11-20T12:15:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 -- 172.21.15.44:0/1157076848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad730 msgr2=0x7f20fc0a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 --2- 172.21.15.44:0/1157076848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad730 0x7f20fc0a5680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f20f000b9a0 tx=0x7f20f0006df0).stop 2020-11-20T12:15:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 -- 172.21.15.44:0/1157076848 shutdown_connections 2020-11-20T12:15:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 --2- 172.21.15.44:0/1157076848 >> v2:172.21.15.79:3300/0 conn(0x7f20fc0a40d0 0x7f20fc1379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 --2- 172.21.15.44:0/1157076848 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad730 0x7f20fc0a5680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 -- 172.21.15.44:0/1157076848 >> 172.21.15.44:0/1157076848 conn(0x7f20fc061b60 msgr2=0x7f20fc061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 -- 172.21.15.44:0/1157076848 shutdown_connections 2020-11-20T12:15:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.750+0000 7f2100cb8700 1 -- 172.21.15.44:0/1157076848 wait complete. 2020-11-20T12:15:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.751+0000 7f2100cb8700 1 Processor -- start 2020-11-20T12:15:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.751+0000 7f2100cb8700 1 -- start start 2020-11-20T12:15:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f2100cb8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20fc0a40d0 0x7f20fc138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f2100cb8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad350 0x7f20fc145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f2100cb8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 0x7f20fc1462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f2100cb8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc066b60 con 0x7f20fc0ad730 2020-11-20T12:15:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f2100cb8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc067240 con 0x7f20fc0a40d0 2020-11-20T12:15:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f2100cb8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc1467b0 con 0x7f20fc0ad350 2020-11-20T12:15:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20fc0a40d0 0x7f20fc138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 0x7f20fc1462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fb7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20fc0a40d0 0x7f20fc138f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33442/0 (socket says 172.21.15.44:33442) 2020-11-20T12:15:32.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 0x7f20fc1462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41396/0 (socket says 172.21.15.44:41396) 2020-11-20T12:15:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fb7fe700 1 -- 172.21.15.44:0/581189657 learned_addr learned my addr 172.21.15.44:0/581189657 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20faffd700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad350 0x7f20fc145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fbfff700 1 -- 172.21.15.44:0/581189657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 msgr2=0x7f20fc1462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:15:32.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fbfff700 1 -- 172.21.15.44:0/581189657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 msgr2=0x7f20fc1462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:15:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fbfff700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 0x7f20fc1462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fbfff700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 0x7f20fc1462c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20faffd700 1 -- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20fc0a40d0 msgr2=0x7f20fc138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:32.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20faffd700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20fc0a40d0 0x7f20fc138f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:32.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20faffd700 1 -- 172.21.15.44:0/581189657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 msgr2=0x7f20fc1462c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:32.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20faffd700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 0x7f20fc1462c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:32.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20faffd700 1 -- 172.21.15.44:0/581189657 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc1395c0 con 0x7f20fc0ad350 2020-11-20T12:15:32.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.752+0000 7f20fb7fe700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20fc0a40d0 0x7f20fc138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:32.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.753+0000 7f20faffd700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad350 0x7f20fc145dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f20f0012000 tx=0x7f20f0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:32.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.753+0000 7f20db7fe700 1 -- 172.21.15.44:0/581189657 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0011c80 con 0x7f20fc0ad350 2020-11-20T12:15:32.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.753+0000 7f20db7fe700 1 -- 172.21.15.44:0/581189657 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f0011de0 con 0x7f20fc0ad350 2020-11-20T12:15:32.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.753+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc139860 con 0x7f20fc0ad350 2020-11-20T12:15:32.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.753+0000 7f20db7fe700 1 -- 172.21.15.44:0/581189657 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f001e670 con 0x7f20fc0ad350 2020-11-20T12:15:32.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.753+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc139fa0 con 0x7f20fc0ad350 2020-11-20T12:15:32.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.754+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc145b10 con 0x7f20fc0ad350 2020-11-20T12:15:32.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.754+0000 7f20db7fe700 1 -- 172.21.15.44:0/581189657 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f20f0025030 con 0x7f20fc0ad350 2020-11-20T12:15:32.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.754+0000 7f20db7fe700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20d4056d20 0x7f20d40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:32.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.754+0000 7f20db7fe700 1 -- 172.21.15.44:0/581189657 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f20f0071850 con 0x7f20fc0ad350 2020-11-20T12:15:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.754+0000 7f20fb7fe700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20d4056d20 0x7f20d40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.755+0000 7f20fb7fe700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20d4056d20 0x7f20d40591e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f20ec0016f0 tx=0x7f20ec007c40).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:32.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:32.758+0000 7f20db7fe700 1 -- 172.21.15.44:0/581189657 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f20fc145b10 con 0x7f20fc0ad350 2020-11-20T12:15:33.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.068+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f20fc145b10 con 0x7f20fc0ad350 2020-11-20T12:15:33.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.069+0000 7f20db7fe700 1 -- 172.21.15.44:0/581189657 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f20fc145b10 con 0x7f20fc0ad350 2020-11-20T12:15:33.072 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":120,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":32,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:33.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20d4056d20 msgr2=0x7f20d40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:33.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20d4056d20 0x7f20d40591e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f20ec0016f0 tx=0x7f20ec007c40).stop 2020-11-20T12:15:33.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad350 msgr2=0x7f20fc145dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:33.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad350 0x7f20fc145dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f20f0012000 tx=0x7f20f0015000).stop 2020-11-20T12:15:33.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 shutdown_connections 2020-11-20T12:15:33.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f20d4056d20 0x7f20d40591e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:33.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20fc0ad730 0x7f20fc1462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:33.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20fc0a40d0 0x7f20fc138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:33.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 --2- 172.21.15.44:0/581189657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f20fc0ad350 0x7f20fc145dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:33.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 >> 172.21.15.44:0/581189657 conn(0x7f20fc061b60 msgr2=0x7f20fc05ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:33.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 shutdown_connections 2020-11-20T12:15:33.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:33.071+0000 7f2100cb8700 1 -- 172.21.15.44:0/581189657 wait complete. 2020-11-20T12:15:36.087 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:36.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.218+0000 7f5e5198b700 1 Processor -- start 2020-11-20T12:15:36.221 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.218+0000 7f5e5198b700 1 -- start start 2020-11-20T12:15:36.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.218+0000 7f5e5198b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5e4c0b1bb0 0x7f5e4c0a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e5198b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5e4c0a7d90 0x7f5e4c145460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e5198b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4c0aa810 con 0x7f5e4c0b17d0 2020-11-20T12:15:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e5198b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c13e120 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e5198b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c13e310 con 0x7f5e4c0a7d90 2020-11-20T12:15:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5e4c0b1bb0 0x7f5e4c0a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.224 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e50989700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5e4c0b17d0 0x7f5e4c13ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45152/0 (socket says 172.21.15.44:45152) 2020-11-20T12:15:36.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e50989700 1 -- 172.21.15.44:0/3864190334 learned_addr learned my addr 172.21.15.44:0/3864190334 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:36.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e5118a700 1 --2- 172.21.15.44:0/3864190334 >> v2:172.21.15.79:3300/0 conn(0x7f5e4c0a7d90 0x7f5e4c145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4bfff700 1 -- 172.21.15.44:0/3864190334 >> v2:172.21.15.79:3300/0 conn(0x7f5e4c0a7d90 msgr2=0x7f5e4c145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:36.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4bfff700 1 --2- 172.21.15.44:0/3864190334 >> v2:172.21.15.79:3300/0 conn(0x7f5e4c0a7d90 0x7f5e4c145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4bfff700 1 -- 172.21.15.44:0/3864190334 >> v1:172.21.15.44:6789/0 conn(0x7f5e4c0b17d0 legacy=0x7f5e4c13ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4bfff700 1 -- 172.21.15.44:0/3864190334 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c13e5b0 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4b7fe700 1 -- 172.21.15.44:0/3864190334 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406290666 0 0) 0x7f5e4c0aa810 con 0x7f5e4c0b17d0 2020-11-20T12:15:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e5118a700 1 --2- 172.21.15.44:0/3864190334 >> v2:172.21.15.79:3300/0 conn(0x7f5e4c0a7d90 0x7f5e4c145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:36.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4bfff700 1 --2- 172.21.15.44:0/3864190334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0a78a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5e4c0a9d20 tx=0x7f5e38006df0).ready entity=mon.0 client_cookie=819f6647896c77ac server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4b7fe700 1 -- 172.21.15.44:0/3864190334 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380097b0 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4b7fe700 1 -- 172.21.15.44:0/3864190334 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38009910 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.219+0000 7f5e4b7fe700 1 -- 172.21.15.44:0/3864190334 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38009c90 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 -- 172.21.15.44:0/3864190334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 msgr2=0x7f5e4c0a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3864190334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0a78a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5e4c0a9d20 tx=0x7f5e38006df0).stop 2020-11-20T12:15:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 -- 172.21.15.44:0/3864190334 shutdown_connections 2020-11-20T12:15:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3864190334 >> v2:172.21.15.79:3300/0 conn(0x7f5e4c0a7d90 0x7f5e4c145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3864190334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0a78a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 -- 172.21.15.44:0/3864190334 >> 172.21.15.44:0/3864190334 conn(0x7f5e4c060c40 msgr2=0x7f5e4c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:36.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 -- 172.21.15.44:0/3864190334 shutdown_connections 2020-11-20T12:15:36.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.220+0000 7f5e5198b700 1 -- 172.21.15.44:0/3864190334 wait complete. 2020-11-20T12:15:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.221+0000 7f5e5198b700 1 Processor -- start 2020-11-20T12:15:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.221+0000 7f5e5198b700 1 -- start start 2020-11-20T12:15:36.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e5198b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e4c0b17d0 0x7f5e4c09ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e5198b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e5198b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e4c09fb70 0x7f5e4c0a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e5198b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0a00d0 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e5198b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c09f080 con 0x7f5e4c09fb70 2020-11-20T12:15:36.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e5198b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0b0c70 con 0x7f5e4c0b17d0 2020-11-20T12:15:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41412/0 (socket says 172.21.15.44:41412) 2020-11-20T12:15:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 -- 172.21.15.44:0/3199843201 learned_addr learned my addr 172.21.15.44:0/3199843201 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:36.233 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e5118a700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e4c09fb70 0x7f5e4c0a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e50989700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e4c0b17d0 0x7f5e4c09ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 -- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e4c0b17d0 msgr2=0x7f5e4c09ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e4c0b17d0 0x7f5e4c09ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 -- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e4c09fb70 msgr2=0x7f5e4c0a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e4c09fb70 0x7f5e4c0a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.222+0000 7f5e4bfff700 1 -- 172.21.15.44:0/3199843201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0a0330 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.223+0000 7f5e4bfff700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0b0060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5e3800c010 tx=0x7f5e38015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.223+0000 7f5e48ff9700 1 -- 172.21.15.44:0/3199843201 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38011d50 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.223+0000 7f5e48ff9700 1 -- 172.21.15.44:0/3199843201 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3801e460 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.235 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.223+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c0a1f80 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.223+0000 7f5e48ff9700 1 -- 172.21.15.44:0/3199843201 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3801e7e0 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.223+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c134190 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.224+0000 7f5e48ff9700 1 -- 172.21.15.44:0/3199843201 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f5e38025030 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.224+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c0616c0 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.225+0000 7f5e48ff9700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e24056d20 0x7f5e240591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.225+0000 7f5e48ff9700 1 -- 172.21.15.44:0/3199843201 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f5e38071810 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.225+0000 7f5e50989700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e24056d20 0x7f5e240591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.237 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.225+0000 7f5e50989700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e24056d20 0x7f5e240591e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f5e400011a0 tx=0x7f5e40007a00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.228+0000 7f5e48ff9700 1 -- 172.21.15.44:0/3199843201 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5e38046b90 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.488+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5e4c0a04c0 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.488+0000 7f5e48ff9700 1 -- 172.21.15.44:0/3199843201 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f5e38023020 con 0x7f5e4c0b1bb0 2020-11-20T12:15:36.492 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:36.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e24056d20 msgr2=0x7f5e240591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e24056d20 0x7f5e240591e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f5e400011a0 tx=0x7f5e40007a00).stop 2020-11-20T12:15:36.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 msgr2=0x7f5e4c0b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0b0060 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5e3800c010 tx=0x7f5e38015000).stop 2020-11-20T12:15:36.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 shutdown_connections 2020-11-20T12:15:36.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5e24056d20 0x7f5e240591e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5e4c09fb70 0x7f5e4c0a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5e4c0b17d0 0x7f5e4c09ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 --2- 172.21.15.44:0/3199843201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e4c0b1bb0 0x7f5e4c0b0060 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 >> 172.21.15.44:0/3199843201 conn(0x7f5e4c060c40 msgr2=0x7f5e4c13c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:36.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 shutdown_connections 2020-11-20T12:15:36.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.490+0000 7f5e5198b700 1 -- 172.21.15.44:0/3199843201 wait complete. 2020-11-20T12:15:36.505 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:15:36.506 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:15:36.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.638+0000 7eff67a2b700 1 Processor -- start 2020-11-20T12:15:36.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.638+0000 7eff67a2b700 1 -- start start 2020-11-20T12:15:36.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff67a2b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff6013c200 0x7eff6013c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff67a2b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7eff60135530 0x7eff601359c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff67a2b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff600a7b90 con 0x7eff60061340 2020-11-20T12:15:36.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff67a2b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff60064f60 con 0x7eff60135530 2020-11-20T12:15:36.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff67a2b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600687d0 con 0x7eff6013c200 2020-11-20T12:15:36.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff65fc8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7eff60135530 0x7eff601359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff65fc8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7eff60135530 0x7eff601359c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41422/0 (socket says 172.21.15.44:41422) 2020-11-20T12:15:36.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff65fc8700 1 -- 172.21.15.44:0/463197960 learned_addr learned my addr 172.21.15.44:0/463197960 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:36.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff64fc6700 1 --2- 172.21.15.44:0/463197960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff6013c200 0x7eff6013c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff65fc8700 1 -- 172.21.15.44:0/463197960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff6013c200 msgr2=0x7eff6013c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:36.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff65fc8700 1 --2- 172.21.15.44:0/463197960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff6013c200 0x7eff6013c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff65fc8700 1 -- 172.21.15.44:0/463197960 >> v1:172.21.15.79:6789/0 conn(0x7eff60061340 legacy=0x7eff600a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.639+0000 7eff65fc8700 1 -- 172.21.15.44:0/463197960 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6013cb70 con 0x7eff60135530 2020-11-20T12:15:36.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff65fc8700 1 --2- 172.21.15.44:0/463197960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff601359c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7eff58009d50 tx=0x7eff58009290).ready entity=mon.0 client_cookie=d8a4ebb3044c298 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff4ffff700 1 -- 172.21.15.44:0/463197960 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800bbc0 con 0x7eff60135530 2020-11-20T12:15:36.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff4ffff700 1 -- 172.21.15.44:0/463197960 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5800bd20 con 0x7eff60135530 2020-11-20T12:15:36.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff4ffff700 1 -- 172.21.15.44:0/463197960 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58011610 con 0x7eff60135530 2020-11-20T12:15:36.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff67a2b700 1 -- 172.21.15.44:0/463197960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 msgr2=0x7eff601359c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff67a2b700 1 --2- 172.21.15.44:0/463197960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff601359c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7eff58009d50 tx=0x7eff58009290).stop 2020-11-20T12:15:36.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff67a2b700 1 -- 172.21.15.44:0/463197960 shutdown_connections 2020-11-20T12:15:36.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff67a2b700 1 --2- 172.21.15.44:0/463197960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff601359c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff67a2b700 1 --2- 172.21.15.44:0/463197960 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff6013c200 0x7eff6013c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.640+0000 7eff67a2b700 1 -- 172.21.15.44:0/463197960 >> 172.21.15.44:0/463197960 conn(0x7eff600ad6e0 msgr2=0x7eff600adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:36.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.641+0000 7eff67a2b700 1 -- 172.21.15.44:0/463197960 shutdown_connections 2020-11-20T12:15:36.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.641+0000 7eff67a2b700 1 -- 172.21.15.44:0/463197960 wait complete. 2020-11-20T12:15:36.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 Processor -- start 2020-11-20T12:15:36.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 -- start start 2020-11-20T12:15:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff60061340 0x7eff600a3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff60136ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff6013c200 0x7eff60149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600a2690 con 0x7eff60135530 2020-11-20T12:15:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff60136fb0 con 0x7eff6013c200 2020-11-20T12:15:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.642+0000 7eff67a2b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600a2c80 con 0x7eff60061340 2020-11-20T12:15:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff60136ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff60136ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41426/0 (socket says 172.21.15.44:41426) 2020-11-20T12:15:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 -- 172.21.15.44:0/2780285821 learned_addr learned my addr 172.21.15.44:0/2780285821 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:36.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff65fc8700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff6013c200 0x7eff60149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff657c7700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff60061340 0x7eff600a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff60061340 msgr2=0x7eff600a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff60061340 0x7eff600a3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff6013c200 msgr2=0x7eff60149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff6013c200 0x7eff60149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 -- 172.21.15.44:0/2780285821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600a1cb0 con 0x7eff60135530 2020-11-20T12:15:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff65fc8700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff6013c200 0x7eff60149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff657c7700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7eff60061340 0x7eff600a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff64fc6700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff60136ac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7eff5000ebd0 tx=0x7eff50006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000b9a0 con 0x7eff60135530 2020-11-20T12:15:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000bb00 con 0x7eff60135530 2020-11-20T12:15:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff600a1f50 con 0x7eff60135530 2020-11-20T12:15:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000be80 con 0x7eff60135530 2020-11-20T12:15:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.643+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff600a2250 con 0x7eff60135530 2020-11-20T12:15:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff67a2b700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff600619a0 0x7eff60134880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600a20e0 con 0x7eff600619a0 2020-11-20T12:15:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7eff50012600 con 0x7eff60135530 2020-11-20T12:15:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff657c7700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff600619a0 0x7eff60134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff4d7fa700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff38056d70 0x7eff38059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff657c7700 1 -- 172.21.15.44:0/2780285821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff600a23f0 con 0x7eff600619a0 2020-11-20T12:15:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7eff500702a0 con 0x7eff60135530 2020-11-20T12:15:36.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff65fc8700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff38056d70 0x7eff38059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.645+0000 7eff657c7700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff600619a0 0x7eff60134880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7eff5c007fd0 tx=0x7eff5c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.646+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c007240 con 0x7eff600619a0 2020-11-20T12:15:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.646+0000 7eff65fc8700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff38056d70 0x7eff38059230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7eff600a4890 tx=0x7eff58016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.646+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7eff5c0073a0 con 0x7eff600619a0 2020-11-20T12:15:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.646+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff600619a0 msgr2=0x7eff60134880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.646+0000 7eff4d7fa700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff600619a0 0x7eff60134880 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7eff5c007fd0 tx=0x7eff5c009040).stop 2020-11-20T12:15:36.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.646+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff600619a0 msgr2=0x7eff60134880 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:36.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.646+0000 7eff4d7fa700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff600619a0 0x7eff60134880 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.655+0000 7eff67a2b700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6014a0d0 0x7eff60134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.655+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600a2250 con 0x7eff6014a0d0 2020-11-20T12:15:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.655+0000 7eff657c7700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6014a0d0 0x7eff60134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.655+0000 7eff657c7700 1 -- 172.21.15.44:0/2780285821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7eff600446d0 con 0x7eff6014a0d0 2020-11-20T12:15:36.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.655+0000 7eff657c7700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6014a0d0 0x7eff60134d70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7eff5c007fd0 tx=0x7eff5c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.655+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01d040 con 0x7eff6014a0d0 2020-11-20T12:15:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4268 (secure 0 0 0) 0x7eff5c007030 con 0x7eff6014a0d0 2020-11-20T12:15:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6014a0d0 msgr2=0x7eff60134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff4d7fa700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6014a0d0 0x7eff60134d70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7eff5c007fd0 tx=0x7eff5c009040).stop 2020-11-20T12:15:36.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff4d7fa700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6014a0d0 msgr2=0x7eff60134d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff4d7fa700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff6014a0d0 0x7eff60134d70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff38056d70 msgr2=0x7eff38059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7eff38056d70 0x7eff38059230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7eff600a4890 tx=0x7eff58016000).stop 2020-11-20T12:15:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 msgr2=0x7eff60136ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:36.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 --2- 172.21.15.44:0/2780285821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff60135530 0x7eff60136ac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7eff5000ebd0 tx=0x7eff50006df0).stop 2020-11-20T12:15:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff65fc8700 1 -- 172.21.15.44:0/2780285821 reap_dead start 2020-11-20T12:15:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff65fc8700 1 -- 172.21.15.44:0/2780285821 reap_dead start 2020-11-20T12:15:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 shutdown_connections 2020-11-20T12:15:36.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 >> 172.21.15.44:0/2780285821 conn(0x7eff600ad6e0 msgr2=0x7eff6013a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:36.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 shutdown_connections 2020-11-20T12:15:36.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.656+0000 7eff67a2b700 1 -- 172.21.15.44:0/2780285821 wait complete. 2020-11-20T12:15:36.675 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:36.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:15:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 124, 2020-11-20T12:15:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:36.676 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:15:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:36.677 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:15:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:36.678 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:36.679 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:36.680 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:36.681 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:36.682 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.683 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:36.684 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:36.685 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:36.686 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:36.687 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:36.688 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:36.689 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:36.690 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:36.691 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:36.692 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:36.693 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:36.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:36.695 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:36.696 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:36.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:36.697 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:36.697 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:36.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.698 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:36.699 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:36.699 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:36.700 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:36.700 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:36.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.701 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:36.701 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:36.702 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:36.702 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:36.703 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:36.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.704 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:36.704 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:36.705 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:36.705 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:36.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:36.706 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:36.707 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:36.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:36.708 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:36.709 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:36.709 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:36.709 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:36.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.710 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:36.711 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:36.711 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:36.711 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:36.712 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:36.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.713 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:36.714 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:36.714 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:36.714 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:36.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.715 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:36.716 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:36.716 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:36.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:36.717 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:36.718 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:36.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:36.719 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:36.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.720 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:36.720 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:36.720 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:36.721 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:36.721 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:36.722 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:36.722 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:36.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.723 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:36.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:36.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.724 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:36.724 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:36.725 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:36.725 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:36.726 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:36.726 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:36.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:36.727 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:36.727 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:36.728 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:15:36.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:36.728 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:36.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:36.729 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:36.730 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:36.732 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:15:36.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.874+0000 7fe2b1d16700 1 Processor -- start 2020-11-20T12:15:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.874+0000 7fe2b1d16700 1 -- start start 2020-11-20T12:15:37.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.874+0000 7fe2b1d16700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac0aa020 0x7fe2ac0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2b1d16700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac0a9770 con 0x7fe2ac0aa400 2020-11-20T12:15:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2b1d16700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac0a6da0 con 0x7fe2ac13ac60 2020-11-20T12:15:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2b1d16700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0a56c0 con 0x7fe2ac0aa020 2020-11-20T12:15:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2aaffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe2ac0aa400 0x7fe2ac0a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45184/0 (socket says 172.21.15.44:45184) 2020-11-20T12:15:37.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2aaffd700 1 -- 172.21.15.44:0/3236771382 learned_addr learned my addr 172.21.15.44:0/3236771382 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/3236771382 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac0aa020 0x7fe2ac0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3236771382 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540665234 0 0) 0x7fe2a00010d0 con 0x7fe2ac0aa400 2020-11-20T12:15:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3236771382 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe298003240 con 0x7fe2ac0aa400 2020-11-20T12:15:37.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3236771382 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841025879 0 0) 0x7fe2ac0a6da0 con 0x7fe2ac13ac60 2020-11-20T12:15:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3236771382 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a00010d0 con 0x7fe2ac13ac60 2020-11-20T12:15:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2ab7fe700 1 -- 172.21.15.44:0/3236771382 >> v1:172.21.15.79:6789/0 conn(0x7fe2ac13ac60 legacy=0x7fe2ac0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2ab7fe700 1 -- 172.21.15.44:0/3236771382 >> v1:172.21.15.44:6789/0 conn(0x7fe2ac0aa400 legacy=0x7fe2ac0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.875+0000 7fe2ab7fe700 1 -- 172.21.15.44:0/3236771382 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0a7040 con 0x7fe2ac0aa020 2020-11-20T12:15:37.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/3236771382 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac0aa020 0x7fe2ac0a9280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe29c00b9d0 tx=0x7fe29c006df0).ready entity=mon.2 client_cookie=d73c67e7a0a4ecaf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3236771382 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c009680 con 0x7fe2ac0aa020 2020-11-20T12:15:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3236771382 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c0097e0 con 0x7fe2ac0aa020 2020-11-20T12:15:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2aa7fc700 1 -- 172.21.15.44:0/3236771382 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c009b60 con 0x7fe2ac0aa020 2020-11-20T12:15:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2b1d16700 1 -- 172.21.15.44:0/3236771382 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac0aa020 msgr2=0x7fe2ac0a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2b1d16700 1 --2- 172.21.15.44:0/3236771382 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac0aa020 0x7fe2ac0a9280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe29c00b9d0 tx=0x7fe29c006df0).stop 2020-11-20T12:15:37.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2b1d16700 1 -- 172.21.15.44:0/3236771382 shutdown_connections 2020-11-20T12:15:37.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2b1d16700 1 --2- 172.21.15.44:0/3236771382 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac0aa020 0x7fe2ac0a9280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:37.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2b1d16700 1 -- 172.21.15.44:0/3236771382 >> 172.21.15.44:0/3236771382 conn(0x7fe2ac062690 msgr2=0x7fe2ac0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:37.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2b1d16700 1 -- 172.21.15.44:0/3236771382 shutdown_connections 2020-11-20T12:15:37.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.876+0000 7fe2b1d16700 1 -- 172.21.15.44:0/3236771382 wait complete. 2020-11-20T12:15:37.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.877+0000 7fe2b1d16700 1 Processor -- start 2020-11-20T12:15:37.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.877+0000 7fe2b1d16700 1 -- start start 2020-11-20T12:15:37.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2b1d16700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac0aa020 0x7fe2ac0ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:37.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2b1d16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0aa400 0x7fe2ac0acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:37.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2b1d16700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac13ac60 0x7fe2ac129040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:37.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2b1d16700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0ad600 con 0x7fe2ac0aa020 2020-11-20T12:15:37.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2b1d16700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0ad8a0 con 0x7fe2ac0aa400 2020-11-20T12:15:37.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2b1d16700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0acff0 con 0x7fe2ac13ac60 2020-11-20T12:15:37.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac0aa020 0x7fe2ac0ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2ab7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac0aa020 0x7fe2ac0ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41442/0 (socket says 172.21.15.44:41442) 2020-11-20T12:15:37.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2ab7fe700 1 -- 172.21.15.44:0/1996520326 learned_addr learned my addr 172.21.15.44:0/1996520326 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:37.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0aa400 0x7fe2ac0acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2abfff700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac13ac60 0x7fe2ac129040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2ab7fe700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac13ac60 msgr2=0x7fe2ac129040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2ac13ac60 0x7fe2ac129040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:37.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2ab7fe700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0aa400 msgr2=0x7fe2ac0acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.878+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0aa400 0x7fe2ac0acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:37.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.879+0000 7fe2ab7fe700 1 -- 172.21.15.44:0/1996520326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0adcd0 con 0x7fe2ac0aa020 2020-11-20T12:15:37.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.879+0000 7fe2ab7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac0aa020 0x7fe2ac0ab1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe2ac061280 tx=0x7fe29c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:37.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.879+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c011e20 con 0x7fe2ac0aa020 2020-11-20T12:15:37.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.879+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c01e460 con 0x7fe2ac0aa020 2020-11-20T12:15:37.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.879+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c01e7e0 con 0x7fe2ac0aa020 2020-11-20T12:15:37.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.879+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac067e30 con 0x7fe2ac0aa020 2020-11-20T12:15:37.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.879+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac0aac60 con 0x7fe2ac0aa020 2020-11-20T12:15:37.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.880+0000 7fe2b1d16700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0663a0 0x7fe2ac129530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:37.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.880+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0aae50 con 0x7fe2ac0663a0 2020-11-20T12:15:37.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.880+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0663a0 0x7fe2ac129530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.880+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe29c025030 con 0x7fe2ac0aa020 2020-11-20T12:15:37.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.880+0000 7fe2aaffd700 1 -- 172.21.15.44:0/1996520326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2ac066780 con 0x7fe2ac0663a0 2020-11-20T12:15:37.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.880+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0663a0 0x7fe2ac129530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe2a00055a0 tx=0x7fe2a0009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:37.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe284056d20 0x7fe2840591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:37.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe29c0719e0 con 0x7fe2ac0aa020 2020-11-20T12:15:37.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe2abfff700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe284056d20 0x7fe2840591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000ee20 con 0x7fe2ac0663a0 2020-11-20T12:15:37.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fe2a0007320 con 0x7fe2ac0663a0 2020-11-20T12:15:37.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0663a0 msgr2=0x7fe2ac129530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0663a0 0x7fe2ac129530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fe2a00055a0 tx=0x7fe2a0009040).stop 2020-11-20T12:15:37.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0663a0 msgr2=0x7fe2ac129530 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:37.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe28b7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac0663a0 0x7fe2ac129530 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0x7fe2a00055a0 tx=0x7fe2a0009040).stop 2020-11-20T12:15:37.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.881+0000 7fe2abfff700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe284056d20 0x7fe2840591e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fe2ac067830 tx=0x7fe2a4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:37.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.890+0000 7fe2b1d16700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac138b60 0x7fe2ac138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:37.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.890+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0ad740 con 0x7fe2ac138b60 2020-11-20T12:15:37.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.890+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac138b60 0x7fe2ac138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe2aaffd700 1 -- 172.21.15.44:0/1996520326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2ac066220 con 0x7fe2ac138b60 2020-11-20T12:15:37.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe2aaffd700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac138b60 0x7fe2ac138f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fe2ac05ef40 tx=0x7fe2a0009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:37.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0005ab0 con 0x7fe2ac138b60 2020-11-20T12:15:37.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7fe2a0007110 con 0x7fe2ac138b60 2020-11-20T12:15:37.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac138b60 msgr2=0x7fe2ac138f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe28b7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac138b60 0x7fe2ac138f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fe2ac05ef40 tx=0x7fe2a0009040).stop 2020-11-20T12:15:37.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe28b7fe700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac138b60 msgr2=0x7fe2ac138f70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:37.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe28b7fe700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2ac138b60 0x7fe2ac138f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:37.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.891+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe284056d20 msgr2=0x7fe2840591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2b1d16700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe284056d20 0x7fe2840591e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fe2ac067830 tx=0x7fe2a4009040).stop 2020-11-20T12:15:37.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac0aa020 msgr2=0x7fe2ac0ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:37.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2b1d16700 1 --2- 172.21.15.44:0/1996520326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2ac0aa020 0x7fe2ac0ab1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe2ac061280 tx=0x7fe29c015000).stop 2020-11-20T12:15:37.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2abfff700 1 -- 172.21.15.44:0/1996520326 reap_dead start 2020-11-20T12:15:37.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2abfff700 1 -- 172.21.15.44:0/1996520326 reap_dead start 2020-11-20T12:15:37.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 shutdown_connections 2020-11-20T12:15:37.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 >> 172.21.15.44:0/1996520326 conn(0x7fe2ac062690 msgr2=0x7fe2ac060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:37.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 shutdown_connections 2020-11-20T12:15:37.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:36.892+0000 7fe2b1d16700 1 -- 172.21.15.44:0/1996520326 wait complete. 2020-11-20T12:15:37.951 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:37.951 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:37.952 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:37.952 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:15:37.952 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 124, 2020-11-20T12:15:37.952 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:37.953 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:15:37.953 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:37.953 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:37.953 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:37.954 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:15:37.954 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:37.954 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:37.954 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:37.954 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:37.955 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:37.955 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:37.955 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:37.955 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:37.955 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:37.956 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:37.956 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:37.956 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:37.956 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:37.956 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:37.957 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:37.957 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:37.957 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:37.957 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:37.958 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:37.958 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:37.958 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:37.958 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:37.958 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:37.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.959 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:37.959 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:37.959 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:37.959 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:37.960 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:37.960 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:37.960 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:37.960 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:37.960 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:37.961 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:37.961 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:37.961 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:37.961 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:37.961 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:37.962 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:37.963 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:37.963 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:37.963 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:37.963 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:37.963 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:37.964 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:37.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.964 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:37.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.964 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:37.965 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:37.965 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:37.965 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:37.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:37.966 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:37.966 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:37.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:37.967 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:37.967 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:37.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.967 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:37.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.968 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:37.968 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:37.968 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:37.968 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:37.969 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:37.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.969 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:37.969 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:37.970 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:37.970 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:37.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:37.970 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:37.971 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:37.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:37.972 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:37.972 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:37.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.972 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:37.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.973 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:37.973 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:37.973 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:37.973 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:37.973 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:37.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.974 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:37.974 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:37.974 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:37.975 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:37.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:37.975 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:37.975 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:37.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:37.976 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:37.976 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:37.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:37.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.977 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:37.977 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:37.978 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:37.978 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:37.978 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:37.978 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.979 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:37.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.979 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:37.979 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:37.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.980 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:37.980 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:37.980 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:37.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.980 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:37.981 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:15:37.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.981 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:37.981 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:37.982 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:15:37.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.982 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:37.982 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:15:37.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.983 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:37.983 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:37.983 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:15:37.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.983 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:37.984 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:15:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:15:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:37.985 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:37.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:37.987 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:37.987 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:37.987 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:15:38.049 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:38.056 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:38.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 Processor -- start 2020-11-20T12:15:38.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 -- start start 2020-11-20T12:15:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f348c0ae050 0x7f348c060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0669d0 0x7f348c13e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f348c13e9d0 0x7f348c137b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c060b40 con 0x7f348c0ae050 2020-11-20T12:15:38.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c13e080 con 0x7f348c13e9d0 2020-11-20T12:15:38.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.134+0000 7f34923ca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c13ee80 con 0x7f348c0669d0 2020-11-20T12:15:38.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f348c0ae050 0x7f348c060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f348c0ae050 0x7f348c060650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41454/0 (socket says 172.21.15.44:41454) 2020-11-20T12:15:38.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 -- 172.21.15.44:0/2215814431 learned_addr learned my addr 172.21.15.44:0/2215814431 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:38.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348b7fe700 1 --2- 172.21.15.44:0/2215814431 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0669d0 0x7f348c13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f3490967700 1 --2- 172.21.15.44:0/2215814431 >> v2:172.21.15.79:3300/0 conn(0x7f348c13e9d0 0x7f348c137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 -- 172.21.15.44:0/2215814431 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0669d0 msgr2=0x7f348c13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:38.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 --2- 172.21.15.44:0/2215814431 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0669d0 0x7f348c13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 -- 172.21.15.44:0/2215814431 >> v2:172.21.15.79:3300/0 conn(0x7f348c13e9d0 msgr2=0x7f348c137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:38.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 --2- 172.21.15.44:0/2215814431 >> v2:172.21.15.79:3300/0 conn(0x7f348c13e9d0 0x7f348c137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 -- 172.21.15.44:0/2215814431 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c060d60 con 0x7f348c0ae050 2020-11-20T12:15:38.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348b7fe700 1 --2- 172.21.15.44:0/2215814431 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0669d0 0x7f348c13e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:38.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348bfff700 1 --2- 172.21.15.44:0/2215814431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c060650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f348c063c60 tx=0x7f347c006eb0).ready entity=mon.0 client_cookie=27a5c2b5488d9349 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348affd700 1 -- 172.21.15.44:0/2215814431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0098d0 con 0x7f348c0ae050 2020-11-20T12:15:38.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348affd700 1 -- 172.21.15.44:0/2215814431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c009a30 con 0x7f348c0ae050 2020-11-20T12:15:38.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.135+0000 7f348affd700 1 -- 172.21.15.44:0/2215814431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c009db0 con 0x7f348c0ae050 2020-11-20T12:15:38.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 -- 172.21.15.44:0/2215814431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 msgr2=0x7f348c060650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 --2- 172.21.15.44:0/2215814431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c060650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f348c063c60 tx=0x7f347c006eb0).stop 2020-11-20T12:15:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 -- 172.21.15.44:0/2215814431 shutdown_connections 2020-11-20T12:15:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 --2- 172.21.15.44:0/2215814431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c060650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 --2- 172.21.15.44:0/2215814431 >> v2:172.21.15.79:3300/0 conn(0x7f348c13e9d0 0x7f348c137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 --2- 172.21.15.44:0/2215814431 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0669d0 0x7f348c13e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 -- 172.21.15.44:0/2215814431 >> 172.21.15.44:0/2215814431 conn(0x7f348c062bc0 msgr2=0x7f348c062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 -- 172.21.15.44:0/2215814431 shutdown_connections 2020-11-20T12:15:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.136+0000 7f34923ca700 1 -- 172.21.15.44:0/2215814431 wait complete. 2020-11-20T12:15:38.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.137+0000 7f34923ca700 1 Processor -- start 2020-11-20T12:15:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.137+0000 7f34923ca700 1 -- start start 2020-11-20T12:15:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f34923ca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f348c0669d0 0x7f348c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f34923ca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f34923ca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c13e9d0 0x7f348c1332f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f34923ca700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0a46b0 con 0x7f348c0ae050 2020-11-20T12:15:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f34923ca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c14d290 con 0x7f348c0669d0 2020-11-20T12:15:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f34923ca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c14d530 con 0x7f348c13e9d0 2020-11-20T12:15:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c14d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41460/0 (socket says 172.21.15.44:41460) 2020-11-20T12:15:38.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 -- 172.21.15.44:0/3278192404 learned_addr learned my addr 172.21.15.44:0/3278192404 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f348c0669d0 0x7f348c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f3490967700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c13e9d0 0x7f348c1332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c13e9d0 msgr2=0x7f348c1332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c13e9d0 0x7f348c1332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f348c0669d0 msgr2=0x7f348c0a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f348c0669d0 0x7f348c0a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 -- 172.21.15.44:0/3278192404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c13cca0 con 0x7f348c0ae050 2020-11-20T12:15:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f348b7fe700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c14d9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f348000db30 tx=0x7f3480008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348000de40 con 0x7f348c0ae050 2020-11-20T12:15:38.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480009820 con 0x7f348c0ae050 2020-11-20T12:15:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480009ba0 con 0x7f348c0ae050 2020-11-20T12:15:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.138+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c0a18e0 con 0x7f348c0ae050 2020-11-20T12:15:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.139+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c14dec0 con 0x7f348c0ae050 2020-11-20T12:15:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.139+0000 7f34923ca700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0a4180 0x7f348c1337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.139+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0a4560 con 0x7f348c0a4180 2020-11-20T12:15:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.139+0000 7f348bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0a4180 0x7f348c1337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3480013480 con 0x7f348c0ae050 2020-11-20T12:15:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f348bfff700 1 -- 172.21.15.44:0/3278192404 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f348c14d6f0 con 0x7f348c0a4180 2020-11-20T12:15:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f348bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0a4180 0x7f348c1337e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f348c063c60 tx=0x7f347c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3464056d20 0x7f34640591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3480070080 con 0x7f348c0ae050 2020-11-20T12:15:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f3490967700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3464056d20 0x7f34640591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c011c80 con 0x7f348c0a4180 2020-11-20T12:15:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f347c011de0 con 0x7f348c0a4180 2020-11-20T12:15:38.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0a4180 msgr2=0x7f348c1337e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0a4180 0x7f348c1337e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f348c063c60 tx=0x7f347c015000).stop 2020-11-20T12:15:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0a4180 msgr2=0x7f348c1337e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.140+0000 7f346bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c0a4180 0x7f348c1337e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.141+0000 7f3490967700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3464056d20 0x7f34640591e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f348400bce0 tx=0x7f3484008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.149+0000 7f34923ca700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c149ca0 0x7f348c14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.149+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c133cd0 con 0x7f348c149ca0 2020-11-20T12:15:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f348bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c149ca0 0x7f348c14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f348bfff700 1 -- 172.21.15.44:0/3278192404 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f348c14e0b0 con 0x7f348c149ca0 2020-11-20T12:15:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f348bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c149ca0 0x7f348c14a080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f347c00b1b0 tx=0x7f347c021070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c029ba0 con 0x7f348c149ca0 2020-11-20T12:15:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f347c011a70 con 0x7f348c149ca0 2020-11-20T12:15:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c149ca0 msgr2=0x7f348c14a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f346bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c149ca0 0x7f348c14a080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f347c00b1b0 tx=0x7f347c021070).stop 2020-11-20T12:15:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f346bfff700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c149ca0 msgr2=0x7f348c14a080 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.150+0000 7f346bfff700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f348c149ca0 0x7f348c14a080 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0x7f347c00b1b0 tx=0x7f347c021070).stop 2020-11-20T12:15:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3464056d20 msgr2=0x7f34640591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3464056d20 0x7f34640591e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f348400bce0 tx=0x7f3484008040).stop 2020-11-20T12:15:38.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 msgr2=0x7f348c14d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 --2- 172.21.15.44:0/3278192404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f348c0ae050 0x7f348c14d9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f348000db30 tx=0x7f3480008040).stop 2020-11-20T12:15:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f3490967700 1 -- 172.21.15.44:0/3278192404 reap_dead start 2020-11-20T12:15:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f3490967700 1 -- 172.21.15.44:0/3278192404 reap_dead start 2020-11-20T12:15:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 shutdown_connections 2020-11-20T12:15:38.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 >> 172.21.15.44:0/3278192404 conn(0x7f348c062bc0 msgr2=0x7f348c067890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:38.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 shutdown_connections 2020-11-20T12:15:38.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.151+0000 7f34923ca700 1 -- 172.21.15.44:0/3278192404 wait complete. 2020-11-20T12:15:38.163 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:15:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 124, 2020-11-20T12:15:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:15:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:15:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.176 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:38.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.176 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:38.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:38.176 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:38.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:38.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:38.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:38.177 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:38.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:38.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:38.178 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:38.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.179 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:38.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:38.179 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:38.180 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:38.180 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:38.180 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:38.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.181 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:38.181 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:38.181 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:38.181 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:38.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:38.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:38.182 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:38.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:38.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:38.183 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:38.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:38.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:38.184 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:38.184 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:38.184 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:38.185 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:38.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.185 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:38.185 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:38.186 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:38.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:38.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:38.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:38.187 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:38.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:38.187 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:38.188 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:38.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.188 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:38.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:38.189 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.190 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:38.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.190 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:38.190 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:38.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.191 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:38.191 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:38.191 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:38.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.191 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:38.192 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:15:38.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.192 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:38.192 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:38.192 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:38.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.193 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:38.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:38.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.193 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:38.194 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:38.194 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:15:38.194 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.194 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:38.194 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:15:38.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:38.195 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:38.195 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:38.195 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:38.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:38.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:38.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:38.196 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:38.196 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:38.197 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:15:38.198 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:15:38.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.371+0000 7fbd50456700 1 Processor -- start 2020-11-20T12:15:38.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.371+0000 7fbd50456700 1 -- start start 2020-11-20T12:15:38.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.372+0000 7fbd50456700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbd480a9130 0x7fbd480adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.372+0000 7fbd50456700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd4813b770 0x7fbd48145420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.372+0000 7fbd50456700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd480ab630 con 0x7fbd480a9510 2020-11-20T12:15:38.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.372+0000 7fbd50456700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd480ab110 con 0x7fbd480a9130 2020-11-20T12:15:38.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.372+0000 7fbd50456700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480ab3b0 con 0x7fbd4813b770 2020-11-20T12:15:38.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.372+0000 7fbd4d9f1700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbd480a9510 0x7fbd4813b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45220/0 (socket says 172.21.15.44:45220) 2020-11-20T12:15:38.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.372+0000 7fbd4d9f1700 1 -- 172.21.15.44:0/1078219058 learned_addr learned my addr 172.21.15.44:0/1078219058 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:38.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e1f2700 1 --2- 172.21.15.44:0/1078219058 >> v2:172.21.15.79:3300/0 conn(0x7fbd480a9130 0x7fbd480adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e9f3700 1 --2- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd4813b770 0x7fbd48145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4d1f0700 1 -- 172.21.15.44:0/1078219058 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550564130 0 0) 0x7fbd480ab630 con 0x7fbd480a9510 2020-11-20T12:15:38.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4d1f0700 1 -- 172.21.15.44:0/1078219058 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd30003240 con 0x7fbd480a9510 2020-11-20T12:15:38.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e1f2700 1 -- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd4813b770 msgr2=0x7fbd48145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:38.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e1f2700 1 --2- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd4813b770 0x7fbd48145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.388 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e1f2700 1 -- 172.21.15.44:0/1078219058 >> v1:172.21.15.44:6789/0 conn(0x7fbd480a9510 legacy=0x7fbd4813b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4d1f0700 1 -- 172.21.15.44:0/1078219058 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 55151041 0 0) 0x7fbd30003240 con 0x7fbd480a9510 2020-11-20T12:15:38.389 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e1f2700 1 -- 172.21.15.44:0/1078219058 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480ad7f0 con 0x7fbd480a9130 2020-11-20T12:15:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e9f3700 1 --2- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd4813b770 0x7fbd48145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4e1f2700 1 --2- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd480a9130 0x7fbd480adc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbd4400b9a0 tx=0x7fbd44006df0).ready entity=mon.1 client_cookie=11d06ec54eef6984 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4d1f0700 1 -- 172.21.15.44:0/1078219058 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd380014e0 con 0x7fbd480a9130 2020-11-20T12:15:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4d1f0700 1 -- 172.21.15.44:0/1078219058 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44011400 con 0x7fbd480a9130 2020-11-20T12:15:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.373+0000 7fbd4d1f0700 1 -- 172.21.15.44:0/1078219058 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44011780 con 0x7fbd480a9130 2020-11-20T12:15:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 -- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd480a9130 msgr2=0x7fbd480adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 --2- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd480a9130 0x7fbd480adc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbd4400b9a0 tx=0x7fbd44006df0).stop 2020-11-20T12:15:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 -- 172.21.15.44:0/1078219058 shutdown_connections 2020-11-20T12:15:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 --2- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd4813b770 0x7fbd48145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 --2- 172.21.15.44:0/1078219058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd480a9130 0x7fbd480adc90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 -- 172.21.15.44:0/1078219058 >> 172.21.15.44:0/1078219058 conn(0x7fbd48060170 msgr2=0x7fbd48060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:38.394 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 -- 172.21.15.44:0/1078219058 shutdown_connections 2020-11-20T12:15:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.374+0000 7fbd50456700 1 -- 172.21.15.44:0/1078219058 wait complete. 2020-11-20T12:15:38.395 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.375+0000 7fbd50456700 1 Processor -- start 2020-11-20T12:15:38.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.375+0000 7fbd50456700 1 -- start start 2020-11-20T12:15:38.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd50456700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd480a9130 0x7fbd48131340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd50456700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd480a9510 0x7fbd480b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.397 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd50456700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd4813b770 0x7fbd48149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd50456700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480af2d0 con 0x7fbd480a9130 2020-11-20T12:15:38.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd50456700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480b0f40 con 0x7fbd4813b770 2020-11-20T12:15:38.398 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd50456700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480af500 con 0x7fbd480a9510 2020-11-20T12:15:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd4e1f2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd480a9130 0x7fbd48131340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd4e1f2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd480a9130 0x7fbd48131340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41476/0 (socket says 172.21.15.44:41476) 2020-11-20T12:15:38.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd4e1f2700 1 -- 172.21.15.44:0/297299169 learned_addr learned my addr 172.21.15.44:0/297299169 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:38.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd4d9f1700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd480a9510 0x7fbd480b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.376+0000 7fbd4e9f3700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd4813b770 0x7fbd48149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd4e1f2700 1 -- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd480a9510 msgr2=0x7fbd480b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd4e1f2700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd480a9510 0x7fbd480b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd4e1f2700 1 -- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd4813b770 msgr2=0x7fbd48149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd4e1f2700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd4813b770 0x7fbd48149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd4e1f2700 1 -- 172.21.15.44:0/297299169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480af7a0 con 0x7fbd480a9130 2020-11-20T12:15:38.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd4e1f2700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd480a9130 0x7fbd48131340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fbd48060cf0 tx=0x7fbd44013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd367fc700 1 -- 172.21.15.44:0/297299169 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44010b10 con 0x7fbd480a9130 2020-11-20T12:15:38.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd367fc700 1 -- 172.21.15.44:0/297299169 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44010c70 con 0x7fbd480a9130 2020-11-20T12:15:38.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd480afa40 con 0x7fbd480a9130 2020-11-20T12:15:38.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd367fc700 1 -- 172.21.15.44:0/297299169 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4401e460 con 0x7fbd480a9130 2020-11-20T12:15:38.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.377+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4814a010 con 0x7fbd480a9130 2020-11-20T12:15:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.378+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd480ae180 con 0x7fbd480a9130 2020-11-20T12:15:38.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.378+0000 7fbd367fc700 1 -- 172.21.15.44:0/297299169 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbd44025030 con 0x7fbd480a9130 2020-11-20T12:15:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.379+0000 7fbd367fc700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbd20056d20 0x7fbd200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.379+0000 7fbd367fc700 1 -- 172.21.15.44:0/297299169 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbd44071380 con 0x7fbd480a9130 2020-11-20T12:15:38.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.379+0000 7fbd4d9f1700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbd20056d20 0x7fbd200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.379+0000 7fbd4d9f1700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbd20056d20 0x7fbd200591e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fbd4813a830 tx=0x7fbd38001280).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:38.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.383+0000 7fbd367fc700 1 -- 172.21.15.44:0/297299169 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbd44046650 con 0x7fbd480a9130 2020-11-20T12:15:38.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.665+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fbd480446d0 con 0x7fbd480a9130 2020-11-20T12:15:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.666+0000 7fbd367fc700 1 -- 172.21.15.44:0/297299169 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fbd44023030 con 0x7fbd480a9130 2020-11-20T12:15:38.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbd20056d20 msgr2=0x7fbd200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbd20056d20 0x7fbd200591e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fbd4813a830 tx=0x7fbd38001280).stop 2020-11-20T12:15:38.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd480a9130 msgr2=0x7fbd48131340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:38.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd480a9130 0x7fbd48131340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fbd48060cf0 tx=0x7fbd44013000).stop 2020-11-20T12:15:38.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 shutdown_connections 2020-11-20T12:15:38.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbd20056d20 0x7fbd200591e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd4813b770 0x7fbd48149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd480a9130 0x7fbd48131340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 --2- 172.21.15.44:0/297299169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbd480a9510 0x7fbd480b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:38.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.668+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 >> 172.21.15.44:0/297299169 conn(0x7fbd48060170 msgr2=0x7fbd4805ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:38.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.669+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 shutdown_connections 2020-11-20T12:15:38.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:38.669+0000 7fbd50456700 1 -- 172.21.15.44:0/297299169 wait complete. 2020-11-20T12:15:38.689 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:15:48.691 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:15:48.694 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:48.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.841+0000 7f011e7d4700 1 Processor -- start 2020-11-20T12:15:48.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.841+0000 7f011e7d4700 1 -- start start 2020-11-20T12:15:48.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011e7d4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180aad00 0x7f01180a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:48.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011e7d4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f01180a86d0 0x7f011813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:48.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011e7d4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01181385f0 con 0x7f01180ab0e0 2020-11-20T12:15:48.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011e7d4700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011813e250 con 0x7f01180a86d0 2020-11-20T12:15:48.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011e7d4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011813e4f0 con 0x7f01180aad00 2020-11-20T12:15:48.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011cd71700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f01180a86d0 0x7f011813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:48.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011cd71700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f01180a86d0 0x7f011813ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41488/0 (socket says 172.21.15.44:41488) 2020-11-20T12:15:48.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f011cd71700 1 -- 172.21.15.44:0/3512600116 learned_addr learned my addr 172.21.15.44:0/3512600116 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:48.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.842+0000 7f0117fff700 1 --2- 172.21.15.44:0/3512600116 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180aad00 0x7f01180a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:48.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011cd71700 1 -- 172.21.15.44:0/3512600116 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180aad00 msgr2=0x7f01180a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:48.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011cd71700 1 --2- 172.21.15.44:0/3512600116 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180aad00 0x7f01180a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:48.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011cd71700 1 -- 172.21.15.44:0/3512600116 >> v1:172.21.15.79:6789/0 conn(0x7f01180ab0e0 legacy=0x7f01180abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:48.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011cd71700 1 -- 172.21.15.44:0/3512600116 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180ac340 con 0x7f01180a86d0 2020-11-20T12:15:48.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011cd71700 1 --2- 172.21.15.44:0/3512600116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180a86d0 0x7f011813ec30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0110009d50 tx=0x7f0110009290).ready entity=mon.0 client_cookie=2dd547e4637167dd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:48.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f01177fe700 1 -- 172.21.15.44:0/3512600116 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011000bc50 con 0x7f01180a86d0 2020-11-20T12:15:48.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f01177fe700 1 -- 172.21.15.44:0/3512600116 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011000bdb0 con 0x7f01180a86d0 2020-11-20T12:15:48.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f01177fe700 1 -- 172.21.15.44:0/3512600116 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110011610 con 0x7f01180a86d0 2020-11-20T12:15:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011e7d4700 1 -- 172.21.15.44:0/3512600116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180a86d0 msgr2=0x7f011813ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011e7d4700 1 --2- 172.21.15.44:0/3512600116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180a86d0 0x7f011813ec30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0110009d50 tx=0x7f0110009290).stop 2020-11-20T12:15:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011e7d4700 1 -- 172.21.15.44:0/3512600116 shutdown_connections 2020-11-20T12:15:48.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011e7d4700 1 --2- 172.21.15.44:0/3512600116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180a86d0 0x7f011813ec30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011e7d4700 1 --2- 172.21.15.44:0/3512600116 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180aad00 0x7f01180a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.843+0000 7f011e7d4700 1 -- 172.21.15.44:0/3512600116 >> 172.21.15.44:0/3512600116 conn(0x7f0118060170 msgr2=0x7f0118060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.844+0000 7f011e7d4700 1 -- 172.21.15.44:0/3512600116 shutdown_connections 2020-11-20T12:15:48.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.844+0000 7f011e7d4700 1 -- 172.21.15.44:0/3512600116 wait complete. 2020-11-20T12:15:48.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.844+0000 7f011e7d4700 1 Processor -- start 2020-11-20T12:15:48.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011e7d4700 1 -- start start 2020-11-20T12:15:48.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011e7d4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01180a86d0 0x7f01180a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:48.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011e7d4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180aad00 0x7f011814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:48.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011e7d4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180ab0e0 0x7f01180a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:48.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011e7d4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011809d7b0 con 0x7f01180aad00 2020-11-20T12:15:48.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011e7d4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180a4310 con 0x7f01180a86d0 2020-11-20T12:15:48.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011e7d4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01180a45b0 con 0x7f01180ab0e0 2020-11-20T12:15:48.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f010ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180aad00 0x7f011814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:48.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f0117fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01180a86d0 0x7f01180a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:48.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f011cd71700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180ab0e0 0x7f01180a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:48.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f0117fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01180a86d0 0x7f01180a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33540/0 (socket says 172.21.15.44:33540) 2020-11-20T12:15:48.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.845+0000 7f0117fff700 1 -- 172.21.15.44:0/221735931 learned_addr learned my addr 172.21.15.44:0/221735931 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:48.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f010ffff700 1 -- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180ab0e0 msgr2=0x7f01180a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:48.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f010ffff700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180ab0e0 0x7f01180a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:48.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f010ffff700 1 -- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01180a86d0 msgr2=0x7f01180a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:48.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f010ffff700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01180a86d0 0x7f01180a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:48.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f010ffff700 1 -- 172.21.15.44:0/221735931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180a54a0 con 0x7f01180aad00 2020-11-20T12:15:48.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f010ffff700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180aad00 0x7f011814d500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0100000bc0 tx=0x7f0100007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:48.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f0114ff9700 1 -- 172.21.15.44:0/221735931 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100013410 con 0x7f01180aad00 2020-11-20T12:15:48.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f0114ff9700 1 -- 172.21.15.44:0/221735931 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0100013570 con 0x7f01180aad00 2020-11-20T12:15:48.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01180a4a50 con 0x7f01180aad00 2020-11-20T12:15:48.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f0114ff9700 1 -- 172.21.15.44:0/221735931 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01000138f0 con 0x7f01180aad00 2020-11-20T12:15:48.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.846+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011814dbb0 con 0x7f01180aad00 2020-11-20T12:15:48.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.847+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0118069680 con 0x7f01180aad00 2020-11-20T12:15:48.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.847+0000 7f0114ff9700 1 -- 172.21.15.44:0/221735931 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0100010030 con 0x7f01180aad00 2020-11-20T12:15:48.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.847+0000 7f0114ff9700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f00f4056d20 0x7f00f40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:48.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.848+0000 7f0114ff9700 1 -- 172.21.15.44:0/221735931 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f010006fd20 con 0x7f01180aad00 2020-11-20T12:15:48.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.848+0000 7f0117fff700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f00f4056d20 0x7f00f40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:48.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.848+0000 7f0117fff700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f00f4056d20 0x7f00f40591e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f01180a5110 tx=0x7f0108006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:48.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:48.853+0000 7f0114ff9700 1 -- 172.21.15.44:0/221735931 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0100044ff0 con 0x7f01180aad00 2020-11-20T12:15:49.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.113+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f01180a71c0 con 0x7f01180aad00 2020-11-20T12:15:49.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.114+0000 7f0114ff9700 1 -- 172.21.15.44:0/221735931 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f0100048610 con 0x7f01180aad00 2020-11-20T12:15:49.120 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:49.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f00f4056d20 msgr2=0x7f00f40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f00f4056d20 0x7f00f40591e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f01180a5110 tx=0x7f0108006df0).stop 2020-11-20T12:15:49.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180aad00 msgr2=0x7f011814d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180aad00 0x7f011814d500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0100000bc0 tx=0x7f0100007b70).stop 2020-11-20T12:15:49.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 shutdown_connections 2020-11-20T12:15:49.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f00f4056d20 0x7f00f40591e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f01180ab0e0 0x7f01180a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01180a86d0 0x7f01180a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 --2- 172.21.15.44:0/221735931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01180aad00 0x7f011814d500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 >> 172.21.15.44:0/221735931 conn(0x7f0118060170 msgr2=0x7f01180ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 shutdown_connections 2020-11-20T12:15:49.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.115+0000 7f011e7d4700 1 -- 172.21.15.44:0/221735931 wait complete. 2020-11-20T12:15:49.134 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:15:49.134 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:15:49.135 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:15:49.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.275+0000 7f82e9edb700 1 Processor -- start 2020-11-20T12:15:49.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.275+0000 7f82e9edb700 1 -- start start 2020-11-20T12:15:49.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e9edb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f82e40aa020 0x7f82e40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e9edb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f82e40aa400 0x7f82e40a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e9edb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e413ac60 0x7f82e40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e9edb700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40a9770 con 0x7f82e40aa400 2020-11-20T12:15:49.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e9edb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40a6da0 con 0x7f82e40aa020 2020-11-20T12:15:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e9edb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40a7040 con 0x7f82e413ac60 2020-11-20T12:15:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e2ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f82e40aa400 0x7f82e40a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e2ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f82e40aa400 0x7f82e40a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41498/0 (socket says 172.21.15.44:41498) 2020-11-20T12:15:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e2ffd700 1 -- 172.21.15.44:0/1838161083 learned_addr learned my addr 172.21.15.44:0/1838161083 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e3fff700 1 --2- 172.21.15.44:0/1838161083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e413ac60 0x7f82e40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.276+0000 7f82e37fe700 1 --2- 172.21.15.44:0/1838161083 >> v2:172.21.15.79:3300/0 conn(0x7f82e40aa020 0x7f82e40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e2ffd700 1 -- 172.21.15.44:0/1838161083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e413ac60 msgr2=0x7f82e40a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e2ffd700 1 --2- 172.21.15.44:0/1838161083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e413ac60 0x7f82e40a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e2ffd700 1 -- 172.21.15.44:0/1838161083 >> v2:172.21.15.79:3300/0 conn(0x7f82e40aa020 msgr2=0x7f82e40a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e2ffd700 1 --2- 172.21.15.44:0/1838161083 >> v2:172.21.15.79:3300/0 conn(0x7f82e40aa020 0x7f82e40a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e2ffd700 1 -- 172.21.15.44:0/1838161083 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40a6a90 con 0x7f82e40aa400 2020-11-20T12:15:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e2ffd700 1 --2- 172.21.15.44:0/1838161083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e40aa400 0x7f82e40a4fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f82e40adec0 tx=0x7f82d8006eb0).ready entity=mon.0 client_cookie=256893285388ab3b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e27fc700 1 -- 172.21.15.44:0/1838161083 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8009720 con 0x7f82e40aa400 2020-11-20T12:15:49.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e27fc700 1 -- 172.21.15.44:0/1838161083 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d8009880 con 0x7f82e40aa400 2020-11-20T12:15:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e27fc700 1 -- 172.21.15.44:0/1838161083 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8009c00 con 0x7f82e40aa400 2020-11-20T12:15:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e9edb700 1 -- 172.21.15.44:0/1838161083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e40aa400 msgr2=0x7f82e40a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e9edb700 1 --2- 172.21.15.44:0/1838161083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e40aa400 0x7f82e40a4fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f82e40adec0 tx=0x7f82d8006eb0).stop 2020-11-20T12:15:49.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e9edb700 1 -- 172.21.15.44:0/1838161083 shutdown_connections 2020-11-20T12:15:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e9edb700 1 --2- 172.21.15.44:0/1838161083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e413ac60 0x7f82e40a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e9edb700 1 --2- 172.21.15.44:0/1838161083 >> v2:172.21.15.79:3300/0 conn(0x7f82e40aa020 0x7f82e40a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e9edb700 1 --2- 172.21.15.44:0/1838161083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e40aa400 0x7f82e40a4fb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.277+0000 7f82e9edb700 1 -- 172.21.15.44:0/1838161083 >> 172.21.15.44:0/1838161083 conn(0x7f82e4062690 msgr2=0x7f82e40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.278+0000 7f82e9edb700 1 -- 172.21.15.44:0/1838161083 shutdown_connections 2020-11-20T12:15:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.278+0000 7f82e9edb700 1 -- 172.21.15.44:0/1838161083 wait complete. 2020-11-20T12:15:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 Processor -- start 2020-11-20T12:15:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 -- start start 2020-11-20T12:15:49.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e40aa020 0x7f82e40ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e40aa400 0x7f82e40672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e413ac60 0x7f82e4131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40abdc0 con 0x7f82e413ac60 2020-11-20T12:15:49.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40ad2b0 con 0x7f82e40aa020 2020-11-20T12:15:49.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.279+0000 7f82e9edb700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40ab050 con 0x7f82e40aa400 2020-11-20T12:15:49.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e413ac60 0x7f82e4131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e413ac60 0x7f82e4131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41508/0 (socket says 172.21.15.44:41508) 2020-11-20T12:15:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 -- 172.21.15.44:0/2361738633 learned_addr learned my addr 172.21.15.44:0/2361738633 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e2ffd700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e40aa400 0x7f82e40672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.296 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e40aa020 0x7f82e40ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e40aa400 msgr2=0x7f82e40672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f82e40aa400 0x7f82e40672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e40aa020 msgr2=0x7f82e40ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e40aa020 0x7f82e40ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 -- 172.21.15.44:0/2361738633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40adc90 con 0x7f82e413ac60 2020-11-20T12:15:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e3fff700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e413ac60 0x7f82e4131990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f82dc00e4a0 tx=0x7f82dc00d040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc015400 con 0x7f82e413ac60 2020-11-20T12:15:49.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc011040 con 0x7f82e413ac60 2020-11-20T12:15:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc015790 con 0x7f82e413ac60 2020-11-20T12:15:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e40ad4e0 con 0x7f82e413ac60 2020-11-20T12:15:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.280+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e413cfd0 con 0x7f82e413ac60 2020-11-20T12:15:49.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.281+0000 7f82e9edb700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e4131e80 0x7f82e4132cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.281+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e413d1d0 con 0x7f82e4131e80 2020-11-20T12:15:49.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.281+0000 7f82e37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e4131e80 0x7f82e4132cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.281+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f82dc015b00 con 0x7f82e413ac60 2020-11-20T12:15:49.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82e37fe700 1 -- 172.21.15.44:0/2361738633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82e4135ad0 con 0x7f82e4131e80 2020-11-20T12:15:49.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82e37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e4131e80 0x7f82e4132cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f82d4011be0 tx=0x7f82d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f82bc056d20 0x7f82bc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f82dc072c50 con 0x7f82e413ac60 2020-11-20T12:15:49.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400fbe0 con 0x7f82e4131e80 2020-11-20T12:15:49.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f82d400fd40 con 0x7f82e4131e80 2020-11-20T12:15:49.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82e2ffd700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f82bc056d20 0x7f82bc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e4131e80 msgr2=0x7f82e4132cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e4131e80 0x7f82e4132cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f82d4011be0 tx=0x7f82d4006df0).stop 2020-11-20T12:15:49.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e4131e80 msgr2=0x7f82e4132cf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:49.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.282+0000 7f82c37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e4131e80 0x7f82e4132cf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.283+0000 7f82e2ffd700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f82bc056d20 0x7f82bc0591e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f82d8006ef0 tx=0x7f82d8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.293+0000 7f82e9edb700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e41331e0 0x7f82e41335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.293+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40aacc0 con 0x7f82e41331e0 2020-11-20T12:15:49.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.293+0000 7f82e37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e41331e0 0x7f82e41335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.293+0000 7f82e37fe700 1 -- 172.21.15.44:0/2361738633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f82e4061650 con 0x7f82e41331e0 2020-11-20T12:15:49.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.293+0000 7f82e37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e41331e0 0x7f82e41335f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f82d4018000 tx=0x7f82d401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4022af0 con 0x7f82e41331e0 2020-11-20T12:15:49.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4269 (secure 0 0 0) 0x7f82d4022c50 con 0x7f82e41331e0 2020-11-20T12:15:49.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e41331e0 msgr2=0x7f82e41335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82c37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e41331e0 0x7f82e41335f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f82d4018000 tx=0x7f82d401a000).stop 2020-11-20T12:15:49.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82c37fe700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e41331e0 msgr2=0x7f82e41335f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:49.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82c37fe700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e41331e0 0x7f82e41335f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f82bc056d20 msgr2=0x7f82bc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f82bc056d20 0x7f82bc0591e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f82d8006ef0 tx=0x7f82d8015000).stop 2020-11-20T12:15:49.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e413ac60 msgr2=0x7f82e4131990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 --2- 172.21.15.44:0/2361738633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82e413ac60 0x7f82e4131990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f82dc00e4a0 tx=0x7f82dc00d040).stop 2020-11-20T12:15:49.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e3fff700 1 -- 172.21.15.44:0/2361738633 reap_dead start 2020-11-20T12:15:49.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e3fff700 1 -- 172.21.15.44:0/2361738633 reap_dead start 2020-11-20T12:15:49.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 shutdown_connections 2020-11-20T12:15:49.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 >> 172.21.15.44:0/2361738633 conn(0x7f82e4062690 msgr2=0x7f82e40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 shutdown_connections 2020-11-20T12:15:49.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.294+0000 7f82e9edb700 1 -- 172.21.15.44:0/2361738633 wait complete. 2020-11-20T12:15:49.309 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:49.310 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:49.310 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:49.310 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:15:49.310 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 124, 2020-11-20T12:15:49.311 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:49.311 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:15:49.311 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:49.311 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:49.311 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.312 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:15:49.312 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:49.312 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:49.312 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:49.313 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.313 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.313 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.313 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.314 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.314 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.314 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.314 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.314 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:49.315 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:49.315 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.315 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.315 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.316 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.316 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.316 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.316 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.317 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.317 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:49.317 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:49.317 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:49.323 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:49.323 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:49.324 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:49.324 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:49.324 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:49.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:49.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:49.325 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:49.325 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:49.325 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:49.325 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:49.326 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:49.326 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.326 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.326 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.327 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.327 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.327 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.327 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.327 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.328 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:49.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.328 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:49.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.328 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:49.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:49.329 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.329 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.330 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:49.330 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.331 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:49.331 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.331 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.332 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:49.332 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:49.333 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.333 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.333 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.334 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:49.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:49.334 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.334 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.335 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:49.335 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.336 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:49.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.337 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.337 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:49.337 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:49.338 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.338 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.338 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.339 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:49.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:49.339 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.339 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.340 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:49.340 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.341 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:49.341 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.342 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.342 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:49.342 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:49.343 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.343 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.343 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.343 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.343 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.344 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:49.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:49.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.344 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.345 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.345 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:49.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.345 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.346 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:49.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.346 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.346 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.346 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:15:49.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.347 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.347 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:49.347 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:49.349 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:15:49.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.499+0000 7fd61d8e7700 1 Processor -- start 2020-11-20T12:15:49.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.499+0000 7fd61d8e7700 1 -- start start 2020-11-20T12:15:49.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.499+0000 7fd61d8e7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd6180ad440 0x7fd6180a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.499+0000 7fd61d8e7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6180a4b00 0x7fd6180a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd61d8e7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180a70f0 con 0x7fd6180a9bb0 2020-11-20T12:15:49.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd61d8e7700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6180a5880 con 0x7fd6180a4b00 2020-11-20T12:15:49.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd61d8e7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6180a4170 con 0x7fd6180ad440 2020-11-20T12:15:49.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd617fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6180a4b00 0x7fd6180a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd61d0e6700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd6180a9bb0 0x7fd618134260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45266/0 (socket says 172.21.15.44:45266) 2020-11-20T12:15:49.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd61d0e6700 1 -- 172.21.15.44:0/1022576616 learned_addr learned my addr 172.21.15.44:0/1022576616 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:49.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd61c8e5700 1 --2- 172.21.15.44:0/1022576616 >> v2:172.21.15.79:3300/0 conn(0x7fd6180ad440 0x7fd6180a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd617fff700 1 -- 172.21.15.44:0/1022576616 >> v2:172.21.15.79:3300/0 conn(0x7fd6180ad440 msgr2=0x7fd6180a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd617fff700 1 --2- 172.21.15.44:0/1022576616 >> v2:172.21.15.79:3300/0 conn(0x7fd6180ad440 0x7fd6180a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd617fff700 1 -- 172.21.15.44:0/1022576616 >> v1:172.21.15.44:6789/0 conn(0x7fd6180a9bb0 legacy=0x7fd618134260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd6177fe700 1 -- 172.21.15.44:0/1022576616 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973262139 0 0) 0x7fd6180a70f0 con 0x7fd6180a9bb0 2020-11-20T12:15:49.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd617fff700 1 -- 172.21.15.44:0/1022576616 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180ad820 con 0x7fd6180a4b00 2020-11-20T12:15:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd617fff700 1 --2- 172.21.15.44:0/1022576616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 0x7fd6180a96c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd618062720 tx=0x7fd604006df0).ready entity=mon.0 client_cookie=3ba43eecc9ae37fc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd6177fe700 1 -- 172.21.15.44:0/1022576616 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604009840 con 0x7fd6180a4b00 2020-11-20T12:15:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd6177fe700 1 -- 172.21.15.44:0/1022576616 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6040099a0 con 0x7fd6180a4b00 2020-11-20T12:15:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.500+0000 7fd6177fe700 1 -- 172.21.15.44:0/1022576616 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604009d20 con 0x7fd6180a4b00 2020-11-20T12:15:49.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 -- 172.21.15.44:0/1022576616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 msgr2=0x7fd6180a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 --2- 172.21.15.44:0/1022576616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 0x7fd6180a96c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd618062720 tx=0x7fd604006df0).stop 2020-11-20T12:15:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 -- 172.21.15.44:0/1022576616 shutdown_connections 2020-11-20T12:15:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 --2- 172.21.15.44:0/1022576616 >> v2:172.21.15.79:3300/0 conn(0x7fd6180ad440 0x7fd6180a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 --2- 172.21.15.44:0/1022576616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 0x7fd6180a96c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 -- 172.21.15.44:0/1022576616 >> 172.21.15.44:0/1022576616 conn(0x7fd618061b60 msgr2=0x7fd618061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 -- 172.21.15.44:0/1022576616 shutdown_connections 2020-11-20T12:15:49.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.501+0000 7fd61d8e7700 1 -- 172.21.15.44:0/1022576616 wait complete. 2020-11-20T12:15:49.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.502+0000 7fd61d8e7700 1 Processor -- start 2020-11-20T12:15:49.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.502+0000 7fd61d8e7700 1 -- start start 2020-11-20T12:15:49.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d8e7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 0x7fd61814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d8e7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6180a9bb0 0x7fd6180a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d8e7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6180ad440 0x7fd618139fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d8e7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180a23a0 con 0x7fd6180a4b00 2020-11-20T12:15:49.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d8e7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618136630 con 0x7fd6180ad440 2020-11-20T12:15:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d8e7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180a25d0 con 0x7fd6180a9bb0 2020-11-20T12:15:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d0e6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6180ad440 0x7fd618139fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d0e6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6180ad440 0x7fd618139fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33576/0 (socket says 172.21.15.44:33576) 2020-11-20T12:15:49.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61d0e6700 1 -- 172.21.15.44:0/3535147741 learned_addr learned my addr 172.21.15.44:0/3535147741 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd617fff700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6180a9bb0 0x7fd6180a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.503+0000 7fd61c8e5700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 0x7fd61814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d0e6700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6180a9bb0 msgr2=0x7fd6180a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d0e6700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6180a9bb0 0x7fd6180a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d0e6700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 msgr2=0x7fd61814d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d0e6700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 0x7fd61814d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d0e6700 1 -- 172.21.15.44:0/3535147741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618135f20 con 0x7fd6180ad440 2020-11-20T12:15:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61c8e5700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6180a4b00 0x7fd61814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:49.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d0e6700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6180ad440 0x7fd618139fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd61000cab0 tx=0x7fd610007d80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610005b80 con 0x7fd6180ad440 2020-11-20T12:15:49.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd610005ce0 con 0x7fd6180ad440 2020-11-20T12:15:49.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6181361d0 con 0x7fd6180ad440 2020-11-20T12:15:49.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610007360 con 0x7fd6180ad440 2020-11-20T12:15:49.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.504+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6180a2c30 con 0x7fd6180ad440 2020-11-20T12:15:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.505+0000 7fd61d8e7700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618137c60 0x7fd618138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.505+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618061180 con 0x7fd618137c60 2020-11-20T12:15:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.505+0000 7fd61c8e5700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618137c60 0x7fd618138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.505+0000 7fd61c8e5700 1 -- 172.21.15.44:0/3535147741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6180a48c0 con 0x7fd618137c60 2020-11-20T12:15:49.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.505+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd610010030 con 0x7fd6180ad440 2020-11-20T12:15:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.505+0000 7fd61c8e5700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618137c60 0x7fd618138040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fd60c010ad0 tx=0x7fd60c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5f0056d20 0x7fd5f00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd6100700f0 con 0x7fd6180ad440 2020-11-20T12:15:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd617fff700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5f0056d20 0x7fd5f00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00eac0 con 0x7fd618137c60 2020-11-20T12:15:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fd60c00ec20 con 0x7fd618137c60 2020-11-20T12:15:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618137c60 msgr2=0x7fd618138040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618137c60 0x7fd618138040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fd60c010ad0 tx=0x7fd60c008040).stop 2020-11-20T12:15:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618137c60 msgr2=0x7fd618138040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:49.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd614ff9700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd618137c60 0x7fd618138040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.506+0000 7fd617fff700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5f0056d20 0x7fd5f00591e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fd604006e30 tx=0x7fd604015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.516+0000 7fd61d8e7700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61814a2f0 0x7fd61814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.516+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180a46c0 con 0x7fd61814a2f0 2020-11-20T12:15:49.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.517+0000 7fd61c8e5700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61814a2f0 0x7fd61814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.517+0000 7fd61c8e5700 1 -- 172.21.15.44:0/3535147741 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd618057d80 con 0x7fd61814a2f0 2020-11-20T12:15:49.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.517+0000 7fd61c8e5700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61814a2f0 0x7fd61814a700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fd60c019000 tx=0x7fd60c01c000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.517+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c014750 con 0x7fd61814a2f0 2020-11-20T12:15:49.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7fd60c00e8b0 con 0x7fd61814a2f0 2020-11-20T12:15:49.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61814a2f0 msgr2=0x7fd61814a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd614ff9700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61814a2f0 0x7fd61814a700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fd60c019000 tx=0x7fd60c01c000).stop 2020-11-20T12:15:49.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd614ff9700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61814a2f0 msgr2=0x7fd61814a700 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:49.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd614ff9700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61814a2f0 0x7fd61814a700 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0x7fd60c019000 tx=0x7fd60c01c000).stop 2020-11-20T12:15:49.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5f0056d20 msgr2=0x7fd5f00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd5f0056d20 0x7fd5f00591e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fd604006e30 tx=0x7fd604015000).stop 2020-11-20T12:15:49.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6180ad440 msgr2=0x7fd618139fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 --2- 172.21.15.44:0/3535147741 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6180ad440 0x7fd618139fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd61000cab0 tx=0x7fd610007d80).stop 2020-11-20T12:15:49.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d0e6700 1 -- 172.21.15.44:0/3535147741 reap_dead start 2020-11-20T12:15:49.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d0e6700 1 -- 172.21.15.44:0/3535147741 reap_dead start 2020-11-20T12:15:49.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 shutdown_connections 2020-11-20T12:15:49.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 >> 172.21.15.44:0/3535147741 conn(0x7fd618061b60 msgr2=0x7fd6180a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 shutdown_connections 2020-11-20T12:15:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.518+0000 7fd61d8e7700 1 -- 172.21.15.44:0/3535147741 wait complete. 2020-11-20T12:15:49.528 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:49.528 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:49.529 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:49.529 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:15:49.529 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 124, 2020-11-20T12:15:49.529 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:49.530 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:15:49.530 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:49.530 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:49.530 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.530 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:15:49.531 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:49.531 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:49.531 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:49.531 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.531 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.532 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.532 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.532 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.532 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.532 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.533 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.533 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.533 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:49.533 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:49.533 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.534 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.534 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.534 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.534 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.534 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.535 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.535 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.535 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.535 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:49.536 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:49.536 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:49.536 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:49.536 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:49.536 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:49.537 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:49.537 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:49.537 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:49.537 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:49.537 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:49.538 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:49.538 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:49.538 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:49.538 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:49.538 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.539 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.539 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.539 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.539 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.539 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.540 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.540 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.540 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.540 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:49.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.541 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:49.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.541 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:49.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:49.541 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.542 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.542 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:49.542 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.543 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:49.543 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.544 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.544 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:49.544 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:49.545 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.545 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.545 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.546 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:49.546 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:49.546 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.546 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.547 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:49.547 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.548 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.549 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:49.549 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.549 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.549 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.550 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:49.550 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:49.550 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.550 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.550 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.551 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:49.551 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:49.551 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.552 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.552 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.552 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:49.552 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.553 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.553 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:49.554 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.554 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.554 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:49.555 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:49.555 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.555 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.555 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.556 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.556 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:49.556 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:49.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.557 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.557 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.557 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:49.557 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.557 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.558 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:15:49.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.558 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.558 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.558 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:15:49.559 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.559 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.559 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:15:49.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.559 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.560 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.560 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:15:49.560 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.560 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.560 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:49.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.561 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.561 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.561 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:15:49.561 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.562 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.562 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:15:49.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.562 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.562 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.563 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:49.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.563 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.563 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:49.564 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:49.565 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:15:49.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.713+0000 7f717cbb2700 1 Processor -- start 2020-11-20T12:15:49.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.713+0000 7f717cbb2700 1 -- start start 2020-11-20T12:15:49.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.714+0000 7f717cbb2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7178068f60 0x7f71780a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.714+0000 7f717cbb2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71780a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.714+0000 7f717cbb2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71780a3250 0x7f7178137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.714+0000 7f717cbb2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71780a93c0 con 0x7f7178068f60 2020-11-20T12:15:49.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.714+0000 7f717cbb2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71780a95f0 con 0x7f71780a3250 2020-11-20T12:15:49.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.714+0000 7f717cbb2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71780a9ea0 con 0x7f71780a84b0 2020-11-20T12:15:49.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.714+0000 7f717659c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7178068f60 0x7f71780a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7178068f60 0x7f71780a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41534/0 (socket says 172.21.15.44:41534) 2020-11-20T12:15:49.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 -- 172.21.15.44:0/264852611 learned_addr learned my addr 172.21.15.44:0/264852611 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:49.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f7175d9b700 1 --2- 172.21.15.44:0/264852611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71780a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f7176d9d700 1 --2- 172.21.15.44:0/264852611 >> v2:172.21.15.79:3300/0 conn(0x7f71780a3250 0x7f7178137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 -- 172.21.15.44:0/264852611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 msgr2=0x7f71780a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:49.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 --2- 172.21.15.44:0/264852611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71780a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 -- 172.21.15.44:0/264852611 >> v2:172.21.15.79:3300/0 conn(0x7f71780a3250 msgr2=0x7f7178137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:49.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 --2- 172.21.15.44:0/264852611 >> v2:172.21.15.79:3300/0 conn(0x7f71780a3250 0x7f7178137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 -- 172.21.15.44:0/264852611 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7178069340 con 0x7f7178068f60 2020-11-20T12:15:49.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f7175d9b700 1 --2- 172.21.15.44:0/264852611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71780a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:49.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f7176d9d700 1 --2- 172.21.15.44:0/264852611 >> v2:172.21.15.79:3300/0 conn(0x7f71780a3250 0x7f7178137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:49.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717659c700 1 --2- 172.21.15.44:0/264852611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f71780a9910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f717813ac00 tx=0x7f7168006eb0).ready entity=mon.0 client_cookie=b6aa197a5837d578 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717559a700 1 -- 172.21.15.44:0/264852611 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168009960 con 0x7f7178068f60 2020-11-20T12:15:49.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717559a700 1 -- 172.21.15.44:0/264852611 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7168009ac0 con 0x7f7178068f60 2020-11-20T12:15:49.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.715+0000 7f717559a700 1 -- 172.21.15.44:0/264852611 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168009e40 con 0x7f7178068f60 2020-11-20T12:15:49.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 -- 172.21.15.44:0/264852611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 msgr2=0x7f71780a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 --2- 172.21.15.44:0/264852611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f71780a9910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f717813ac00 tx=0x7f7168006eb0).stop 2020-11-20T12:15:49.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 -- 172.21.15.44:0/264852611 shutdown_connections 2020-11-20T12:15:49.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 --2- 172.21.15.44:0/264852611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f71780a9910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 --2- 172.21.15.44:0/264852611 >> v2:172.21.15.79:3300/0 conn(0x7f71780a3250 0x7f7178137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 --2- 172.21.15.44:0/264852611 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71780a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 -- 172.21.15.44:0/264852611 >> 172.21.15.44:0/264852611 conn(0x7f71780ad6d0 msgr2=0x7f71780adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 -- 172.21.15.44:0/264852611 shutdown_connections 2020-11-20T12:15:49.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.716+0000 7f717cbb2700 1 -- 172.21.15.44:0/264852611 wait complete. 2020-11-20T12:15:49.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.717+0000 7f717cbb2700 1 Processor -- start 2020-11-20T12:15:49.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.717+0000 7f717cbb2700 1 -- start start 2020-11-20T12:15:49.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717cbb2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f7178135820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717cbb2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a3250 0x7f7178066ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717cbb2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7178067dd0 0x7f71780acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717cbb2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7178065b30 con 0x7f7178068f60 2020-11-20T12:15:49.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717cbb2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7178068250 con 0x7f7178067dd0 2020-11-20T12:15:49.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717cbb2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7178068690 con 0x7f71780a3250 2020-11-20T12:15:49.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f7178135820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f7178135820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41540/0 (socket says 172.21.15.44:41540) 2020-11-20T12:15:49.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f717659c700 1 -- 172.21.15.44:0/753987234 learned_addr learned my addr 172.21.15.44:0/753987234 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:49.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f7175d9b700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a3250 0x7f7178066ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.718+0000 7f7176d9d700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7178067dd0 0x7f71780acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717659c700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a3250 msgr2=0x7f7178066ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717659c700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a3250 0x7f7178066ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717659c700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7178067dd0 msgr2=0x7f71780acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717659c700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7178067dd0 0x7f71780acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717659c700 1 -- 172.21.15.44:0/753987234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71780673f0 con 0x7f7178068f60 2020-11-20T12:15:49.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f7176d9d700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7178067dd0 0x7f71780acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:49.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717659c700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f7178135820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7168006ef0 tx=0x7f7168015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168011e50 con 0x7f7178068f60 2020-11-20T12:15:49.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716801e460 con 0x7f7178068f60 2020-11-20T12:15:49.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716801e800 con 0x7f7178068f60 2020-11-20T12:15:49.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7178067690 con 0x7f7178068f60 2020-11-20T12:15:49.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.719+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71780ad240 con 0x7f7178068f60 2020-11-20T12:15:49.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.720+0000 7f717cbb2700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71781341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.720+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7178069670 con 0x7f71780a84b0 2020-11-20T12:15:49.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.720+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7168025030 con 0x7f7178068f60 2020-11-20T12:15:49.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.720+0000 7f7175d9b700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71781341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.720+0000 7f71667fc700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f714c056d20 0x7f714c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f7168071980 con 0x7f7178068f60 2020-11-20T12:15:49.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f7175d9b700 1 -- 172.21.15.44:0/753987234 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7178138f90 con 0x7f71780a84b0 2020-11-20T12:15:49.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f7176d9d700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f714c056d20 0x7f714c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f7175d9b700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71781341b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f716c010ad0 tx=0x7f716c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c00e9f0 con 0x7f71780a84b0 2020-11-20T12:15:49.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f7176d9d700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f714c056d20 0x7f714c0591e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f7178066380 tx=0x7f7170008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f716c00eb50 con 0x7f71780a84b0 2020-11-20T12:15:49.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 msgr2=0x7f71781341b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f71667fc700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71781341b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f716c010ad0 tx=0x7f716c008040).stop 2020-11-20T12:15:49.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 msgr2=0x7f71781341b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:49.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.721+0000 7f71667fc700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71780a84b0 0x7f71781341b0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0x7f716c010ad0 tx=0x7f716c008040).stop 2020-11-20T12:15:49.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.731+0000 7f717cbb2700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71781346a0 0x7f71780ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.731+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7178064170 con 0x7f71781346a0 2020-11-20T12:15:49.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.731+0000 7f7175d9b700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71781346a0 0x7f71780ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.731+0000 7f7175d9b700 1 -- 172.21.15.44:0/753987234 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7178063000 con 0x7f71781346a0 2020-11-20T12:15:49.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f7175d9b700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71781346a0 0x7f71780ab6e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f716c010ad0 tx=0x7f716c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c006240 con 0x7f71781346a0 2020-11-20T12:15:49.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7f716c026070 con 0x7f71781346a0 2020-11-20T12:15:49.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71781346a0 msgr2=0x7f71780ab6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f71667fc700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71781346a0 0x7f71780ab6e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f716c010ad0 tx=0x7f716c008040).stop 2020-11-20T12:15:49.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f71667fc700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71781346a0 msgr2=0x7f71780ab6e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:15:49.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f71667fc700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71781346a0 0x7f71780ab6e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f714c056d20 msgr2=0x7f714c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f714c056d20 0x7f714c0591e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f7178066380 tx=0x7f7170008040).stop 2020-11-20T12:15:49.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 msgr2=0x7f7178135820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 --2- 172.21.15.44:0/753987234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7178068f60 0x7f7178135820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7168006ef0 tx=0x7f7168015000).stop 2020-11-20T12:15:49.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f7176d9d700 1 -- 172.21.15.44:0/753987234 reap_dead start 2020-11-20T12:15:49.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f7176d9d700 1 -- 172.21.15.44:0/753987234 reap_dead start 2020-11-20T12:15:49.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 shutdown_connections 2020-11-20T12:15:49.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 >> 172.21.15.44:0/753987234 conn(0x7f71780ad6d0 msgr2=0x7f71780a88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 shutdown_connections 2020-11-20T12:15:49.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.732+0000 7f717cbb2700 1 -- 172.21.15.44:0/753987234 wait complete. 2020-11-20T12:15:49.744 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:15:49.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:49.744 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:49.744 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:15:49.745 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 124, 2020-11-20T12:15:49.745 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:15:49.745 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:15:49.745 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:15:49.745 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:15:49.746 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.746 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 15, 2020-11-20T12:15:49.746 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:49.746 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:15:49.746 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:15:49.747 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.747 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.747 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.747 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.748 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.748 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.748 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.748 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.748 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:15:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:15:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.750 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.750 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.750 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.750 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.753 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.754 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:15:49.754 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:15:49.754 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:15:49.755 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:15:49.755 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:15:49.755 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:15:49.755 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:49.755 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:15:49.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:15:49.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:15:49.756 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:15:49.756 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:15:49.756 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:15:49.757 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:15:49.757 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:15:49.757 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:15:49.757 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:15:49.758 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:15:49.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:15:49.758 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:15:49.758 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:15:49.758 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:15:49.759 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:15:49.759 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.759 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:15:49.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.760 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:15:49.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.760 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:15:49.761 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:15:49.761 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.761 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.762 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:15:49.763 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.764 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:15:49.764 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.765 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.765 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.766 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:15:49.766 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:15:49.766 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.767 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.767 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.768 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:15:49.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:15:49.768 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.769 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.770 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:15:49.770 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.771 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.771 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:15:49.771 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.772 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.772 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.773 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:15:49.773 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:15:49.773 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.774 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.774 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:15:49.775 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:15:49.775 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:15:49.776 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:15:49.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.776 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:15:49.777 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:15:49.777 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:15:49.778 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:15:49.779 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:15:49.779 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.779 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.780 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:15:49.780 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:15:49.781 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:15:49.781 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:15:49.781 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:15:49.782 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.782 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.782 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:15:49.783 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:15:49.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.783 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.784 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.784 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:49.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.785 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.785 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:15:49.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.786 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.786 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.786 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:49.787 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.787 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.787 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:15:49.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.788 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.788 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.788 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:15:49.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.789 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:15:49.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:15:49.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:15:49.790 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:15:49.790 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:15:49.791 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:15:49.791 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:15:49.791 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:15:49.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:15:49.792 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:15:49.792 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:15:49.794 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:15:49.794 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:15:49.795 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:15:49.795 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:15:49.796 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:15:49.796 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:15:49.813 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:15:49.813 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:15:49.814 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:49.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.951+0000 7f9ef3fff700 1 Processor -- start 2020-11-20T12:15:49.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.951+0000 7f9ef3fff700 1 -- start start 2020-11-20T12:15:49.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef3fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40a9880 0x7f9ef40a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef3fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9ef40a6600 0x7f9ef40a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef3fff700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40a6b80 con 0x7f9ef40ad280 2020-11-20T12:15:49.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef3fff700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef40a71b0 con 0x7f9ef40a6600 2020-11-20T12:15:49.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef3fff700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40a6e20 con 0x7f9ef40a9880 2020-11-20T12:15:49.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef37fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9ef40ad280 0x7f9ef40a7630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45344/0 (socket says 172.21.15.44:45344) 2020-11-20T12:15:49.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef37fe700 1 -- 172.21.15.44:0/2081702138 learned_addr learned my addr 172.21.15.44:0/2081702138 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:49.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9eebfff700 1 -- 172.21.15.44:0/2081702138 >> v2:172.21.15.79:3300/0 conn(0x7f9ef40a6600 msgr2=0x7f9ef40a8af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:15:49.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9eebfff700 1 --2- 172.21.15.44:0/2081702138 >> v2:172.21.15.79:3300/0 conn(0x7f9ef40a6600 0x7f9ef40a8af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:49.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef2ffd700 1 --2- 172.21.15.44:0/2081702138 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40a9880 0x7f9ef40a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef27fc700 1 -- 172.21.15.44:0/2081702138 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128666450 0 0) 0x7f9ef40a6b80 con 0x7f9ef40ad280 2020-11-20T12:15:49.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef27fc700 1 -- 172.21.15.44:0/2081702138 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed4003240 con 0x7f9ef40ad280 2020-11-20T12:15:49.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef2ffd700 1 -- 172.21.15.44:0/2081702138 >> v2:172.21.15.79:3300/0 conn(0x7f9ef40a6600 msgr2=0x7f9ef40a8af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:49.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef2ffd700 1 --2- 172.21.15.44:0/2081702138 >> v2:172.21.15.79:3300/0 conn(0x7f9ef40a6600 0x7f9ef40a8af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:49.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef2ffd700 1 -- 172.21.15.44:0/2081702138 >> v1:172.21.15.44:6789/0 conn(0x7f9ef40ad280 legacy=0x7f9ef40a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.952+0000 7f9ef2ffd700 1 -- 172.21.15.44:0/2081702138 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40a4a50 con 0x7f9ef40a9880 2020-11-20T12:15:49.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.953+0000 7f9ef2ffd700 1 --2- 172.21.15.44:0/2081702138 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40a9880 0x7f9ef40a4ef0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f9ee400b9a0 tx=0x7f9ee4006df0).ready entity=mon.2 client_cookie=9eadebcfddf4086f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.953+0000 7f9ef27fc700 1 -- 172.21.15.44:0/2081702138 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4009840 con 0x7f9ef40a9880 2020-11-20T12:15:49.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.953+0000 7f9ef27fc700 1 -- 172.21.15.44:0/2081702138 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee40099a0 con 0x7f9ef40a9880 2020-11-20T12:15:49.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.953+0000 7f9ef27fc700 1 -- 172.21.15.44:0/2081702138 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4009d20 con 0x7f9ef40a9880 2020-11-20T12:15:49.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.953+0000 7f9ef3fff700 1 -- 172.21.15.44:0/2081702138 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40a9880 msgr2=0x7f9ef40a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.953+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/2081702138 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40a9880 0x7f9ef40a4ef0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f9ee400b9a0 tx=0x7f9ee4006df0).stop 2020-11-20T12:15:49.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.954+0000 7f9ef3fff700 1 -- 172.21.15.44:0/2081702138 shutdown_connections 2020-11-20T12:15:49.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.954+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/2081702138 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40a9880 0x7f9ef40a4ef0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.954+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/2081702138 >> v2:172.21.15.79:3300/0 conn(0x7f9ef40a6600 0x7f9ef40a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:49.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.954+0000 7f9ef3fff700 1 -- 172.21.15.44:0/2081702138 >> 172.21.15.44:0/2081702138 conn(0x7f9ef4061b60 msgr2=0x7f9ef4061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:49.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.954+0000 7f9ef3fff700 1 -- 172.21.15.44:0/2081702138 shutdown_connections 2020-11-20T12:15:49.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.954+0000 7f9ef3fff700 1 -- 172.21.15.44:0/2081702138 wait complete. 2020-11-20T12:15:49.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 Processor -- start 2020-11-20T12:15:49.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 -- start start 2020-11-20T12:15:49.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef40a6600 0x7f9ef409ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ef40a9880 0x7f9ef414a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40ad280 0x7f9ef40a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef409dca0 con 0x7f9ef40a6600 2020-11-20T12:15:49.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40673b0 con 0x7f9ef40a9880 2020-11-20T12:15:49.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef3fff700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef409df00 con 0x7f9ef40ad280 2020-11-20T12:15:49.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9eebfff700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ef40a9880 msgr2=0x7f9ef414a180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:15:49.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef40a6600 0x7f9ef409ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ef40a9880 0x7f9ef414a180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:49.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef40a6600 0x7f9ef409ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41598/0 (socket says 172.21.15.44:41598) 2020-11-20T12:15:49.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.955+0000 7f9ef2ffd700 1 -- 172.21.15.44:0/3024391991 learned_addr learned my addr 172.21.15.44:0/3024391991 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:49.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef37fe700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40ad280 0x7f9ef40a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef2ffd700 1 -- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40ad280 msgr2=0x7f9ef40a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:49.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef2ffd700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40ad280 0x7f9ef40a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:49.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef2ffd700 1 -- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ef40a9880 msgr2=0x7f9ef414a180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:49.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef2ffd700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ef40a9880 0x7f9ef414a180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:49.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef2ffd700 1 -- 172.21.15.44:0/3024391991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef409e1a0 con 0x7f9ef40a6600 2020-11-20T12:15:49.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef2ffd700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef40a6600 0x7f9ef409ebc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9ee400be60 tx=0x7f9ee4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9eeaffd700 1 -- 172.21.15.44:0/3024391991 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee401e460 con 0x7f9ef40a6600 2020-11-20T12:15:49.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9eeaffd700 1 -- 172.21.15.44:0/3024391991 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee401e5c0 con 0x7f9ef40a6600 2020-11-20T12:15:49.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9eeaffd700 1 -- 172.21.15.44:0/3024391991 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee401e960 con 0x7f9ef40a6600 2020-11-20T12:15:49.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef409e640 con 0x7f9ef40a6600 2020-11-20T12:15:49.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.956+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef414a8b0 con 0x7f9ef40a6600 2020-11-20T12:15:49.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.957+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef40aa4a0 con 0x7f9ef40a6600 2020-11-20T12:15:49.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.957+0000 7f9eeaffd700 1 -- 172.21.15.44:0/3024391991 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9ee401a040 con 0x7f9ef40a6600 2020-11-20T12:15:49.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.958+0000 7f9eeaffd700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ed0056cd0 0x7f9ed0059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.958+0000 7f9eeaffd700 1 -- 172.21.15.44:0/3024391991 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9ee4071590 con 0x7f9ef40a6600 2020-11-20T12:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.958+0000 7f9eebfff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ed0056cd0 0x7f9ed0059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.958+0000 7f9eebfff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ed0056cd0 0x7f9ed0059190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f9ef409f320 tx=0x7f9edc006ab0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:49.962+0000 7f9eeaffd700 1 -- 172.21.15.44:0/3024391991 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9ee4046990 con 0x7f9ef40a6600 2020-11-20T12:15:50.238 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.235+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9ef4057d80 con 0x7f9ef40a6600 2020-11-20T12:15:50.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.236+0000 7f9eeaffd700 1 -- 172.21.15.44:0/3024391991 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f9ee4023020 con 0x7f9ef40a6600 2020-11-20T12:15:50.239 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:50.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ed0056cd0 msgr2=0x7f9ed0059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:50.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ed0056cd0 0x7f9ed0059190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f9ef409f320 tx=0x7f9edc006ab0).stop 2020-11-20T12:15:50.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef40a6600 msgr2=0x7f9ef409ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:50.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef40a6600 0x7f9ef409ebc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9ee400be60 tx=0x7f9ee4015000).stop 2020-11-20T12:15:50.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 shutdown_connections 2020-11-20T12:15:50.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9ed0056cd0 0x7f9ed0059190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:50.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9ef40ad280 0x7f9ef40a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:50.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ef40a6600 0x7f9ef409ebc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 --2- 172.21.15.44:0/3024391991 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ef40a9880 0x7f9ef414a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.237+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 >> 172.21.15.44:0/3024391991 conn(0x7f9ef4061b60 msgr2=0x7f9ef4137710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.238+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 shutdown_connections 2020-11-20T12:15:50.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:50.238+0000 7f9ef3fff700 1 -- 172.21.15.44:0/3024391991 wait complete. 2020-11-20T12:15:53.253 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.393+0000 7fe1e3d36700 1 Processor -- start 2020-11-20T12:15:53.396 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.393+0000 7fe1e3d36700 1 -- start start 2020-11-20T12:15:53.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e3d36700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe1dc0aac10 0x7fe1dc13eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:53.399 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e3d36700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe1dc143630 0x7fe1dc145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:53.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e3d36700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc0ab390 con 0x7fe1dc0abc70 2020-11-20T12:15:53.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e3d36700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0a9f20 con 0x7fe1dc0aac10 2020-11-20T12:15:53.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e3d36700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0ac960 con 0x7fe1dc143630 2020-11-20T12:15:53.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe1dc0aac10 0x7fe1dc13eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:53.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e1ad2700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fe1dc0abc70 legacy=0x7fe1dc0a2710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:15:53.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e22d3700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7fe1dc143630 msgr2=0x7fe1dc145a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:15:53.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe1dc0aac10 0x7fe1dc13eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41616/0 (socket says 172.21.15.44:41616) 2020-11-20T12:15:53.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 -- 172.21.15.44:0/799688520 learned_addr learned my addr 172.21.15.44:0/799688520 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e22d3700 1 --2- 172.21.15.44:0/799688520 >> v2:172.21.15.79:3300/0 conn(0x7fe1dc143630 0x7fe1dc145a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 -- 172.21.15.44:0/799688520 >> v2:172.21.15.79:3300/0 conn(0x7fe1dc143630 msgr2=0x7fe1dc145a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:53.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 --2- 172.21.15.44:0/799688520 >> v2:172.21.15.79:3300/0 conn(0x7fe1dc143630 0x7fe1dc145a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 -- 172.21.15.44:0/799688520 >> v1:172.21.15.79:6789/0 conn(0x7fe1dc0abc70 legacy=0x7fe1dc0a2710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 -- 172.21.15.44:0/799688520 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0a1fd0 con 0x7fe1dc0aac10 2020-11-20T12:15:53.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e12d1700 1 --2- 172.21.15.44:0/799688520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0aac10 0x7fe1dc13eb10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe1dc0a9630 tx=0x7fe1cc006eb0).ready entity=mon.0 client_cookie=a42d9791c9bffa27 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e0ad0700 1 -- 172.21.15.44:0/799688520 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc009890 con 0x7fe1dc0aac10 2020-11-20T12:15:53.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e0ad0700 1 -- 172.21.15.44:0/799688520 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc0099f0 con 0x7fe1dc0aac10 2020-11-20T12:15:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.394+0000 7fe1e0ad0700 1 -- 172.21.15.44:0/799688520 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc009d70 con 0x7fe1dc0aac10 2020-11-20T12:15:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 -- 172.21.15.44:0/799688520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0aac10 msgr2=0x7fe1dc13eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/799688520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0aac10 0x7fe1dc13eb10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe1dc0a9630 tx=0x7fe1cc006eb0).stop 2020-11-20T12:15:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 -- 172.21.15.44:0/799688520 shutdown_connections 2020-11-20T12:15:53.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/799688520 >> v2:172.21.15.79:3300/0 conn(0x7fe1dc143630 0x7fe1dc145a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/799688520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0aac10 0x7fe1dc13eb10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:53.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 -- 172.21.15.44:0/799688520 >> 172.21.15.44:0/799688520 conn(0x7fe1dc060170 msgr2=0x7fe1dc060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 -- 172.21.15.44:0/799688520 shutdown_connections 2020-11-20T12:15:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.395+0000 7fe1e3d36700 1 -- 172.21.15.44:0/799688520 wait complete. 2020-11-20T12:15:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.396+0000 7fe1e3d36700 1 Processor -- start 2020-11-20T12:15:53.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.396+0000 7fe1e3d36700 1 -- start start 2020-11-20T12:15:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e3d36700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1dc0aac10 0x7fe1dc0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:53.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e3d36700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0abc70 0x7fe1dc0a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e3d36700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe1dc143630 0x7fe1dc0a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e3d36700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc132430 con 0x7fe1dc0abc70 2020-11-20T12:15:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e3d36700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0a8250 con 0x7fe1dc0aac10 2020-11-20T12:15:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e3d36700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0a6e90 con 0x7fe1dc143630 2020-11-20T12:15:53.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e1ad2700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1dc0aac10 msgr2=0x7fe1dc0a5ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:15:53.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e1ad2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1dc0aac10 0x7fe1dc0a5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:53.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0abc70 0x7fe1dc0a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:53.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0abc70 0x7fe1dc0a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41620/0 (socket says 172.21.15.44:41620) 2020-11-20T12:15:53.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 -- 172.21.15.44:0/2216724239 learned_addr learned my addr 172.21.15.44:0/2216724239 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e22d3700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe1dc143630 0x7fe1dc0a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 -- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe1dc143630 msgr2=0x7fe1dc0a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe1dc143630 0x7fe1dc0a7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:53.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 -- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1dc0aac10 msgr2=0x7fe1dc0a5ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:53.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1dc0aac10 0x7fe1dc0a5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:53.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 -- 172.21.15.44:0/2216724239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0a7130 con 0x7fe1dc0abc70 2020-11-20T12:15:53.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1e12d1700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0abc70 0x7fe1dc0a7870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe1cc006fd0 tx=0x7fe1cc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.397+0000 7fe1c9ffb700 1 -- 172.21.15.44:0/2216724239 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc01e460 con 0x7fe1dc0abc70 2020-11-20T12:15:53.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.398+0000 7fe1c9ffb700 1 -- 172.21.15.44:0/2216724239 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc01e5e0 con 0x7fe1dc0abc70 2020-11-20T12:15:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.398+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0a73d0 con 0x7fe1dc0abc70 2020-11-20T12:15:53.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.398+0000 7fe1c9ffb700 1 -- 172.21.15.44:0/2216724239 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc01e9a0 con 0x7fe1dc0abc70 2020-11-20T12:15:53.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.398+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc0a5fc0 con 0x7fe1dc0abc70 2020-11-20T12:15:53.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.399+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc061650 con 0x7fe1dc0abc70 2020-11-20T12:15:53.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.399+0000 7fe1c9ffb700 1 -- 172.21.15.44:0/2216724239 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe1cc025030 con 0x7fe1dc0abc70 2020-11-20T12:15:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.399+0000 7fe1c9ffb700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe1b4056cd0 0x7fe1b4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:53.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.400+0000 7fe1c9ffb700 1 -- 172.21.15.44:0/2216724239 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe1cc0718d0 con 0x7fe1dc0abc70 2020-11-20T12:15:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.400+0000 7fe1e1ad2700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe1b4056cd0 0x7fe1b4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:53.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.400+0000 7fe1e1ad2700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe1b4056cd0 0x7fe1b4059190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fe1d8003650 tx=0x7fe1d8006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:53.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.403+0000 7fe1c9ffb700 1 -- 172.21.15.44:0/2216724239 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe1cc046c50 con 0x7fe1dc0abc70 2020-11-20T12:15:53.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.668+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe1dc0a7560 con 0x7fe1dc0abc70 2020-11-20T12:15:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.668+0000 7fe1c9ffb700 1 -- 172.21.15.44:0/2216724239 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fe1cc023020 con 0x7fe1dc0abc70 2020-11-20T12:15:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe1b4056cd0 msgr2=0x7fe1b4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe1b4056cd0 0x7fe1b4059190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fe1d8003650 tx=0x7fe1d8006ae0).stop 2020-11-20T12:15:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0abc70 msgr2=0x7fe1dc0a7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0abc70 0x7fe1dc0a7870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe1cc006fd0 tx=0x7fe1cc015000).stop 2020-11-20T12:15:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 shutdown_connections 2020-11-20T12:15:53.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe1b4056cd0 0x7fe1b4059190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe1dc143630 0x7fe1dc0a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1dc0aac10 0x7fe1dc0a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:53.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 --2- 172.21.15.44:0/2216724239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1dc0abc70 0x7fe1dc0a7870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:53.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 >> 172.21.15.44:0/2216724239 conn(0x7fe1dc060170 msgr2=0x7fe1dc0679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:53.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 shutdown_connections 2020-11-20T12:15:53.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:53.670+0000 7fe1e3d36700 1 -- 172.21.15.44:0/2216724239 wait complete. 2020-11-20T12:15:53.680 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:56.686 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:15:56.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.816+0000 7feebba11700 1 Processor -- start 2020-11-20T12:15:56.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.816+0000 7feebba11700 1 -- start start 2020-11-20T12:15:56.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feebba11700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feeb40abc30 0x7feeb40666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.822 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feebba11700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feeb4066bc0 0x7feeb41382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feebba11700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb4067350 con 0x7feeb40ab620 2020-11-20T12:15:56.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feebba11700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb40a9b40 con 0x7feeb4066bc0 2020-11-20T12:15:56.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feebba11700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb40ab040 con 0x7feeb40abc30 2020-11-20T12:15:56.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb97ad700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7feeb40abc30 msgr2=0x7feeb40666d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:15:56.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb97ad700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feeb40abc30 0x7feeb40666d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:56.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb9fae700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feeb4066bc0 0x7feeb41382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb8fac700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7feeb40ab620 0x7feeb4137bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45380/0 (socket says 172.21.15.44:45380) 2020-11-20T12:15:56.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb9fae700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feeb4066bc0 0x7feeb41382d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41634/0 (socket says 172.21.15.44:41634) 2020-11-20T12:15:56.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb8fac700 1 -- 172.21.15.44:0/2589418032 learned_addr learned my addr 172.21.15.44:0/2589418032 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:15:56.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb9fae700 1 -- 172.21.15.44:0/2589418032 >> v2:172.21.15.79:3300/0 conn(0x7feeb40abc30 msgr2=0x7feeb40666d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:56.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb9fae700 1 --2- 172.21.15.44:0/2589418032 >> v2:172.21.15.79:3300/0 conn(0x7feeb40abc30 0x7feeb40666d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:56.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb9fae700 1 -- 172.21.15.44:0/2589418032 >> v1:172.21.15.44:6789/0 conn(0x7feeb40ab620 legacy=0x7feeb4137bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.817+0000 7feeb9fae700 1 -- 172.21.15.44:0/2589418032 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb4067880 con 0x7feeb4066bc0 2020-11-20T12:15:56.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feeb9fae700 1 --2- 172.21.15.44:0/2589418032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb4066bc0 0x7feeb41382d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7feeb4064350 tx=0x7feeac0091d0).ready entity=mon.0 client_cookie=6ee59e6f8e77a89 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:56.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feea3fff700 1 -- 172.21.15.44:0/2589418032 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00bbc0 con 0x7feeb4066bc0 2020-11-20T12:15:56.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feea3fff700 1 -- 172.21.15.44:0/2589418032 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac00bd20 con 0x7feeb4066bc0 2020-11-20T12:15:56.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feea3fff700 1 -- 172.21.15.44:0/2589418032 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac011610 con 0x7feeb4066bc0 2020-11-20T12:15:56.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 -- 172.21.15.44:0/2589418032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb4066bc0 msgr2=0x7feeb41382d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 --2- 172.21.15.44:0/2589418032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb4066bc0 0x7feeb41382d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7feeb4064350 tx=0x7feeac0091d0).stop 2020-11-20T12:15:56.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 -- 172.21.15.44:0/2589418032 shutdown_connections 2020-11-20T12:15:56.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 --2- 172.21.15.44:0/2589418032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb4066bc0 0x7feeb41382d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:56.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 --2- 172.21.15.44:0/2589418032 >> v2:172.21.15.79:3300/0 conn(0x7feeb40abc30 0x7feeb40666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:56.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 -- 172.21.15.44:0/2589418032 >> 172.21.15.44:0/2589418032 conn(0x7feeb4065290 msgr2=0x7feeb4060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:56.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 -- 172.21.15.44:0/2589418032 shutdown_connections 2020-11-20T12:15:56.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.818+0000 7feebba11700 1 -- 172.21.15.44:0/2589418032 wait complete. 2020-11-20T12:15:56.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 Processor -- start 2020-11-20T12:15:56.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 -- start start 2020-11-20T12:15:56.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feeb4066bc0 0x7feeb41368d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb40ab620 0x7feeb4137060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb40abc30 0x7feeb414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb4137550 con 0x7feeb40abc30 2020-11-20T12:15:56.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40ae5d0 con 0x7feeb40ab620 2020-11-20T12:15:56.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.820+0000 7feebba11700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40ae870 con 0x7feeb4066bc0 2020-11-20T12:15:56.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb8fac700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb40ab620 msgr2=0x7feeb4137060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:15:56.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb40abc30 0x7feeb414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb8fac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb40ab620 0x7feeb4137060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:56.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb40abc30 0x7feeb414de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41640/0 (socket says 172.21.15.44:41640) 2020-11-20T12:15:56.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 -- 172.21.15.44:0/2314453959 learned_addr learned my addr 172.21.15.44:0/2314453959 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:15:56.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb97ad700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feeb4066bc0 0x7feeb41368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 -- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feeb4066bc0 msgr2=0x7feeb41368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feeb4066bc0 0x7feeb41368d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:56.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 -- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb40ab620 msgr2=0x7feeb4137060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:15:56.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb40ab620 0x7feeb4137060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:56.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 -- 172.21.15.44:0/2314453959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb40aeb10 con 0x7feeb40abc30 2020-11-20T12:15:56.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feeb9fae700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb40abc30 0x7feeb414de10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7feeac011000 tx=0x7feeac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:56.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feea17fa700 1 -- 172.21.15.44:0/2314453959 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00bbc0 con 0x7feeb40abc30 2020-11-20T12:15:56.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feea17fa700 1 -- 172.21.15.44:0/2314453959 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac00be90 con 0x7feeb40abc30 2020-11-20T12:15:56.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb40ae090 con 0x7feeb40abc30 2020-11-20T12:15:56.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feea17fa700 1 -- 172.21.15.44:0/2314453959 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac010e70 con 0x7feeb40abc30 2020-11-20T12:15:56.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.821+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb414e540 con 0x7feeb40abc30 2020-11-20T12:15:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.822+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb40aeca0 con 0x7feeb40abc30 2020-11-20T12:15:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.823+0000 7feea17fa700 1 -- 172.21.15.44:0/2314453959 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7feeac017070 con 0x7feeb40abc30 2020-11-20T12:15:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.823+0000 7feea17fa700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee8c056cd0 0x7fee8c059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.823+0000 7feea17fa700 1 -- 172.21.15.44:0/2314453959 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7feeac0727d0 con 0x7feeb40abc30 2020-11-20T12:15:56.836 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.823+0000 7feeb97ad700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee8c056cd0 0x7fee8c059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.824+0000 7feeb97ad700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee8c056cd0 0x7fee8c059190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7feeb40a60b0 tx=0x7feeb0006c70).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:56.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:56.827+0000 7feea17fa700 1 -- 172.21.15.44:0/2314453959 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7feeac047aa0 con 0x7feeb40abc30 2020-11-20T12:15:57.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.097+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feeb405c2a0 con 0x7feeb40abc30 2020-11-20T12:15:57.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.098+0000 7feea17fa700 1 -- 172.21.15.44:0/2314453959 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7feeac015020 con 0x7feeb40abc30 2020-11-20T12:15:57.102 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:15:57.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee8c056cd0 msgr2=0x7fee8c059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee8c056cd0 0x7fee8c059190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7feeb40a60b0 tx=0x7feeb0006c70).stop 2020-11-20T12:15:57.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb40abc30 msgr2=0x7feeb414de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb40abc30 0x7feeb414de10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7feeac011000 tx=0x7feeac016000).stop 2020-11-20T12:15:57.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 shutdown_connections 2020-11-20T12:15:57.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee8c056cd0 0x7fee8c059190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feeb40abc30 0x7feeb414de10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feeb4066bc0 0x7feeb41368d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 --2- 172.21.15.44:0/2314453959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feeb40ab620 0x7feeb4137060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:57.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 >> 172.21.15.44:0/2314453959 conn(0x7feeb4065290 msgr2=0x7feeb41340a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:57.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 shutdown_connections 2020-11-20T12:15:57.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:15:57.100+0000 7feebba11700 1 -- 172.21.15.44:0/2314453959 wait complete. 2020-11-20T12:16:00.118 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:16:00.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.258+0000 7f5d70588700 1 Processor -- start 2020-11-20T12:16:00.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.258+0000 7f5d70588700 1 -- start start 2020-11-20T12:16:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.259+0000 7f5d70588700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.259+0000 7f5d70588700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d680a9770 con 0x7f5d680aa400 2020-11-20T12:16:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.259+0000 7f5d70588700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d680a6da0 con 0x7f5d680aa020 2020-11-20T12:16:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.259+0000 7f5d70588700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680a56c0 con 0x7f5d680a3e10 2020-11-20T12:16:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.259+0000 7f5d6e324700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f5d680aa020 legacy=0x7f5d680a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:16:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.259+0000 7f5d6db23700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5d680aa400 0x7f5d680a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45394/0 (socket says 172.21.15.44:45394) 2020-11-20T12:16:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.259+0000 7f5d6db23700 1 -- 172.21.15.44:0/1393152377 learned_addr learned my addr 172.21.15.44:0/1393152377 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:16:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.260+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.260+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:16:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.260+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.260+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.260+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:00.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.460+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v1:172.21.15.79:6789/0 conn(0x7f5d680aa020 legacy=0x7f5d680a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:16:00.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.461+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.461+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:16:00.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.461+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:00.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.461+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:00.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.461+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:16:00.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.861+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v1:172.21.15.79:6789/0 conn(0x7f5d680aa020 legacy=0x7f5d680a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:16:01.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.862+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.862+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:16:01.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.862+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:01.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.862+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:01.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:00.862+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:16:01.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:01.662+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v1:172.21.15.79:6789/0 conn(0x7f5d680aa020 legacy=0x7f5d680a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:16:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:01.663+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:01.663+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:16:01.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:01.663+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:01.663+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:01.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:01.663+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:16:03.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 msgr2=0x7f5d680a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680a3e10 0x7f5d680a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 -- 172.21.15.44:0/1393152377 >> v1:172.21.15.79:6789/0 conn(0x7f5d680aa020 legacy=0x7f5d680a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:03.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 -- 172.21.15.44:0/1393152377 >> v1:172.21.15.44:6789/0 conn(0x7f5d680aa400 legacy=0x7f5d680a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 0x7f5d48008300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 -- 172.21.15.44:0/1393152377 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d680a56c0 con 0x7f5d480033c0 2020-11-20T12:16:03.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.259+0000 7f5d57fff700 1 -- 172.21.15.44:0/1393152377 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d48008970 con 0x7f5d48005e40 2020-11-20T12:16:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.260+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.260+0000 7f5d6db23700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 msgr2=0x7f5d48008300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:03.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.260+0000 7f5d6db23700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 0x7f5d48008300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.260+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 msgr2=0x7f5d48005840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:16:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.260+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 msgr2=0x7f5d48005840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.260+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:03.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.260+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:03.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.461+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.461+0000 7f5d6db23700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 msgr2=0x7f5d48008300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:03.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.461+0000 7f5d6db23700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 0x7f5d48008300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:16:03.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.461+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 msgr2=0x7f5d48005840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:16:03.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.461+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 msgr2=0x7f5d48005840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:03.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.461+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:03.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.461+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:16:03.865 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6db23700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 msgr2=0x7f5d48008300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:03.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.44:3300/0 conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6db23700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 0x7f5d48008300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:16:03.866 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 msgr2=0x7f5d48008300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:03.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> v2:172.21.15.79:3300/0 conn(0x7f5d48005e40 0x7f5d48008300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:03.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6e324700 1 -- 172.21.15.44:0/1393152377 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680a7040 con 0x7f5d480033c0 2020-11-20T12:16:03.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6e324700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d480033c0 0x7f5d48005840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f5d48005e10 tx=0x7f5d64008040).ready entity=mon.0 client_cookie=7824c7ae8cd1c76c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.867 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6d322700 1 -- 172.21.15.44:0/1393152377 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400f610 con 0x7f5d480033c0 2020-11-20T12:16:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6d322700 1 -- 172.21.15.44:0/1393152377 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6400f770 con 0x7f5d480033c0 2020-11-20T12:16:03.868 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.862+0000 7f5d6d322700 1 -- 172.21.15.44:0/1393152377 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400faf0 con 0x7f5d480033c0 2020-11-20T12:16:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d70588700 1 -- 172.21.15.44:0/1393152377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d480033c0 msgr2=0x7f5d48005840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d70588700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d480033c0 0x7f5d48005840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f5d48005e10 tx=0x7f5d64008040).stop 2020-11-20T12:16:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d6eb25700 1 -- 172.21.15.44:0/1393152377 reap_dead start 2020-11-20T12:16:03.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d70588700 1 -- 172.21.15.44:0/1393152377 shutdown_connections 2020-11-20T12:16:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d70588700 1 --2- 172.21.15.44:0/1393152377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d480033c0 0x7f5d48005840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d70588700 1 -- 172.21.15.44:0/1393152377 >> 172.21.15.44:0/1393152377 conn(0x7f5d68062690 msgr2=0x7f5d680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d70588700 1 -- 172.21.15.44:0/1393152377 shutdown_connections 2020-11-20T12:16:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.863+0000 7f5d70588700 1 -- 172.21.15.44:0/1393152377 wait complete. 2020-11-20T12:16:03.872 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.864+0000 7f5d70588700 1 Processor -- start 2020-11-20T12:16:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.864+0000 7f5d70588700 1 -- start start 2020-11-20T12:16:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d70588700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d480033c0 0x7f5d681349e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d70588700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680673d0 0x7f5d680666c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d70588700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d680ac2f0 0x7f5d68066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d70588700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680678e0 con 0x7f5d680ac2f0 2020-11-20T12:16:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d70588700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68067b80 con 0x7f5d480033c0 2020-11-20T12:16:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d70588700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68066220 con 0x7f5d680673d0 2020-11-20T12:16:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d680ac2f0 0x7f5d68066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d680ac2f0 0x7f5d68066bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41680/0 (socket says 172.21.15.44:41680) 2020-11-20T12:16:03.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 -- 172.21.15.44:0/764035098 learned_addr learned my addr 172.21.15.44:0/764035098 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:03.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6e324700 1 -- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d480033c0 msgr2=0x7f5d681349e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:03.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6e324700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d480033c0 0x7f5d681349e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:03.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6db23700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680673d0 0x7f5d680666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 -- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680673d0 msgr2=0x7f5d680666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680673d0 0x7f5d680666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 -- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d480033c0 msgr2=0x7f5d681349e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:16:03.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d480033c0 0x7f5d681349e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:03.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.865+0000 7f5d6eb25700 1 -- 172.21.15.44:0/764035098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680a2bc0 con 0x7f5d680ac2f0 2020-11-20T12:16:03.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.866+0000 7f5d6eb25700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d680ac2f0 0x7f5d68066bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5d48008910 tx=0x7f5d60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.866+0000 7f5d567fc700 1 -- 172.21.15.44:0/764035098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010920 con 0x7f5d680ac2f0 2020-11-20T12:16:03.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.866+0000 7f5d567fc700 1 -- 172.21.15.44:0/764035098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6000c040 con 0x7f5d680ac2f0 2020-11-20T12:16:03.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.866+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d68133210 con 0x7f5d680ac2f0 2020-11-20T12:16:03.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.866+0000 7f5d567fc700 1 -- 172.21.15.44:0/764035098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010cd0 con 0x7f5d680ac2f0 2020-11-20T12:16:03.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.866+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d681336f0 con 0x7f5d680ac2f0 2020-11-20T12:16:03.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.867+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d680a7220 con 0x7f5d680ac2f0 2020-11-20T12:16:03.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.867+0000 7f5d567fc700 1 -- 172.21.15.44:0/764035098 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f5d60017680 con 0x7f5d680ac2f0 2020-11-20T12:16:03.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.867+0000 7f5d567fc700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5d40056cd0 0x7f5d40059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.867+0000 7f5d567fc700 1 -- 172.21.15.44:0/764035098 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f5d600707f0 con 0x7f5d680ac2f0 2020-11-20T12:16:03.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.868+0000 7f5d6e324700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5d40056cd0 0x7f5d40059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.868+0000 7f5d6e324700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5d40056cd0 0x7f5d40059190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5d48005e10 tx=0x7f5d64019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.880 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:03.871+0000 7f5d567fc700 1 -- 172.21.15.44:0/764035098 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5d60045ac0 con 0x7f5d680ac2f0 2020-11-20T12:16:04.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.140+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5d68061650 con 0x7f5d680ac2f0 2020-11-20T12:16:04.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.140+0000 7f5d567fc700 1 -- 172.21.15.44:0/764035098 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f5d600490e0 con 0x7f5d680ac2f0 2020-11-20T12:16:04.144 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":128,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:16:04.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5d40056cd0 msgr2=0x7f5d40059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5d40056cd0 0x7f5d40059190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5d48005e10 tx=0x7f5d64019000).stop 2020-11-20T12:16:04.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d680ac2f0 msgr2=0x7f5d68066bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d680ac2f0 0x7f5d68066bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5d48008910 tx=0x7f5d60008040).stop 2020-11-20T12:16:04.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 shutdown_connections 2020-11-20T12:16:04.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5d40056cd0 0x7f5d40059190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d680ac2f0 0x7f5d68066bb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d480033c0 0x7f5d681349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 --2- 172.21.15.44:0/764035098 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f5d680673d0 0x7f5d680666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 >> 172.21.15.44:0/764035098 conn(0x7f5d68062690 msgr2=0x7f5d680a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 shutdown_connections 2020-11-20T12:16:04.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.142+0000 7f5d70588700 1 -- 172.21.15.44:0/764035098 wait complete. 2020-11-20T12:16:04.159 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:16:04.159 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:16:04.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.291+0000 7f4890e39700 1 Processor -- start 2020-11-20T12:16:04.295 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.291+0000 7f4890e39700 1 -- start start 2020-11-20T12:16:04.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f4890e39700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f488c0b17c0 0x7f488c13abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f4890e39700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c0a9000 con 0x7f488c0b1ba0 2020-11-20T12:16:04.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f4890e39700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c0a92a0 con 0x7f488c0a9740 2020-11-20T12:16:04.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f4890e39700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c138600 con 0x7f488c0b17c0 2020-11-20T12:16:04.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f488b7fe700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f488c0b17c0 msgr2=0x7f488c13abc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:04.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f488affd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f488c0b1ba0 0x7f488c0aad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45432/0 (socket says 172.21.15.44:45432) 2020-11-20T12:16:04.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f488affd700 1 -- 172.21.15.44:0/2763220975 learned_addr learned my addr 172.21.15.44:0/2763220975 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:16:04.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f488b7fe700 1 --2- 172.21.15.44:0/2763220975 >> v2:172.21.15.79:3300/0 conn(0x7f488c0b17c0 0x7f488c13abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:04.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.292+0000 7f488bfff700 1 -- 172.21.15.44:0/2763220975 >> v1:172.21.15.79:6789/0 conn(0x7f488c0a9740 legacy=0x7f488c137ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:16:04.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488969407 0 0) 0x7f488c0a9000 con 0x7f488c0b1ba0 2020-11-20T12:16:04.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4878003240 con 0x7f488c0b1ba0 2020-11-20T12:16:04.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3744566069 0 0) 0x7f4878003240 con 0x7f488c0b1ba0 2020-11-20T12:16:04.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 >> v2:172.21.15.79:3300/0 conn(0x7f488c0b17c0 msgr2=0x7f488c13abc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:04.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 --2- 172.21.15.44:0/2763220975 >> v2:172.21.15.79:3300/0 conn(0x7f488c0b17c0 0x7f488c13abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 >> v1:172.21.15.79:6789/0 conn(0x7f488c0a9740 legacy=0x7f488c137ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:04.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0a4c30 con 0x7f488c0b1ba0 2020-11-20T12:16:04.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f48800014e0 con 0x7f488c0b1ba0 2020-11-20T12:16:04.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 >> v1:172.21.15.44:6789/0 conn(0x7f488c0b1ba0 legacy=0x7f488c0aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4878003730 0x7f4878003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004090 0x7f4878004560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4878004b60 0x7f487800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c138600 con 0x7f4878004b60 2020-11-20T12:16:04.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4878005160 con 0x7f4878004090 2020-11-20T12:16:04.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4878005320 con 0x7f4878003730 2020-11-20T12:16:04.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488bfff700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4878004b60 0x7f487800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488affd700 1 -- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004090 msgr2=0x7f4878004560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:04.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488affd700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004090 0x7f4878004560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:04.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488b7fe700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4878003730 0x7f4878003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488bfff700 1 -- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4878003730 msgr2=0x7f4878003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488bfff700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4878003730 0x7f4878003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488bfff700 1 -- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004090 msgr2=0x7f4878004560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488bfff700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004090 0x7f4878004560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.293+0000 7f488bfff700 1 -- 172.21.15.44:0/2763220975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878003240 con 0x7f4878004b60 2020-11-20T12:16:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f488bfff700 1 -- 172.21.15.44:0/2763220975 reap_dead start 2020-11-20T12:16:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f488bfff700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4878004b60 0x7f487800b730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4884009d00 tx=0x7f4884009270).ready entity=mon.0 client_cookie=5e9b929974fe5483 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884003c60 con 0x7f4878004b60 2020-11-20T12:16:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4884003de0 con 0x7f4878004b60 2020-11-20T12:16:04.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f488a7fc700 1 -- 172.21.15.44:0/2763220975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400bbc0 con 0x7f4878004b60 2020-11-20T12:16:04.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f4890e39700 1 -- 172.21.15.44:0/2763220975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4878004b60 msgr2=0x7f487800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f4890e39700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4878004b60 0x7f487800b730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4884009d00 tx=0x7f4884009270).stop 2020-11-20T12:16:04.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f4890e39700 1 -- 172.21.15.44:0/2763220975 shutdown_connections 2020-11-20T12:16:04.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f4890e39700 1 --2- 172.21.15.44:0/2763220975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4878004b60 0x7f487800b730 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f4890e39700 1 -- 172.21.15.44:0/2763220975 >> 172.21.15.44:0/2763220975 conn(0x7f488c060c10 msgr2=0x7f488c0b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f4890e39700 1 -- 172.21.15.44:0/2763220975 shutdown_connections 2020-11-20T12:16:04.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.294+0000 7f4890e39700 1 -- 172.21.15.44:0/2763220975 wait complete. 2020-11-20T12:16:04.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.296+0000 7f4890e39700 1 Processor -- start 2020-11-20T12:16:04.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.296+0000 7f4890e39700 1 -- start start 2020-11-20T12:16:04.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f4890e39700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004b60 0x7f488c09e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f4890e39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c13fa40 0x7f488c1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f4890e39700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f488c14d3f0 0x7f488c14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f4890e39700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c13f060 con 0x7f488c13fa40 2020-11-20T12:16:04.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f4890e39700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c13f300 con 0x7f4878004b60 2020-11-20T12:16:04.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f4890e39700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c13f5a0 con 0x7f488c14d3f0 2020-11-20T12:16:04.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488b7fe700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004b60 msgr2=0x7f488c09e3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:04.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c13fa40 0x7f488c1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004b60 0x7f488c09e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:04.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c13fa40 0x7f488c1405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41698/0 (socket says 172.21.15.44:41698) 2020-11-20T12:16:04.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 -- 172.21.15.44:0/750912977 learned_addr learned my addr 172.21.15.44:0/750912977 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:04.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488bfff700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f488c14d3f0 0x7f488c14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f488c14d3f0 msgr2=0x7f488c14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f488c14d3f0 0x7f488c14d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004b60 msgr2=0x7f488c09e3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:16:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4878004b60 0x7f488c09e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 -- 172.21.15.44:0/750912977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c09e8b0 con 0x7f488c13fa40 2020-11-20T12:16:04.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f488affd700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c13fa40 0x7f488c1405a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f488000cab0 tx=0x7f4880001ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880007170 con 0x7f488c13fa40 2020-11-20T12:16:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48800072d0 con 0x7f488c13fa40 2020-11-20T12:16:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.297+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c0a44d0 con 0x7f488c13fa40 2020-11-20T12:16:04.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.298+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c140a90 con 0x7f488c13fa40 2020-11-20T12:16:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.298+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488000a960 con 0x7f488c13fa40 2020-11-20T12:16:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.298+0000 7f4890e39700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c0611c0 0x7f488c14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.298+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c05c430 con 0x7f488c0611c0 2020-11-20T12:16:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.299+0000 7f488b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c0611c0 0x7f488c14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.299+0000 7f488b7fe700 1 -- 172.21.15.44:0/750912977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f488c14e230 con 0x7f488c0611c0 2020-11-20T12:16:04.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.299+0000 7f488b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c0611c0 0x7f488c14dd40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f487c00d5d0 tx=0x7f487c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.299+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f488001b060 con 0x7f488c13fa40 2020-11-20T12:16:04.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.299+0000 7f486b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4864056cd0 0x7f4864059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.299+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f488006f9c0 con 0x7f488c13fa40 2020-11-20T12:16:04.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.299+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c005320 con 0x7f488c0611c0 2020-11-20T12:16:04.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.300+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f487c00f040 con 0x7f488c0611c0 2020-11-20T12:16:04.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.300+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c0611c0 msgr2=0x7f488c14dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.300+0000 7f486b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c0611c0 0x7f488c14dd40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f487c00d5d0 tx=0x7f487c008040).stop 2020-11-20T12:16:04.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.300+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c0611c0 msgr2=0x7f488c14dd40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:04.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.300+0000 7f486b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c0611c0 0x7f488c14dd40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.300+0000 7f488bfff700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4864056cd0 0x7f4864059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.300+0000 7f488bfff700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4864056cd0 0x7f4864059190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f4878005130 tx=0x7f4884015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.308+0000 7f4890e39700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c149e20 0x7f488c14a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.308+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c057d80 con 0x7f488c149e20 2020-11-20T12:16:04.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.308+0000 7f488b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c149e20 0x7f488c14a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.308+0000 7f488b7fe700 1 -- 172.21.15.44:0/750912977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f488c0446d0 con 0x7f488c149e20 2020-11-20T12:16:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.308+0000 7f488b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c149e20 0x7f488c14a230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f487c00d5d0 tx=0x7f487c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c014a90 con 0x7f488c149e20 2020-11-20T12:16:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4597 (secure 0 0 0) 0x7f487c005110 con 0x7f488c149e20 2020-11-20T12:16:04.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c149e20 msgr2=0x7f488c14a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f486b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c149e20 0x7f488c14a230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f487c00d5d0 tx=0x7f487c017000).stop 2020-11-20T12:16:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f486b7fe700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c149e20 msgr2=0x7f488c14a230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f486b7fe700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c149e20 0x7f488c14a230 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4864056cd0 msgr2=0x7f4864059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4864056cd0 0x7f4864059190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f4878005130 tx=0x7f4884015000).stop 2020-11-20T12:16:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c13fa40 msgr2=0x7f488c1405a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 --2- 172.21.15.44:0/750912977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f488c13fa40 0x7f488c1405a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f488000cab0 tx=0x7f4880001ad0).stop 2020-11-20T12:16:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f488bfff700 1 -- 172.21.15.44:0/750912977 reap_dead start 2020-11-20T12:16:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f488bfff700 1 -- 172.21.15.44:0/750912977 reap_dead start 2020-11-20T12:16:04.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 shutdown_connections 2020-11-20T12:16:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 >> 172.21.15.44:0/750912977 conn(0x7f488c060c10 msgr2=0x7f488c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 shutdown_connections 2020-11-20T12:16:04.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.309+0000 7f4890e39700 1 -- 172.21.15.44:0/750912977 wait complete. 2020-11-20T12:16:04.325 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:04.326 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:04.326 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:04.326 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:16:04.326 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 128, 2020-11-20T12:16:04.327 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:04.327 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:04.327 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:04.327 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.327 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:16:04.328 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:04.328 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:04.328 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:04.328 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:04.328 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:04.329 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:04.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:04.329 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:04.329 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:04.329 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:04.330 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:04.330 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.330 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:04.330 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:04.330 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:04.331 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:04.331 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:04.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:04.331 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:04.331 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:04.332 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:04.332 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:04.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.332 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:04.333 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:04.333 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:04.333 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:04.333 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:04.334 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:04.334 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:04.334 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:04.334 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:04.334 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:04.335 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:04.335 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:04.335 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:04.335 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:04.335 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:04.336 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:04.336 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:04.336 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:04.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:04.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:04.337 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:04.337 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:04.337 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:04.338 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.338 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:04.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.338 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:04.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.339 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:04.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:04.339 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:04.339 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:04.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:04.340 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:04.340 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:04.341 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:04.341 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.341 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.342 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:04.342 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:04.342 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:04.343 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:04.343 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:04.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.343 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:04.344 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:04.344 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:04.344 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:04.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:04.345 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:04.345 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:04.346 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:04.346 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.346 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.346 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.347 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:04.347 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:04.347 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:04.347 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:04.347 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:04.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.348 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:04.348 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:04.348 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:04.349 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:04.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:04.349 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:04.349 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.350 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:04.350 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:04.350 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.351 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.351 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:04.351 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:04.352 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:04.352 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:04.352 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:04.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.352 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.353 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:04.353 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:04.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.353 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.354 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.354 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:04.354 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.354 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.354 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:16:04.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.355 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.355 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.355 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:04.355 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.356 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.356 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:16:04.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.356 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.356 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.357 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:04.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.357 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.357 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:04.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.358 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.358 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.358 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:16:04.358 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.358 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.359 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:04.359 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:04.360 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:16:04.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 Processor -- start 2020-11-20T12:16:04.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 -- start start 2020-11-20T12:16:04.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8d40aad00 0x7fb8d40a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40ab0e0 0x7fb8d413ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb8d40a86d0 0x7fb8d40abc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d41385f0 con 0x7fb8d40aad00 2020-11-20T12:16:04.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d413e250 con 0x7fb8d40a86d0 2020-11-20T12:16:04.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.510+0000 7fb8d9dd6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d413e4f0 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8d40aad00 0x7fb8d40a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8d40aad00 0x7fb8d40a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41708/0 (socket says 172.21.15.44:41708) 2020-11-20T12:16:04.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 -- 172.21.15.44:0/2361604945 learned_addr learned my addr 172.21.15.44:0/2361604945 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:04.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d3fff700 1 -- 172.21.15.44:0/2361604945 >> v2:172.21.15.79:3300/0 conn(0x7fb8d40a86d0 msgr2=0x7fb8d40abc30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:04.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d3fff700 1 --2- 172.21.15.44:0/2361604945 >> v2:172.21.15.79:3300/0 conn(0x7fb8d40a86d0 0x7fb8d40abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:04.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d2ffd700 1 --2- 172.21.15.44:0/2361604945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40ab0e0 0x7fb8d413ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 -- 172.21.15.44:0/2361604945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40ab0e0 msgr2=0x7fb8d413ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:04.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/2361604945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40ab0e0 0x7fb8d413ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 -- 172.21.15.44:0/2361604945 >> v2:172.21.15.79:3300/0 conn(0x7fb8d40a86d0 msgr2=0x7fb8d40abc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:04.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/2361604945 >> v2:172.21.15.79:3300/0 conn(0x7fb8d40a86d0 0x7fb8d40abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 -- 172.21.15.44:0/2361604945 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40ac120 con 0x7fb8d40aad00 2020-11-20T12:16:04.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/2361604945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40aad00 0x7fb8d40a94d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb8d40a99c0 tx=0x7fb8c4006eb0).ready entity=mon.0 client_cookie=3a4a07303b14a49c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2361604945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40098a0 con 0x7fb8d40aad00 2020-11-20T12:16:04.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2361604945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c4009a00 con 0x7fb8d40aad00 2020-11-20T12:16:04.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.511+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2361604945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4009d80 con 0x7fb8d40aad00 2020-11-20T12:16:04.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/2361604945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40aad00 msgr2=0x7fb8d40a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/2361604945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40aad00 0x7fb8d40a94d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb8d40a99c0 tx=0x7fb8c4006eb0).stop 2020-11-20T12:16:04.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/2361604945 shutdown_connections 2020-11-20T12:16:04.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/2361604945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40aad00 0x7fb8d40a94d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/2361604945 >> v2:172.21.15.79:3300/0 conn(0x7fb8d40a86d0 0x7fb8d40abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/2361604945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40ab0e0 0x7fb8d413ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/2361604945 >> 172.21.15.44:0/2361604945 conn(0x7fb8d4060170 msgr2=0x7fb8d4060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/2361604945 shutdown_connections 2020-11-20T12:16:04.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.512+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/2361604945 wait complete. 2020-11-20T12:16:04.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.513+0000 7fb8d9dd6700 1 Processor -- start 2020-11-20T12:16:04.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.513+0000 7fb8d9dd6700 1 -- start start 2020-11-20T12:16:04.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.513+0000 7fb8d9dd6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a86d0 0x7fb8d409e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.513+0000 7fb8d9dd6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d40aad00 0x7fb8d40a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.513+0000 7fb8d9dd6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40ab0e0 0x7fb8d4149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d9dd6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40a5260 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d9dd6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40a5bf0 con 0x7fb8d40aad00 2020-11-20T12:16:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d9dd6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40a5e90 con 0x7fb8d40a86d0 2020-11-20T12:16:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40ab0e0 0x7fb8d4149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d2ffd700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d40aad00 msgr2=0x7fb8d40a7220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40ab0e0 0x7fb8d4149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41718/0 (socket says 172.21.15.44:41718) 2020-11-20T12:16:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 -- 172.21.15.44:0/3930918990 learned_addr learned my addr 172.21.15.44:0/3930918990 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d2ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d40aad00 0x7fb8d40a7220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:04.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a86d0 0x7fb8d409e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a86d0 msgr2=0x7fb8d409e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a86d0 0x7fb8d409e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d40aad00 msgr2=0x7fb8d40a7220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:16:04.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d40aad00 0x7fb8d40a7220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 -- 172.21.15.44:0/3930918990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40a6130 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d3fff700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40ab0e0 0x7fb8d4149fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb8cc00bae0 tx=0x7fb8cc006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc003c10 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc003d70 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc006170 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d40a63d0 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.514+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d40a7790 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.515+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d414a520 0x7fb8d40a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.515+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d405c5f0 con 0x7fb8d414a520 2020-11-20T12:16:04.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.515+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d414a520 0x7fb8d40a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb8cc0065a0 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8d37fe700 1 -- 172.21.15.44:0/3930918990 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8d4069680 con 0x7fb8d414a520 2020-11-20T12:16:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d414a520 0x7fb8d40a3ac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fb8d40a99c0 tx=0x7fb8c4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8ac056cd0 0x7fb8ac059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb8cc011020 con 0x7fb8d40ab0e0 2020-11-20T12:16:04.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8d2ffd700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8ac056cd0 0x7fb8ac059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4011de0 con 0x7fb8d414a520 2020-11-20T12:16:04.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb8c401f440 con 0x7fb8d414a520 2020-11-20T12:16:04.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d414a520 msgr2=0x7fb8d40a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d414a520 0x7fb8d40a3ac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fb8d40a99c0 tx=0x7fb8c4015000).stop 2020-11-20T12:16:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d414a520 msgr2=0x7fb8d40a3ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:04.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.516+0000 7fb8b37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d414a520 0x7fb8d40a3ac0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0x7fb8d40a99c0 tx=0x7fb8c4015000).stop 2020-11-20T12:16:04.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.517+0000 7fb8d2ffd700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8ac056cd0 0x7fb8ac059190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fb8d414b0b0 tx=0x7fb8c8006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.525+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a3fb0 0x7fb8d414d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.525+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40446d0 con 0x7fb8d40a3fb0 2020-11-20T12:16:04.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.525+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a3fb0 0x7fb8d414d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8d37fe700 1 -- 172.21.15.44:0/3930918990 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb8d40613c0 con 0x7fb8d40a3fb0 2020-11-20T12:16:04.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8d37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a3fb0 0x7fb8d414d5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb8d405ef10 tx=0x7fb8c4020000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c401fe50 con 0x7fb8d40a3fb0 2020-11-20T12:16:04.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4765 (secure 0 0 0) 0x7fb8c4011bd0 con 0x7fb8d40a3fb0 2020-11-20T12:16:04.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a3fb0 msgr2=0x7fb8d414d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8b37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a3fb0 0x7fb8d414d5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb8d405ef10 tx=0x7fb8c4020000).stop 2020-11-20T12:16:04.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8b37fe700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a3fb0 msgr2=0x7fb8d414d5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:04.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.526+0000 7fb8b37fe700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8d40a3fb0 0x7fb8d414d5a0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0x7fb8d405ef10 tx=0x7fb8c4020000).stop 2020-11-20T12:16:04.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8ac056cd0 msgr2=0x7fb8ac059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8ac056cd0 0x7fb8ac059190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fb8d414b0b0 tx=0x7fb8c8006df0).stop 2020-11-20T12:16:04.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40ab0e0 msgr2=0x7fb8d4149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 --2- 172.21.15.44:0/3930918990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8d40ab0e0 0x7fb8d4149fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb8cc00bae0 tx=0x7fb8cc006f90).stop 2020-11-20T12:16:04.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d3fff700 1 -- 172.21.15.44:0/3930918990 reap_dead start 2020-11-20T12:16:04.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d3fff700 1 -- 172.21.15.44:0/3930918990 reap_dead start 2020-11-20T12:16:04.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 shutdown_connections 2020-11-20T12:16:04.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 >> 172.21.15.44:0/3930918990 conn(0x7fb8d4060170 msgr2=0x7fb8d413cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 shutdown_connections 2020-11-20T12:16:04.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:04.527+0000 7fb8d9dd6700 1 -- 172.21.15.44:0/3930918990 wait complete. 2020-11-20T12:16:04.555 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:04.557 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:04.557 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:04.557 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:16:04.558 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 128, 2020-11-20T12:16:04.558 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:04.559 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:04.559 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:04.559 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.560 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:16:04.560 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:04.561 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:04.561 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:04.562 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:04.562 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:04.562 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:04.563 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:04.563 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:04.564 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:04.564 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:04.564 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:04.565 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.565 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:04.566 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:04.566 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:04.566 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:04.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:04.567 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:04.568 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:04.568 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:04.568 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:04.569 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:04.569 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.570 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:04.571 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:04.571 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:04.571 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:04.571 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:04.572 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:04.572 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:04.572 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:04.572 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:04.573 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:04.573 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:04.573 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:04.573 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:04.573 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:04.574 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:04.574 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:04.574 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:04.574 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:04.574 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:04.575 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:04.575 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:04.575 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:04.575 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:04.575 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.576 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:04.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.576 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:04.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.576 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:04.577 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:04.577 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:04.577 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:04.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:04.578 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:04.578 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:04.579 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:04.579 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.579 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.579 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.580 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:04.580 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:04.580 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:04.580 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:04.581 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:04.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.581 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:04.582 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:04.582 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:04.582 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:04.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:04.583 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:04.583 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:04.584 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:04.584 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.584 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.585 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:04.585 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:04.585 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:04.585 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:04.586 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:04.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.586 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:04.586 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:04.587 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:04.587 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:04.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:04.587 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:04.588 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:04.588 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:04.589 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:04.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.589 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.590 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:04.590 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:04.590 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:04.590 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:04.590 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:04.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.591 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.591 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:04.591 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:04.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.592 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.592 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.592 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:04.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.593 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.593 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:16:04.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.593 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.593 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.594 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:04.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.594 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.594 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:16:04.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.595 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.595 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.595 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:04.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.596 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.596 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:04.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.596 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.596 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.597 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:04.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.597 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:04.597 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:16:04.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:04.598 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:04.598 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:04.598 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:16:04.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:04.598 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:04.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:04.599 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:04.599 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:04.600 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:16:08.160 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:08.166 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:24.601 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:16:24.601 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:16:24.602 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:16:24.602 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:16:24.611 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:16:24.611 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:16:24.612 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:16:24.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.746+0000 7fd6e740e700 1 Processor -- start 2020-11-20T12:16:24.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.746+0000 7fd6e740e700 1 -- start start 2020-11-20T12:16:24.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e740e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e00a9460 0x7fd6e00aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:24.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e740e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6e013b780 0x7fd6e0137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:24.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e740e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00ab310 con 0x7fd6e00a4af0 2020-11-20T12:16:24.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e740e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e013ac80 con 0x7fd6e013b780 2020-11-20T12:16:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e740e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00ab5b0 con 0x7fd6e00a9460 2020-11-20T12:16:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e59ab700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6e013b780 0x7fd6e0137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e59ab700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6e013b780 0x7fd6e0137c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41730/0 (socket says 172.21.15.44:41730) 2020-11-20T12:16:24.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e59ab700 1 -- 172.21.15.44:0/1334119358 learned_addr learned my addr 172.21.15.44:0/1334119358 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e51aa700 1 --2- 172.21.15.44:0/1334119358 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e00a9460 0x7fd6e00aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e59ab700 1 -- 172.21.15.44:0/1334119358 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e00a9460 msgr2=0x7fd6e00aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e59ab700 1 --2- 172.21.15.44:0/1334119358 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e00a9460 0x7fd6e00aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e59ab700 1 -- 172.21.15.44:0/1334119358 >> v1:172.21.15.79:6789/0 conn(0x7fd6e00a4af0 legacy=0x7fd6e013b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.747+0000 7fd6e59ab700 1 -- 172.21.15.44:0/1334119358 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00a4f50 con 0x7fd6e013b780 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e59ab700 1 --2- 172.21.15.44:0/1334119358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e013b780 0x7fd6e0137c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fd6d8009d50 tx=0x7fd6d8009290).ready entity=mon.0 client_cookie=bea016b905ea4ed9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6cffff700 1 -- 172.21.15.44:0/1334119358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800bbc0 con 0x7fd6e013b780 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6cffff700 1 -- 172.21.15.44:0/1334119358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d800bd20 con 0x7fd6e013b780 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6cffff700 1 -- 172.21.15.44:0/1334119358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8011610 con 0x7fd6e013b780 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 -- 172.21.15.44:0/1334119358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e013b780 msgr2=0x7fd6e0137c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1334119358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e013b780 0x7fd6e0137c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fd6d8009d50 tx=0x7fd6d8009290).stop 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 -- 172.21.15.44:0/1334119358 shutdown_connections 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1334119358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e013b780 0x7fd6e0137c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1334119358 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e00a9460 0x7fd6e00aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 -- 172.21.15.44:0/1334119358 >> 172.21.15.44:0/1334119358 conn(0x7fd6e00621c0 msgr2=0x7fd6e0060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 -- 172.21.15.44:0/1334119358 shutdown_connections 2020-11-20T12:16:24.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.748+0000 7fd6e740e700 1 -- 172.21.15.44:0/1334119358 wait complete. 2020-11-20T12:16:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.749+0000 7fd6e740e700 1 Processor -- start 2020-11-20T12:16:24.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e740e700 1 -- start start 2020-11-20T12:16:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e740e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e00a4af0 0x7fd6e013e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e740e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 0x7fd6e0135e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e740e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e013b780 0x7fd6e0130830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e740e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0138a30 con 0x7fd6e00a4af0 2020-11-20T12:16:24.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e740e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e013c080 con 0x7fd6e00a9460 2020-11-20T12:16:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e740e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e013c320 con 0x7fd6e013b780 2020-11-20T12:16:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e51aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e00a4af0 0x7fd6e013e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e51aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e00a4af0 0x7fd6e013e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41734/0 (socket says 172.21.15.44:41734) 2020-11-20T12:16:24.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.750+0000 7fd6e51aa700 1 -- 172.21.15.44:0/1166412967 learned_addr learned my addr 172.21.15.44:0/1166412967 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e59ab700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e013b780 0x7fd6e0130830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e49a9700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 0x7fd6e0135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e51aa700 1 -- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e013b780 msgr2=0x7fd6e0130830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e51aa700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e013b780 0x7fd6e0130830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:24.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e51aa700 1 -- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 msgr2=0x7fd6e0135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e49a9700 1 -- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 msgr2=0x7fd6e0135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:16:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e49a9700 1 -- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 msgr2=0x7fd6e0135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:16:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e49a9700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 0x7fd6e0135e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:24.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e49a9700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 0x7fd6e0135e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e51aa700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 0x7fd6e0135e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e51aa700 1 -- 172.21.15.44:0/1166412967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0136db0 con 0x7fd6e00a4af0 2020-11-20T12:16:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e59ab700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e013b780 0x7fd6e0130830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e51aa700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e00a4af0 0x7fd6e013e390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd6dc00eba0 tx=0x7fd6dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:24.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6cd7fa700 1 -- 172.21.15.44:0/1166412967 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00ca60 con 0x7fd6e00a4af0 2020-11-20T12:16:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6cd7fa700 1 -- 172.21.15.44:0/1166412967 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc00cbc0 con 0x7fd6e00a4af0 2020-11-20T12:16:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e0137050 con 0x7fd6e00a4af0 2020-11-20T12:16:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6cd7fa700 1 -- 172.21.15.44:0/1166412967 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0123f0 con 0x7fd6e00a4af0 2020-11-20T12:16:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.751+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e0140de0 con 0x7fd6e00a4af0 2020-11-20T12:16:24.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.752+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e00aaee0 con 0x7fd6e00a4af0 2020-11-20T12:16:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.753+0000 7fd6cd7fa700 1 -- 172.21.15.44:0/1166412967 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd6dc0127a0 con 0x7fd6e00a4af0 2020-11-20T12:16:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.753+0000 7fd6cd7fa700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6b8056d50 0x7fd6b8059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:24.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.753+0000 7fd6cd7fa700 1 -- 172.21.15.44:0/1166412967 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd6dc070560 con 0x7fd6e00a4af0 2020-11-20T12:16:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.757+0000 7fd6cd7fa700 1 -- 172.21.15.44:0/1166412967 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd6dc045830 con 0x7fd6e00a4af0 2020-11-20T12:16:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.757+0000 7fd6e49a9700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6b8056d50 0x7fd6b8059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:24.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:24.758+0000 7fd6e49a9700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6b8056d50 0x7fd6b8059210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fd6e013d510 tx=0x7fd6d0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:25.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.028+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd6e0060880 con 0x7fd6e00a4af0 2020-11-20T12:16:25.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.029+0000 7fd6cd7fa700 1 -- 172.21.15.44:0/1166412967 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fd6dc048e50 con 0x7fd6e00a4af0 2020-11-20T12:16:25.033 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":128,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:16:25.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6b8056d50 msgr2=0x7fd6b8059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6b8056d50 0x7fd6b8059210 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fd6e013d510 tx=0x7fd6d0009040).stop 2020-11-20T12:16:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e00a4af0 msgr2=0x7fd6e013e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e00a4af0 0x7fd6e013e390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd6dc00eba0 tx=0x7fd6dc006df0).stop 2020-11-20T12:16:25.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 shutdown_connections 2020-11-20T12:16:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6b8056d50 0x7fd6b8059210 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e013b780 0x7fd6e0130830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e00a4af0 0x7fd6e013e390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 --2- 172.21.15.44:0/1166412967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e00a9460 0x7fd6e0135e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 >> 172.21.15.44:0/1166412967 conn(0x7fd6e00621c0 msgr2=0x7fd6e0133a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:25.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 shutdown_connections 2020-11-20T12:16:25.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:25.031+0000 7fd6e740e700 1 -- 172.21.15.44:0/1166412967 wait complete. 2020-11-20T12:16:28.046 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:16:28.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.183+0000 7f0973057700 1 Processor -- start 2020-11-20T12:16:28.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.183+0000 7f0973057700 1 -- start start 2020-11-20T12:16:28.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.184+0000 7f0973057700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f096c0ad440 0x7f096c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.184+0000 7f0973057700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 0x7f096c0a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.184+0000 7f0973057700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c0a70f0 con 0x7f096c0a9bb0 2020-11-20T12:16:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.184+0000 7f0973057700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096c0a5880 con 0x7f096c0ad440 2020-11-20T12:16:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.184+0000 7f0973057700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096c0a4170 con 0x7f096c0a4b00 2020-11-20T12:16:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.184+0000 7f0972055700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f096c0ad440 0x7f096c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0972055700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f096c0ad440 0x7f096c0a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41740/0 (socket says 172.21.15.44:41740) 2020-11-20T12:16:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0972055700 1 -- 172.21.15.44:0/993424229 learned_addr learned my addr 172.21.15.44:0/993424229 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:28.191 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971854700 1 --2- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 0x7f096c0a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971854700 1 -- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 msgr2=0x7f096c0a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:16:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971854700 1 -- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 msgr2=0x7f096c0a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971854700 1 --2- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 0x7f096c0a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971854700 1 --2- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 0x7f096c0a96c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0972055700 1 -- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 msgr2=0x7f096c0a96c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:16:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0972055700 1 --2- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 0x7f096c0a96c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0972055700 1 -- 172.21.15.44:0/993424229 >> v1:172.21.15.79:6789/0 conn(0x7f096c0a9bb0 legacy=0x7f096c134260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0972055700 1 -- 172.21.15.44:0/993424229 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c0ad820 con 0x7f096c0ad440 2020-11-20T12:16:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0972055700 1 --2- 172.21.15.44:0/993424229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ad440 0x7f096c0a4610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f096c0a9610 tx=0x7f0968006eb0).ready entity=mon.0 client_cookie=5464f5b30eeafbe8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971053700 1 -- 172.21.15.44:0/993424229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968009960 con 0x7f096c0ad440 2020-11-20T12:16:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971053700 1 -- 172.21.15.44:0/993424229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968009ac0 con 0x7f096c0ad440 2020-11-20T12:16:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.185+0000 7f0971053700 1 -- 172.21.15.44:0/993424229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968009e40 con 0x7f096c0ad440 2020-11-20T12:16:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 -- 172.21.15.44:0/993424229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ad440 msgr2=0x7f096c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 --2- 172.21.15.44:0/993424229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ad440 0x7f096c0a4610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f096c0a9610 tx=0x7f0968006eb0).stop 2020-11-20T12:16:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 -- 172.21.15.44:0/993424229 shutdown_connections 2020-11-20T12:16:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 --2- 172.21.15.44:0/993424229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0ad440 0x7f096c0a4610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 --2- 172.21.15.44:0/993424229 >> v2:172.21.15.79:3300/0 conn(0x7f096c0a4b00 0x7f096c0a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 -- 172.21.15.44:0/993424229 >> 172.21.15.44:0/993424229 conn(0x7f096c061b60 msgr2=0x7f096c061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:28.195 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 -- 172.21.15.44:0/993424229 shutdown_connections 2020-11-20T12:16:28.196 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.186+0000 7f0973057700 1 -- 172.21.15.44:0/993424229 wait complete. 2020-11-20T12:16:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.187+0000 7f0973057700 1 Processor -- start 2020-11-20T12:16:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.187+0000 7f0973057700 1 -- start start 2020-11-20T12:16:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0973057700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 0x7f096c0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0973057700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0a9bb0 0x7f096c067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0973057700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f096c0ad440 0x7f096c130290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0973057700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0648f0 con 0x7f096c0a9bb0 2020-11-20T12:16:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0973057700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0669b0 con 0x7f096c0a4b00 2020-11-20T12:16:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0973057700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c066c50 con 0x7f096c0ad440 2020-11-20T12:16:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0a9bb0 0x7f096c067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0a9bb0 0x7f096c067410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41748/0 (socket says 172.21.15.44:41748) 2020-11-20T12:16:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 -- 172.21.15.44:0/206728679 learned_addr learned my addr 172.21.15.44:0/206728679 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0972055700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 0x7f096c0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0972856700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f096c0ad440 0x7f096c130290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0972055700 1 -- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 msgr2=0x7f096c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:16:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0972055700 1 -- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 msgr2=0x7f096c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:16:28.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 -- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f096c0ad440 msgr2=0x7f096c130290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:28.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0972055700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 0x7f096c0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:28.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f096c0ad440 0x7f096c130290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 -- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 msgr2=0x7f096c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0972055700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 0x7f096c0abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 0x7f096c0abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 -- 172.21.15.44:0/206728679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c066f50 con 0x7f096c0a9bb0 2020-11-20T12:16:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0972856700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f096c0ad440 0x7f096c130290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.188+0000 7f0971854700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0a9bb0 0x7f096c067410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f095c00db30 tx=0x7f095c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:28.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.189+0000 7f095a7fc700 1 -- 172.21.15.44:0/206728679 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c009730 con 0x7f096c0a9bb0 2020-11-20T12:16:28.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.189+0000 7f095a7fc700 1 -- 172.21.15.44:0/206728679 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c009890 con 0x7f096c0a9bb0 2020-11-20T12:16:28.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.189+0000 7f095a7fc700 1 -- 172.21.15.44:0/206728679 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c009c50 con 0x7f096c0a9bb0 2020-11-20T12:16:28.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.189+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c0a2c20 con 0x7f096c0a9bb0 2020-11-20T12:16:28.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.189+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c067900 con 0x7f096c0a9bb0 2020-11-20T12:16:28.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.190+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c067170 con 0x7f096c0a9bb0 2020-11-20T12:16:28.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.190+0000 7f095a7fc700 1 -- 172.21.15.44:0/206728679 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f095c013480 con 0x7f096c0a9bb0 2020-11-20T12:16:28.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.190+0000 7f095a7fc700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0944056d20 0x7f09440591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:28.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.190+0000 7f095a7fc700 1 -- 172.21.15.44:0/206728679 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f095c0700a0 con 0x7f096c0a9bb0 2020-11-20T12:16:28.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.191+0000 7f0972055700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0944056d20 0x7f09440591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:28.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.191+0000 7f0972055700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0944056d20 0x7f09440591e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f0968006ef0 tx=0x7f0968015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:28.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.194+0000 7f095a7fc700 1 -- 172.21.15.44:0/206728679 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f096c067170 con 0x7f096c0a9bb0 2020-11-20T12:16:28.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.461+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f096c067140 con 0x7f096c0a9bb0 2020-11-20T12:16:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.461+0000 7f095a7fc700 1 -- 172.21.15.44:0/206728679 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f095c045310 con 0x7f096c0a9bb0 2020-11-20T12:16:28.465 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":128,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:16:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0944056d20 msgr2=0x7f09440591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0944056d20 0x7f09440591e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f0968006ef0 tx=0x7f0968015000).stop 2020-11-20T12:16:28.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0a9bb0 msgr2=0x7f096c067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0a9bb0 0x7f096c067410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f095c00db30 tx=0x7f095c008040).stop 2020-11-20T12:16:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 shutdown_connections 2020-11-20T12:16:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0944056d20 0x7f09440591e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f096c0ad440 0x7f096c130290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f096c0a4b00 0x7f096c0abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 --2- 172.21.15.44:0/206728679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f096c0a9bb0 0x7f096c067410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:28.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 >> 172.21.15.44:0/206728679 conn(0x7f096c061b60 msgr2=0x7f096c0608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:28.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 shutdown_connections 2020-11-20T12:16:28.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:28.463+0000 7f0973057700 1 -- 172.21.15.44:0/206728679 wait complete. 2020-11-20T12:16:31.479 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:16:31.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.612+0000 7fb143d32700 1 Processor -- start 2020-11-20T12:16:31.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.612+0000 7fb143d32700 1 -- start start 2020-11-20T12:16:31.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb143d32700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb13c061560 0x7fb13c13a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb143d32700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c065b20 con 0x7fb13c0681e0 2020-11-20T12:16:31.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb143d32700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c065dc0 con 0x7fb13c061180 2020-11-20T12:16:31.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb143d32700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c0a9cc0 con 0x7fb13c061560 2020-11-20T12:16:31.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb1412cd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb13c061560 0x7fb13c13a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb1422cf700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb13c0681e0 0x7fb13c141a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45508/0 (socket says 172.21.15.44:45508) 2020-11-20T12:16:31.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb1412cd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb13c061560 0x7fb13c13a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41758/0 (socket says 172.21.15.44:41758) 2020-11-20T12:16:31.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.613+0000 7fb1422cf700 1 -- 172.21.15.44:0/3656785833 learned_addr learned my addr 172.21.15.44:0/3656785833 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:16:31.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb1412cd700 1 -- 172.21.15.44:0/3656785833 >> v1:172.21.15.79:6789/0 conn(0x7fb13c061180 legacy=0x7fb13c0a95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb1412cd700 1 -- 172.21.15.44:0/3656785833 >> v1:172.21.15.44:6789/0 conn(0x7fb13c0681e0 legacy=0x7fb13c141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb1412cd700 1 -- 172.21.15.44:0/3656785833 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c066060 con 0x7fb13c061560 2020-11-20T12:16:31.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb140acc700 1 -- 172.21.15.44:0/3656785833 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160831521 0 0) 0x7fb13c065b20 con 0x7fb13c0681e0 2020-11-20T12:16:31.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb1412cd700 1 --2- 172.21.15.44:0/3656785833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c061560 0x7fb13c13a660 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb12c00cb30 tx=0x7fb12c006e80).ready entity=mon.0 client_cookie=97bc6eb549214827 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb140acc700 1 -- 172.21.15.44:0/3656785833 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c014070 con 0x7fb13c061560 2020-11-20T12:16:31.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb140acc700 1 -- 172.21.15.44:0/3656785833 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c00a8d0 con 0x7fb13c061560 2020-11-20T12:16:31.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.614+0000 7fb140acc700 1 -- 172.21.15.44:0/3656785833 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00ac50 con 0x7fb13c061560 2020-11-20T12:16:31.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.615+0000 7fb143d32700 1 -- 172.21.15.44:0/3656785833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c061560 msgr2=0x7fb13c13a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.615+0000 7fb143d32700 1 --2- 172.21.15.44:0/3656785833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c061560 0x7fb13c13a660 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb12c00cb30 tx=0x7fb12c006e80).stop 2020-11-20T12:16:31.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.615+0000 7fb143d32700 1 -- 172.21.15.44:0/3656785833 shutdown_connections 2020-11-20T12:16:31.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.615+0000 7fb143d32700 1 --2- 172.21.15.44:0/3656785833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c061560 0x7fb13c13a660 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.615+0000 7fb143d32700 1 -- 172.21.15.44:0/3656785833 >> 172.21.15.44:0/3656785833 conn(0x7fb13c0ae170 msgr2=0x7fb13c0aa170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:31.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.615+0000 7fb143d32700 1 -- 172.21.15.44:0/3656785833 shutdown_connections 2020-11-20T12:16:31.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.615+0000 7fb143d32700 1 -- 172.21.15.44:0/3656785833 wait complete. 2020-11-20T12:16:31.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 Processor -- start 2020-11-20T12:16:31.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 -- start start 2020-11-20T12:16:31.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb13c061180 0x7fb13c0a7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb13c061560 0x7fb13c09d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c0681e0 0x7fb13c0a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c09d960 con 0x7fb13c0681e0 2020-11-20T12:16:31.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0a2880 con 0x7fb13c061560 2020-11-20T12:16:31.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb143d32700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c149430 con 0x7fb13c061180 2020-11-20T12:16:31.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb1422cf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c0681e0 0x7fb13c0a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb1422cf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c0681e0 0x7fb13c0a3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41766/0 (socket says 172.21.15.44:41766) 2020-11-20T12:16:31.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb1422cf700 1 -- 172.21.15.44:0/2642563304 learned_addr learned my addr 172.21.15.44:0/2642563304 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:31.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb1412cd700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb13c061560 0x7fb13c09d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.617+0000 7fb141ace700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb13c061180 0x7fb13c0a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb1422cf700 1 -- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb13c061180 msgr2=0x7fb13c0a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb1422cf700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb13c061180 0x7fb13c0a7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb1422cf700 1 -- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb13c061560 msgr2=0x7fb13c09d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb1422cf700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb13c061560 0x7fb13c09d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb1422cf700 1 -- 172.21.15.44:0/2642563304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c09f1d0 con 0x7fb13c0681e0 2020-11-20T12:16:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb1422cf700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c0681e0 0x7fb13c0a3720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb13400cb10 tx=0x7fb134001320).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb129ffb700 1 -- 172.21.15.44:0/2642563304 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400cee0 con 0x7fb13c0681e0 2020-11-20T12:16:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb129ffb700 1 -- 172.21.15.44:0/2642563304 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134005b80 con 0x7fb13c0681e0 2020-11-20T12:16:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb129ffb700 1 -- 172.21.15.44:0/2642563304 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134007110 con 0x7fb13c0681e0 2020-11-20T12:16:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c09f470 con 0x7fb13c0681e0 2020-11-20T12:16:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.618+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c09ef70 con 0x7fb13c0681e0 2020-11-20T12:16:31.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.620+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c0aacd0 con 0x7fb13c0681e0 2020-11-20T12:16:31.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.620+0000 7fb129ffb700 1 -- 172.21.15.44:0/2642563304 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb134010030 con 0x7fb13c0681e0 2020-11-20T12:16:31.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.620+0000 7fb129ffb700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb114056d20 0x7fb1140591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.620+0000 7fb129ffb700 1 -- 172.21.15.44:0/2642563304 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb13406ff30 con 0x7fb13c0681e0 2020-11-20T12:16:31.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.620+0000 7fb141ace700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb114056d20 0x7fb1140591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.620+0000 7fb141ace700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb114056d20 0x7fb1140591e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fb1380035f0 tx=0x7fb138007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.624+0000 7fb129ffb700 1 -- 172.21.15.44:0/2642563304 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb134045200 con 0x7fb13c0681e0 2020-11-20T12:16:31.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.901+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb13c0a3400 con 0x7fb13c0681e0 2020-11-20T12:16:31.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.901+0000 7fb129ffb700 1 -- 172.21.15.44:0/2642563304 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7fb134048820 con 0x7fb13c0681e0 2020-11-20T12:16:31.905 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":130,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:16:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb114056d20 msgr2=0x7fb1140591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb114056d20 0x7fb1140591e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fb1380035f0 tx=0x7fb138007b90).stop 2020-11-20T12:16:31.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c0681e0 msgr2=0x7fb13c0a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c0681e0 0x7fb13c0a3720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb13400cb10 tx=0x7fb134001320).stop 2020-11-20T12:16:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 shutdown_connections 2020-11-20T12:16:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb114056d20 0x7fb1140591e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb13c0681e0 0x7fb13c0a3720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb13c061180 0x7fb13c0a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 --2- 172.21.15.44:0/2642563304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb13c061560 0x7fb13c09d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 >> 172.21.15.44:0/2642563304 conn(0x7fb13c0ae170 msgr2=0x7fb13c05ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 shutdown_connections 2020-11-20T12:16:31.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:31.903+0000 7fb143d32700 1 -- 172.21.15.44:0/2642563304 wait complete. 2020-11-20T12:16:31.918 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:16:31.918 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:16:32.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.050+0000 7feecb3f6700 1 Processor -- start 2020-11-20T12:16:32.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feecb3f6700 1 -- start start 2020-11-20T12:16:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feecb3f6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feec4060330 0x7feec40a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feecb3f6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 0x7feec413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feecb3f6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feec413ae80 0x7feec4136f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feecb3f6700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec40a52e0 con 0x7feec413ae80 2020-11-20T12:16:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feecb3f6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec40a6260 con 0x7feec4060330 2020-11-20T12:16:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feecb3f6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec40a6490 con 0x7feec40a4c30 2020-11-20T12:16:32.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.051+0000 7feec9993700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feec413ae80 0x7feec4136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feec413ae80 0x7feec4136f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41772/0 (socket says 172.21.15.44:41772) 2020-11-20T12:16:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 -- 172.21.15.44:0/346951192 learned_addr learned my addr 172.21.15.44:0/346951192 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9192700 1 --2- 172.21.15.44:0/346951192 >> v2:172.21.15.79:3300/0 conn(0x7feec4060330 0x7feec40a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec8991700 1 --2- 172.21.15.44:0/346951192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 0x7feec413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 -- 172.21.15.44:0/346951192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 msgr2=0x7feec413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:32.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 --2- 172.21.15.44:0/346951192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 0x7feec413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 -- 172.21.15.44:0/346951192 >> v2:172.21.15.79:3300/0 conn(0x7feec4060330 msgr2=0x7feec40a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:32.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 --2- 172.21.15.44:0/346951192 >> v2:172.21.15.79:3300/0 conn(0x7feec4060330 0x7feec40a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 -- 172.21.15.44:0/346951192 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40a6730 con 0x7feec413ae80 2020-11-20T12:16:32.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9192700 1 --2- 172.21.15.44:0/346951192 >> v2:172.21.15.79:3300/0 conn(0x7feec4060330 0x7feec40a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:32.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feec9993700 1 --2- 172.21.15.44:0/346951192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec413ae80 0x7feec4136f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7feec40adec0 tx=0x7feebc009290).ready entity=mon.0 client_cookie=d97ea66eadca58ac server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feeb3fff700 1 -- 172.21.15.44:0/346951192 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc00bbc0 con 0x7feec413ae80 2020-11-20T12:16:32.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feeb3fff700 1 -- 172.21.15.44:0/346951192 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc00bd20 con 0x7feec413ae80 2020-11-20T12:16:32.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.052+0000 7feeb3fff700 1 -- 172.21.15.44:0/346951192 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc011610 con 0x7feec413ae80 2020-11-20T12:16:32.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 -- 172.21.15.44:0/346951192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec413ae80 msgr2=0x7feec4136f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 --2- 172.21.15.44:0/346951192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec413ae80 0x7feec4136f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7feec40adec0 tx=0x7feebc009290).stop 2020-11-20T12:16:32.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 -- 172.21.15.44:0/346951192 shutdown_connections 2020-11-20T12:16:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 --2- 172.21.15.44:0/346951192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec413ae80 0x7feec4136f40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 --2- 172.21.15.44:0/346951192 >> v2:172.21.15.79:3300/0 conn(0x7feec4060330 0x7feec40a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 --2- 172.21.15.44:0/346951192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 0x7feec413a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 -- 172.21.15.44:0/346951192 >> 172.21.15.44:0/346951192 conn(0x7feec4062690 msgr2=0x7feec40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 -- 172.21.15.44:0/346951192 shutdown_connections 2020-11-20T12:16:32.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.053+0000 7feecb3f6700 1 -- 172.21.15.44:0/346951192 wait complete. 2020-11-20T12:16:32.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.054+0000 7feecb3f6700 1 Processor -- start 2020-11-20T12:16:32.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.054+0000 7feecb3f6700 1 -- start start 2020-11-20T12:16:32.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec4060330 0x7feec4135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 0x7feec40668b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec413ae80 0x7feec40abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec40aac60 con 0x7feec4060330 2020-11-20T12:16:32.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec40ac2e0 con 0x7feec413ae80 2020-11-20T12:16:32.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec40ac570 con 0x7feec40a4c30 2020-11-20T12:16:32.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec4060330 0x7feec4135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec4060330 0x7feec4135230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41776/0 (socket says 172.21.15.44:41776) 2020-11-20T12:16:32.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 -- 172.21.15.44:0/3090409851 learned_addr learned my addr 172.21.15.44:0/3090409851 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:32.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec8991700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 0x7feec40668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9993700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec413ae80 0x7feec40abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 msgr2=0x7feec40668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feec40a4c30 0x7feec40668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec413ae80 msgr2=0x7feec40abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec413ae80 0x7feec40abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 -- 172.21.15.44:0/3090409851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40ac810 con 0x7feec4060330 2020-11-20T12:16:32.072 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9993700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feec413ae80 0x7feec40abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feec9192700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec4060330 0x7feec4135230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7feec000db60 tx=0x7feec0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0009790 con 0x7feec4060330 2020-11-20T12:16:32.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec00098f0 con 0x7feec4060330 2020-11-20T12:16:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec40ab090 con 0x7feec4060330 2020-11-20T12:16:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0009c90 con 0x7feec4060330 2020-11-20T12:16:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.055+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec40ab5d0 con 0x7feec4060330 2020-11-20T12:16:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7feec0013480 con 0x7feec4060330 2020-11-20T12:16:32.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feecb3f6700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec414d620 0x7feec414da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec414d150 con 0x7feec414d620 2020-11-20T12:16:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feec8991700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec414d620 0x7feec414da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feec8991700 1 -- 172.21.15.44:0/3090409851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feec4066da0 con 0x7feec414d620 2020-11-20T12:16:32.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feeb17fa700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee9c056d50 0x7fee9c059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7feec0070560 con 0x7feec4060330 2020-11-20T12:16:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feec8991700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec414d620 0x7feec414da00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7feec40ade90 tx=0x7feeb4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feec9993700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee9c056d50 0x7fee9c059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.057+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb400e9f0 con 0x7feec414d620 2020-11-20T12:16:32.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.058+0000 7feec9993700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee9c056d50 0x7fee9c059210 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7feebc007fd0 tx=0x7feebc016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.058+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7feeb400eb50 con 0x7feec414d620 2020-11-20T12:16:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.058+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec414d620 msgr2=0x7feec414da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.058+0000 7feeb17fa700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec414d620 0x7feec414da00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7feec40ade90 tx=0x7feeb4008040).stop 2020-11-20T12:16:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.058+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec414d620 msgr2=0x7feec414da00 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.058+0000 7feeb17fa700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec414d620 0x7feec414da00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.067+0000 7feecb3f6700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec40a1880 0x7feec40a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.067+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec40ab8b0 con 0x7feec40a1880 2020-11-20T12:16:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.067+0000 7feec8991700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec40a1880 0x7feec40a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.067+0000 7feec8991700 1 -- 172.21.15.44:0/3090409851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feec4061650 con 0x7feec40a1880 2020-11-20T12:16:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.067+0000 7feec8991700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec40a1880 0x7feec40a1c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7feec40ade90 tx=0x7feeb4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.067+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4013730 con 0x7feec40a1880 2020-11-20T12:16:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.067+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4608 (secure 0 0 0) 0x7feeb400e7e0 con 0x7feec40a1880 2020-11-20T12:16:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec40a1880 msgr2=0x7feec40a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feeb17fa700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec40a1880 0x7feec40a1c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7feec40ade90 tx=0x7feeb4008040).stop 2020-11-20T12:16:32.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feeb17fa700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec40a1880 msgr2=0x7feec40a1c90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:32.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feeb17fa700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec40a1880 0x7feec40a1c90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee9c056d50 msgr2=0x7fee9c059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee9c056d50 0x7fee9c059210 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7feebc007fd0 tx=0x7feebc016000).stop 2020-11-20T12:16:32.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec4060330 msgr2=0x7feec4135230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec4060330 0x7feec4135230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7feec000db60 tx=0x7feec0008040).stop 2020-11-20T12:16:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feec9993700 1 -- 172.21.15.44:0/3090409851 reap_dead start 2020-11-20T12:16:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 shutdown_connections 2020-11-20T12:16:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 --2- 172.21.15.44:0/3090409851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feec4060330 0x7feec4135230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 >> 172.21.15.44:0/3090409851 conn(0x7feec4062690 msgr2=0x7feec405ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 shutdown_connections 2020-11-20T12:16:32.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.068+0000 7feecb3f6700 1 -- 172.21.15.44:0/3090409851 wait complete. 2020-11-20T12:16:32.085 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:32.085 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:32.085 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:32.085 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:16:32.086 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 130, 2020-11-20T12:16:32.086 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:32.086 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:32.086 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:16:32.087 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:32.087 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.087 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:16:32.087 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:32.087 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:32.088 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:32.088 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.088 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.088 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.088 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.089 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.089 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.089 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.089 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.089 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.090 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:32.090 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:32.090 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.090 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.091 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.091 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.091 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.091 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.091 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.092 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.092 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.092 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:32.092 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:32.093 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:32.093 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:32.093 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:32.093 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:32.093 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:32.094 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:32.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:32.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:32.094 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:32.095 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:32.095 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:32.095 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:32.096 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:32.096 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.096 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.096 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.096 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.097 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.097 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.097 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.097 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.098 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.098 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:32.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.098 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:32.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.099 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:32.099 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:32.099 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.099 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.100 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:32.100 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:32.101 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.101 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.101 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.102 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:32.102 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:32.102 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.102 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.103 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.103 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:32.103 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:32.104 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.104 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.104 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.104 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:32.105 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.105 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:32.106 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.106 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.106 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.106 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:32.107 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:32.107 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.107 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.107 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.108 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:32.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:32.108 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.108 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.109 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:32.109 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.110 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:32.110 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.111 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.111 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:32.111 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:32.111 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.112 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.112 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.112 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.113 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:32.113 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:32.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.113 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.113 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.114 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:32.114 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.114 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.114 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:16:32.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.115 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.115 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.115 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:32.115 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.115 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.116 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:16:32.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.116 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.116 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.116 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:32.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.117 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:32.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.118 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.118 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.118 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:16:32.118 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.119 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.119 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:32.119 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:32.120 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:16:32.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.255+0000 7f1a31a9a700 1 Processor -- start 2020-11-20T12:16:32.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.255+0000 7f1a31a9a700 1 -- start start 2020-11-20T12:16:32.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a31a9a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0aad00 0x7f1a2c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a31a9a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1a2c0a86d0 0x7f1a2c13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a31a9a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2c1385f0 con 0x7f1a2c0ab0e0 2020-11-20T12:16:32.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a31a9a700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c13e250 con 0x7f1a2c0a86d0 2020-11-20T12:16:32.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a31a9a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c13e4f0 con 0x7f1a2c0aad00 2020-11-20T12:16:32.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1a2c0a86d0 0x7f1a2c13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1a2c0a86d0 0x7f1a2c13ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41792/0 (socket says 172.21.15.44:41792) 2020-11-20T12:16:32.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 -- 172.21.15.44:0/2878016727 learned_addr learned my addr 172.21.15.44:0/2878016727 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:32.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2b7fe700 1 --2- 172.21.15.44:0/2878016727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0aad00 0x7f1a2c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 -- 172.21.15.44:0/2878016727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0aad00 msgr2=0x7f1a2c0a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:32.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 --2- 172.21.15.44:0/2878016727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0aad00 0x7f1a2c0a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 -- 172.21.15.44:0/2878016727 >> v1:172.21.15.79:6789/0 conn(0x7f1a2c0ab0e0 legacy=0x7f1a2c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 -- 172.21.15.44:0/2878016727 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0ac340 con 0x7f1a2c0a86d0 2020-11-20T12:16:32.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2bfff700 1 --2- 172.21.15.44:0/2878016727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0a86d0 0x7f1a2c13ec30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1a24009d50 tx=0x7f1a24009290).ready entity=mon.0 client_cookie=5bdb0bf5a471bbee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2a7fc700 1 -- 172.21.15.44:0/2878016727 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400bd10 con 0x7f1a2c0a86d0 2020-11-20T12:16:32.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2a7fc700 1 -- 172.21.15.44:0/2878016727 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2400be70 con 0x7f1a2c0a86d0 2020-11-20T12:16:32.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.256+0000 7f1a2a7fc700 1 -- 172.21.15.44:0/2878016727 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24011610 con 0x7f1a2c0a86d0 2020-11-20T12:16:32.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2878016727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0a86d0 msgr2=0x7f1a2c13ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2878016727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0a86d0 0x7f1a2c13ec30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1a24009d50 tx=0x7f1a24009290).stop 2020-11-20T12:16:32.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2878016727 shutdown_connections 2020-11-20T12:16:32.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2878016727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0a86d0 0x7f1a2c13ec30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2878016727 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0aad00 0x7f1a2c0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2878016727 >> 172.21.15.44:0/2878016727 conn(0x7f1a2c060170 msgr2=0x7f1a2c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2878016727 shutdown_connections 2020-11-20T12:16:32.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.257+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2878016727 wait complete. 2020-11-20T12:16:32.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.258+0000 7f1a31a9a700 1 Processor -- start 2020-11-20T12:16:32.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.258+0000 7f1a31a9a700 1 -- start start 2020-11-20T12:16:32.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a31a9a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0a86d0 0x7f1a2c09e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a31a9a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0aad00 0x7f1a2c0a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a31a9a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0ab0e0 0x7f1a2c14d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a31a9a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0a5210 con 0x7f1a2c0aad00 2020-11-20T12:16:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a31a9a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0a5d30 con 0x7f1a2c0ab0e0 2020-11-20T12:16:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a31a9a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0a5fd0 con 0x7f1a2c0a86d0 2020-11-20T12:16:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0aad00 0x7f1a2c0a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0aad00 0x7f1a2c0a7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41796/0 (socket says 172.21.15.44:41796) 2020-11-20T12:16:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 -- 172.21.15.44:0/2572352192 learned_addr learned my addr 172.21.15.44:0/2572352192 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0a86d0 0x7f1a2c09e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2bfff700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0ab0e0 0x7f1a2c14d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0a86d0 msgr2=0x7f1a2c09e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1a2c0a86d0 0x7f1a2c09e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0ab0e0 msgr2=0x7f1a2c14d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0ab0e0 0x7f1a2c14d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.259+0000 7f1a2affd700 1 -- 172.21.15.44:0/2572352192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0a6270 con 0x7f1a2c0aad00 2020-11-20T12:16:32.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.260+0000 7f1a2affd700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0aad00 0x7f1a2c0a7330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1a2000c710 tx=0x7f1a20007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.260+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20013410 con 0x7f1a2c0aad00 2020-11-20T12:16:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.260+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20013570 con 0x7f1a2c0aad00 2020-11-20T12:16:32.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.260+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c0a6510 con 0x7f1a2c0aad00 2020-11-20T12:16:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.260+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20013910 con 0x7f1a2c0aad00 2020-11-20T12:16:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.260+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c14d9d0 con 0x7f1a2c0aad00 2020-11-20T12:16:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.261+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0a4520 0x7f1a2c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.261+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0a68d0 con 0x7f1a2c0a4520 2020-11-20T12:16:32.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.261+0000 7f1a2b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0a4520 0x7f1a2c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.261+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1a20010030 con 0x7f1a2c0aad00 2020-11-20T12:16:32.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.261+0000 7f1a2b7fe700 1 -- 172.21.15.44:0/2572352192 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a2c0a6a60 con 0x7f1a2c0a4520 2020-11-20T12:16:32.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.261+0000 7f1a0b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a04056d50 0x7f1a04059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1a20001e40 con 0x7f1a2c0aad00 2020-11-20T12:16:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a2b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0a4520 0x7f1a2c149f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1a1c010a80 tx=0x7f1a1c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a2bfff700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a04056d50 0x7f1a04059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00eac0 con 0x7f1a2c0a4520 2020-11-20T12:16:32.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a2bfff700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a04056d50 0x7f1a04059210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f1a2400be40 tx=0x7f1a24016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f1a1c00ec20 con 0x7f1a2c0a4520 2020-11-20T12:16:32.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0a4520 msgr2=0x7f1a2c149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a0b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0a4520 0x7f1a2c149f50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1a1c010a80 tx=0x7f1a1c008040).stop 2020-11-20T12:16:32.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0a4520 msgr2=0x7f1a2c149f50 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:32.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.262+0000 7f1a0b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c0a4520 0x7f1a2c149f50 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0x7f1a1c010a80 tx=0x7f1a1c008040).stop 2020-11-20T12:16:32.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.270+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c14a440 0x7f1a2c14a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.270+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c05c5f0 con 0x7f1a2c14a440 2020-11-20T12:16:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a2b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c14a440 0x7f1a2c14a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a2b7fe700 1 -- 172.21.15.44:0/2572352192 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1a2c14dc80 con 0x7f1a2c14a440 2020-11-20T12:16:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a2b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c14a440 0x7f1a2c14a850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1a1c018000 tx=0x7f1a1c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c013830 con 0x7f1a2c14a440 2020-11-20T12:16:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f1a1c00e8b0 con 0x7f1a2c14a440 2020-11-20T12:16:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c14a440 msgr2=0x7f1a2c14a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a0b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c14a440 0x7f1a2c14a850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1a1c018000 tx=0x7f1a1c008040).stop 2020-11-20T12:16:32.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a0b7fe700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c14a440 msgr2=0x7f1a2c14a850 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.271+0000 7f1a0b7fe700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a2c14a440 0x7f1a2c14a850 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a04056d50 msgr2=0x7f1a04059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1a04056d50 0x7f1a04059210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f1a2400be40 tx=0x7f1a24016000).stop 2020-11-20T12:16:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0aad00 msgr2=0x7f1a2c0a7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0aad00 0x7f1a2c0a7330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1a2000c710 tx=0x7f1a20007ba0).stop 2020-11-20T12:16:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a2bfff700 1 -- 172.21.15.44:0/2572352192 reap_dead start 2020-11-20T12:16:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 shutdown_connections 2020-11-20T12:16:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 --2- 172.21.15.44:0/2572352192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a2c0aad00 0x7f1a2c0a7330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 >> 172.21.15.44:0/2572352192 conn(0x7f1a2c060170 msgr2=0x7f1a2c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 shutdown_connections 2020-11-20T12:16:32.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.272+0000 7f1a31a9a700 1 -- 172.21.15.44:0/2572352192 wait complete. 2020-11-20T12:16:32.285 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:16:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 130, 2020-11-20T12:16:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:16:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:16:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:32.305 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.305 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.305 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:32.306 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:32.306 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.306 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.306 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.307 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:32.307 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:32.307 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.307 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.308 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:32.308 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.309 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:32.309 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.309 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.310 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.310 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:32.310 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:32.310 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.311 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.311 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.311 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.312 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:32.312 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:32.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.312 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.312 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.313 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:32.313 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.313 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.313 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:32.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.314 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.314 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.314 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:32.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.314 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.315 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:32.315 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:32.316 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:16:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.451+0000 7fc214c19700 1 Processor -- start 2020-11-20T12:16:32.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.451+0000 7fc214c19700 1 -- start start 2020-11-20T12:16:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.451+0000 7fc214c19700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.451+0000 7fc214c19700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2100a6260 con 0x7fc210060330 2020-11-20T12:16:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.451+0000 7fc214c19700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2100a6490 con 0x7fc21013f800 2020-11-20T12:16:32.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.451+0000 7fc214c19700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100a52e0 con 0x7fc21013a990 2020-11-20T12:16:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc20e59c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc210060330 0x7fc2100a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45554/0 (socket says 172.21.15.44:45554) 2020-11-20T12:16:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc20e59c700 1 -- 172.21.15.44:0/4078986948 learned_addr learned my addr 172.21.15.44:0/4078986948 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:16:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc207fff700 1 --2- 172.21.15.44:0/4078986948 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109374066 0 0) 0x7fc2100a6260 con 0x7fc210060330 2020-11-20T12:16:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f0003240 con 0x7fc210060330 2020-11-20T12:16:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109396537 0 0) 0x7fc2100a6490 con 0x7fc21013f800 2020-11-20T12:16:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2100a6260 con 0x7fc21013f800 2020-11-20T12:16:32.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2107658665 0 0) 0x7fc1f0003240 con 0x7fc210060330 2020-11-20T12:16:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc207fff700 1 -- 172.21.15.44:0/4078986948 >> v1:172.21.15.79:6789/0 conn(0x7fc21013f800 legacy=0x7fc210141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc207fff700 1 -- 172.21.15.44:0/4078986948 >> v1:172.21.15.44:6789/0 conn(0x7fc210060330 legacy=0x7fc2100a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc207fff700 1 -- 172.21.15.44:0/4078986948 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100a6730 con 0x7fc21013a990 2020-11-20T12:16:32.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.452+0000 7fc207fff700 1 --2- 172.21.15.44:0/4078986948 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013adf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fc1f800b9d0 tx=0x7fc1f8006df0).ready entity=mon.2 client_cookie=f7f7080d168acb3e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000014e0 con 0x7fc21013a990 2020-11-20T12:16:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f8011400 con 0x7fc21013a990 2020-11-20T12:16:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc20dd9b700 1 -- 172.21.15.44:0/4078986948 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8011780 con 0x7fc21013a990 2020-11-20T12:16:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc214c19700 1 -- 172.21.15.44:0/4078986948 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 msgr2=0x7fc21013adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc214c19700 1 --2- 172.21.15.44:0/4078986948 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013adf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fc1f800b9d0 tx=0x7fc1f8006df0).stop 2020-11-20T12:16:32.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc214c19700 1 -- 172.21.15.44:0/4078986948 shutdown_connections 2020-11-20T12:16:32.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc214c19700 1 --2- 172.21.15.44:0/4078986948 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013adf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc214c19700 1 -- 172.21.15.44:0/4078986948 >> 172.21.15.44:0/4078986948 conn(0x7fc210062690 msgr2=0x7fc2100a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc214c19700 1 -- 172.21.15.44:0/4078986948 shutdown_connections 2020-11-20T12:16:32.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.453+0000 7fc214c19700 1 -- 172.21.15.44:0/4078986948 wait complete. 2020-11-20T12:16:32.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.454+0000 7fc214c19700 1 Processor -- start 2020-11-20T12:16:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.454+0000 7fc214c19700 1 -- start start 2020-11-20T12:16:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc214c19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc210060330 0x7fc210131550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc214c19700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc214c19700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21013f800 0x7fc21013b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc214c19700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21013b830 con 0x7fc21013f800 2020-11-20T12:16:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc214c19700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21013e390 con 0x7fc210060330 2020-11-20T12:16:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc214c19700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21013e5c0 con 0x7fc21013a990 2020-11-20T12:16:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21013f800 0x7fc21013b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21013f800 0x7fc21013b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41816/0 (socket says 172.21.15.44:41816) 2020-11-20T12:16:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 -- 172.21.15.44:0/1682849776 learned_addr learned my addr 172.21.15.44:0/1682849776 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc207fff700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20e59c700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc210060330 0x7fc210131550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 msgr2=0x7fc21013ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.468 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc210060330 msgr2=0x7fc210131550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc210060330 0x7fc210131550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20ed9d700 1 -- 172.21.15.44:0/1682849776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21013d820 con 0x7fc21013f800 2020-11-20T12:16:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc207fff700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013a990 0x7fc21013ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:32.469 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.455+0000 7fc20e59c700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc210060330 0x7fc210131550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:32.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.456+0000 7fc20ed9d700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21013f800 0x7fc21013b340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fc20800cac0 tx=0x7fc208007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.456+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800ce90 con 0x7fc21013f800 2020-11-20T12:16:32.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.456+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208002240 con 0x7fc21013f800 2020-11-20T12:16:32.470 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.456+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21013dac0 con 0x7fc21013f800 2020-11-20T12:16:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.456+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800a5b0 con 0x7fc21013f800 2020-11-20T12:16:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.456+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21013c6b0 con 0x7fc21013f800 2020-11-20T12:16:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc214c19700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013e980 0x7fc21014b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.471 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21013dc50 con 0x7fc21013e980 2020-11-20T12:16:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc20e59c700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013e980 0x7fc21014b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc208019030 con 0x7fc21013f800 2020-11-20T12:16:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc20e59c700 1 -- 172.21.15.44:0/1682849776 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc210061650 con 0x7fc21013e980 2020-11-20T12:16:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc20e59c700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013e980 0x7fc21014b350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc200004de0 tx=0x7fc200000b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.472 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc2067fc700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc1ec056d20 0x7fc1ec0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.457+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc20806f790 con 0x7fc21013f800 2020-11-20T12:16:32.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20000fe30 con 0x7fc21013e980 2020-11-20T12:16:32.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc207fff700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc1ec056d20 0x7fc1ec0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fc200012350 con 0x7fc21013e980 2020-11-20T12:16:32.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013e980 msgr2=0x7fc21014b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc2067fc700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013e980 0x7fc21014b350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc200004de0 tx=0x7fc200000b80).stop 2020-11-20T12:16:32.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013e980 msgr2=0x7fc21014b350 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:32.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc2067fc700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21013e980 0x7fc21014b350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.458+0000 7fc207fff700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc1ec056d20 0x7fc1ec0591e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc1f8006e30 tx=0x7fc1f8015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.467+0000 7fc214c19700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21012f030 0x7fc21012f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.467+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100446d0 con 0x7fc21012f030 2020-11-20T12:16:32.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.467+0000 7fc20e59c700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21012f030 0x7fc21012f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc20e59c700 1 -- 172.21.15.44:0/1682849776 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc2100680f0 con 0x7fc21012f030 2020-11-20T12:16:32.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc20e59c700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21012f030 0x7fc21012f410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fc200017040 tx=0x7fc200009000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200014a60 con 0x7fc21012f030 2020-11-20T12:16:32.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7fc20000fbc0 con 0x7fc21012f030 2020-11-20T12:16:32.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21012f030 msgr2=0x7fc21012f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc2067fc700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21012f030 0x7fc21012f410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fc200017040 tx=0x7fc200009000).stop 2020-11-20T12:16:32.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc2067fc700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21012f030 msgr2=0x7fc21012f410 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:32.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.468+0000 7fc2067fc700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc21012f030 0x7fc21012f410 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc1ec056d20 msgr2=0x7fc1ec0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc1ec056d20 0x7fc1ec0591e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fc1f8006e30 tx=0x7fc1f8015000).stop 2020-11-20T12:16:32.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21013f800 msgr2=0x7fc21013b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 --2- 172.21.15.44:0/1682849776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc21013f800 0x7fc21013b340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fc20800cac0 tx=0x7fc208007da0).stop 2020-11-20T12:16:32.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc20ed9d700 1 -- 172.21.15.44:0/1682849776 reap_dead start 2020-11-20T12:16:32.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc20ed9d700 1 -- 172.21.15.44:0/1682849776 reap_dead start 2020-11-20T12:16:32.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 shutdown_connections 2020-11-20T12:16:32.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 >> 172.21.15.44:0/1682849776 conn(0x7fc210062690 msgr2=0x7fc2100a8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 shutdown_connections 2020-11-20T12:16:32.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.469+0000 7fc214c19700 1 -- 172.21.15.44:0/1682849776 wait complete. 2020-11-20T12:16:32.481 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:32.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:32.481 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:32.482 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:16:32.482 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 130, 2020-11-20T12:16:32.482 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:32.482 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:32.482 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:16:32.483 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:32.483 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.483 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:16:32.483 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:32.483 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:32.484 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:32.484 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.484 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.485 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.485 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.485 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.485 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.486 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.486 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.486 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.486 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:32.487 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:32.487 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.487 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.487 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.487 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.488 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.488 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.488 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.488 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.488 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.489 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:32.489 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:32.489 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:32.489 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:32.490 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:32.490 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:32.490 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:32.490 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:32.490 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:32.491 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:32.491 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:32.491 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:32.491 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:32.491 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:32.492 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:32.492 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:32.492 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:32.492 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:32.492 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:32.493 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:32.493 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:32.493 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:32.493 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:32.493 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.494 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:32.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.494 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:32.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.494 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:32.495 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:32.495 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.495 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.495 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.496 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:32.496 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.497 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:32.497 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.497 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.498 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:32.498 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:32.498 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.498 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.498 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.499 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:32.499 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:32.499 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.500 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.500 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.500 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:32.500 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.501 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:32.501 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.502 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.502 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:32.502 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:32.503 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.503 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.503 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.504 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:32.504 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:32.504 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:32.504 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:32.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:32.505 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:32.505 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.506 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:32.506 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:32.506 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:32.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.506 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.507 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:32.507 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:32.507 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:32.508 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:32.508 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:32.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.508 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.509 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:32.509 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:32.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.509 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.509 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.510 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:32.510 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.510 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.510 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:16:32.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.511 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.511 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.511 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:32.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.512 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.512 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:16:32.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.512 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.512 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.513 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:32.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.513 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.513 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:32.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.514 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.514 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.514 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:32.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.514 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:32.515 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:16:32.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:32.515 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:32.515 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:32.515 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:16:32.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:32.516 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:32.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:32.516 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:32.516 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:32.517 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:16:32.518 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:16:32.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.651+0000 7f2db85e6700 1 Processor -- start 2020-11-20T12:16:32.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.651+0000 7f2db85e6700 1 -- start start 2020-11-20T12:16:32.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db85e6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2db00aa020 0x7f2db00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db85e6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2db00aa400 0x7f2db00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db85e6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00a9770 con 0x7f2db013ac60 2020-11-20T12:16:32.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db85e6700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db00a6da0 con 0x7f2db00aa400 2020-11-20T12:16:32.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db85e6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db00a7040 con 0x7f2db00aa020 2020-11-20T12:16:32.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2db00aa400 0x7f2db00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2db00aa400 0x7f2db00a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41826/0 (socket says 172.21.15.44:41826) 2020-11-20T12:16:32.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 -- 172.21.15.44:0/1362418550 learned_addr learned my addr 172.21.15.44:0/1362418550 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:32.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db6382700 1 --2- 172.21.15.44:0/1362418550 >> v2:172.21.15.79:3300/0 conn(0x7f2db00aa020 0x7f2db00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 -- 172.21.15.44:0/1362418550 >> v2:172.21.15.79:3300/0 conn(0x7f2db00aa020 msgr2=0x7f2db00a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:32.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 --2- 172.21.15.44:0/1362418550 >> v2:172.21.15.79:3300/0 conn(0x7f2db00aa020 0x7f2db00a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 -- 172.21.15.44:0/1362418550 >> v1:172.21.15.79:6789/0 conn(0x7f2db013ac60 legacy=0x7f2db00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 -- 172.21.15.44:0/1362418550 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00a6a90 con 0x7f2db00aa400 2020-11-20T12:16:32.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.652+0000 7f2db5b81700 1 --2- 172.21.15.44:0/1362418550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa400 0x7f2db00a4fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2db00adec0 tx=0x7f2da0006eb0).ready entity=mon.0 client_cookie=4d3c3026a00af473 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db5380700 1 -- 172.21.15.44:0/1362418550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0009810 con 0x7f2db00aa400 2020-11-20T12:16:32.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db5380700 1 -- 172.21.15.44:0/1362418550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da0009970 con 0x7f2db00aa400 2020-11-20T12:16:32.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db5380700 1 -- 172.21.15.44:0/1362418550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0009cf0 con 0x7f2db00aa400 2020-11-20T12:16:32.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 -- 172.21.15.44:0/1362418550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa400 msgr2=0x7f2db00a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1362418550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa400 0x7f2db00a4fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2db00adec0 tx=0x7f2da0006eb0).stop 2020-11-20T12:16:32.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 -- 172.21.15.44:0/1362418550 shutdown_connections 2020-11-20T12:16:32.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1362418550 >> v2:172.21.15.79:3300/0 conn(0x7f2db00aa020 0x7f2db00a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1362418550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa400 0x7f2db00a4fb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 -- 172.21.15.44:0/1362418550 >> 172.21.15.44:0/1362418550 conn(0x7f2db0062690 msgr2=0x7f2db00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 -- 172.21.15.44:0/1362418550 shutdown_connections 2020-11-20T12:16:32.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.653+0000 7f2db85e6700 1 -- 172.21.15.44:0/1362418550 wait complete. 2020-11-20T12:16:32.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.654+0000 7f2db85e6700 1 Processor -- start 2020-11-20T12:16:32.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.654+0000 7f2db85e6700 1 -- start start 2020-11-20T12:16:32.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db85e6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa020 0x7f2db0067140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db85e6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2db00aa400 0x7f2db00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db85e6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2db013ac60 0x7f2db014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db85e6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db014a0e0 con 0x7f2db00aa020 2020-11-20T12:16:32.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db85e6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00ad890 con 0x7f2db00aa400 2020-11-20T12:16:32.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db85e6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0146b10 con 0x7f2db013ac60 2020-11-20T12:16:32.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db6382700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa020 0x7f2db0067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db6382700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa020 0x7f2db0067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41830/0 (socket says 172.21.15.44:41830) 2020-11-20T12:16:32.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db6382700 1 -- 172.21.15.44:0/1940483521 learned_addr learned my addr 172.21.15.44:0/1940483521 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:32.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db5b81700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2db00aa400 0x7f2db00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.655+0000 7f2db6b83700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2db013ac60 0x7f2db014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db6382700 1 -- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2db013ac60 msgr2=0x7f2db014a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db6382700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2db013ac60 0x7f2db014a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db6382700 1 -- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2db00aa400 msgr2=0x7f2db00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db6382700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2db00aa400 0x7f2db00ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db6382700 1 -- 172.21.15.44:0/1940483521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00adb20 con 0x7f2db00aa020 2020-11-20T12:16:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db6382700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa020 0x7f2db0067140 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2dac00eba0 tx=0x7f2dac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2d9e7fc700 1 -- 172.21.15.44:0/1940483521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00c8e0 con 0x7f2db00aa020 2020-11-20T12:16:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2d9e7fc700 1 -- 172.21.15.44:0/1940483521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac00ca40 con 0x7f2db00aa020 2020-11-20T12:16:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db014a310 con 0x7f2db00aa020 2020-11-20T12:16:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2d9e7fc700 1 -- 172.21.15.44:0/1940483521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00cdc0 con 0x7f2db00aa020 2020-11-20T12:16:32.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.656+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db009ee20 con 0x7f2db00aa020 2020-11-20T12:16:32.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.657+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db00adcb0 con 0x7f2db00aa020 2020-11-20T12:16:32.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.658+0000 7f2d9e7fc700 1 -- 172.21.15.44:0/1940483521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2dac012600 con 0x7f2db00aa020 2020-11-20T12:16:32.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.658+0000 7f2d9e7fc700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2d88056d20 0x7f2d880591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.658+0000 7f2d9e7fc700 1 -- 172.21.15.44:0/1940483521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2dac0704b0 con 0x7f2db00aa020 2020-11-20T12:16:32.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.658+0000 7f2db5b81700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2d88056d20 0x7f2d880591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.659+0000 7f2db5b81700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2d88056d20 0x7f2d880591e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f2da0006ef0 tx=0x7f2da0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.662+0000 7f2d9e7fc700 1 -- 172.21.15.44:0/1940483521 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2dac045780 con 0x7f2db00aa020 2020-11-20T12:16:32.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.930+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f2db0061650 con 0x7f2db00aa020 2020-11-20T12:16:32.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.931+0000 7f2d9e7fc700 1 -- 172.21.15.44:0/1940483521 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f2dac048da0 con 0x7f2db00aa020 2020-11-20T12:16:32.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.933+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2d88056d20 msgr2=0x7f2d880591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.933+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2d88056d20 0x7f2d880591e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f2da0006ef0 tx=0x7f2da0015000).stop 2020-11-20T12:16:32.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.933+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa020 msgr2=0x7f2db0067140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa020 0x7f2db0067140 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2dac00eba0 tx=0x7f2dac006df0).stop 2020-11-20T12:16:32.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 shutdown_connections 2020-11-20T12:16:32.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2d88056d20 0x7f2d880591e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2db013ac60 0x7f2db014a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2db00aa020 0x7f2db0067140 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 --2- 172.21.15.44:0/1940483521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2db00aa400 0x7f2db00ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 >> 172.21.15.44:0/1940483521 conn(0x7f2db0062690 msgr2=0x7f2db0134a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 shutdown_connections 2020-11-20T12:16:32.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:32.934+0000 7f2db85e6700 1 -- 172.21.15.44:0/1940483521 wait complete. 2020-11-20T12:16:32.951 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:16:38.241 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:38.244 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:42.952 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:16:42.953 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:16:43.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.089+0000 7f7ed6e46700 1 Processor -- start 2020-11-20T12:16:43.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.089+0000 7f7ed6e46700 1 -- start start 2020-11-20T12:16:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ed6e46700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7ed00aa400 0x7f7ed00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ed6e46700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7ed013ac60 0x7f7ed00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ed6e46700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed00a9770 con 0x7f7ed00aa020 2020-11-20T12:16:43.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ed6e46700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00a56c0 con 0x7f7ed00aa400 2020-11-20T12:16:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ed6e46700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00a6da0 con 0x7f7ed013ac60 2020-11-20T12:16:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7ed00aa400 0x7f7ed00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7ed00aa400 0x7f7ed00a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41840/0 (socket says 172.21.15.44:41840) 2020-11-20T12:16:43.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 -- 172.21.15.44:0/3690670880 learned_addr learned my addr 172.21.15.44:0/3690670880 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ed53e3700 1 --2- 172.21.15.44:0/3690670880 >> v2:172.21.15.79:3300/0 conn(0x7f7ed013ac60 0x7f7ed00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 -- 172.21.15.44:0/3690670880 >> v2:172.21.15.79:3300/0 conn(0x7f7ed013ac60 msgr2=0x7f7ed00a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 --2- 172.21.15.44:0/3690670880 >> v2:172.21.15.79:3300/0 conn(0x7f7ed013ac60 0x7f7ed00a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 -- 172.21.15.44:0/3690670880 >> v1:172.21.15.79:6789/0 conn(0x7f7ed00aa020 legacy=0x7f7ed00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:43.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 -- 172.21.15.44:0/3690670880 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed00a7040 con 0x7f7ed00aa400 2020-11-20T12:16:43.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ed53e3700 1 --2- 172.21.15.44:0/3690670880 >> v2:172.21.15.79:3300/0 conn(0x7f7ed013ac60 0x7f7ed00a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:43.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecffff700 1 --2- 172.21.15.44:0/3690670880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a9280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f7ed00a8f70 tx=0x7f7ebc006eb0).ready entity=mon.0 client_cookie=c416df1862caa88 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecf7fe700 1 -- 172.21.15.44:0/3690670880 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc009890 con 0x7f7ed00aa400 2020-11-20T12:16:43.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecf7fe700 1 -- 172.21.15.44:0/3690670880 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc0099f0 con 0x7f7ed00aa400 2020-11-20T12:16:43.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.090+0000 7f7ecf7fe700 1 -- 172.21.15.44:0/3690670880 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc009d70 con 0x7f7ed00aa400 2020-11-20T12:16:43.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 -- 172.21.15.44:0/3690670880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 msgr2=0x7f7ed00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/3690670880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a9280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f7ed00a8f70 tx=0x7f7ebc006eb0).stop 2020-11-20T12:16:43.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 -- 172.21.15.44:0/3690670880 shutdown_connections 2020-11-20T12:16:43.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/3690670880 >> v2:172.21.15.79:3300/0 conn(0x7f7ed013ac60 0x7f7ed00a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/3690670880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a9280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 -- 172.21.15.44:0/3690670880 >> 172.21.15.44:0/3690670880 conn(0x7f7ed0062690 msgr2=0x7f7ed00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 -- 172.21.15.44:0/3690670880 shutdown_connections 2020-11-20T12:16:43.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.091+0000 7f7ed6e46700 1 -- 172.21.15.44:0/3690670880 wait complete. 2020-11-20T12:16:43.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.092+0000 7f7ed6e46700 1 Processor -- start 2020-11-20T12:16:43.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.092+0000 7f7ed6e46700 1 -- start start 2020-11-20T12:16:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed6e46700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed00aa020 0x7f7ed0060420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed6e46700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed6e46700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ed013ac60 0x7f7ed0149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed6e46700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00a38a0 con 0x7f7ed00aa400 2020-11-20T12:16:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed6e46700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed014d700 con 0x7f7ed00aa020 2020-11-20T12:16:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed6e46700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed014d9a0 con 0x7f7ed013ac60 2020-11-20T12:16:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41846/0 (socket says 172.21.15.44:41846) 2020-11-20T12:16:43.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ecffff700 1 -- 172.21.15.44:0/758886875 learned_addr learned my addr 172.21.15.44:0/758886875 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:43.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed4be2700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed00aa020 0x7f7ed0060420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.093+0000 7f7ed53e3700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ed013ac60 0x7f7ed0149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ecffff700 1 -- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ed013ac60 msgr2=0x7f7ed0149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ecffff700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ed013ac60 0x7f7ed0149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ecffff700 1 -- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed00aa020 msgr2=0x7f7ed0060420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ecffff700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed00aa020 0x7f7ed0060420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ecffff700 1 -- 172.21.15.44:0/758886875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed014dc40 con 0x7f7ed00aa400 2020-11-20T12:16:43.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ecffff700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a33b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7ebc006ef0 tx=0x7f7ebc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7eccff9700 1 -- 172.21.15.44:0/758886875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc011e50 con 0x7f7ed00aa400 2020-11-20T12:16:43.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7eccff9700 1 -- 172.21.15.44:0/758886875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc01e460 con 0x7f7ed00aa400 2020-11-20T12:16:43.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7eccff9700 1 -- 172.21.15.44:0/758886875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc01e800 con 0x7f7ed00aa400 2020-11-20T12:16:43.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed014d3f0 con 0x7f7ed00aa400 2020-11-20T12:16:43.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.094+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed014a310 con 0x7f7ed00aa400 2020-11-20T12:16:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.095+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed014ddd0 con 0x7f7ed00aa400 2020-11-20T12:16:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.095+0000 7f7eccff9700 1 -- 172.21.15.44:0/758886875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7ebc025030 con 0x7f7ed00aa400 2020-11-20T12:16:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.095+0000 7f7eccff9700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ea8056d20 0x7f7ea80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.096+0000 7f7eccff9700 1 -- 172.21.15.44:0/758886875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f7ebc071650 con 0x7f7ed00aa400 2020-11-20T12:16:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.096+0000 7f7ed4be2700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ea8056d20 0x7f7ea80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.096+0000 7f7ed4be2700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ea8056d20 0x7f7ea80591e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f7ec4006fd0 tx=0x7f7ec4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.099+0000 7f7eccff9700 1 -- 172.21.15.44:0/758886875 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7ebc0469d0 con 0x7f7ed00aa400 2020-11-20T12:16:43.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.370+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7ed00a9770 con 0x7f7ed00aa400 2020-11-20T12:16:43.374 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":130,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:16:43.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.371+0000 7f7eccff9700 1 -- 172.21.15.44:0/758886875 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f7ed00a9770 con 0x7f7ed00aa400 2020-11-20T12:16:43.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ea8056d20 msgr2=0x7f7ea80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ea8056d20 0x7f7ea80591e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f7ec4006fd0 tx=0x7f7ec4008040).stop 2020-11-20T12:16:43.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 msgr2=0x7f7ed00a33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a33b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7ebc006ef0 tx=0x7f7ebc015000).stop 2020-11-20T12:16:43.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 shutdown_connections 2020-11-20T12:16:43.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7ea8056d20 0x7f7ea80591e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7ed013ac60 0x7f7ed0149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed00aa020 0x7f7ed0060420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 --2- 172.21.15.44:0/758886875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ed00aa400 0x7f7ed00a33b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.372+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 >> 172.21.15.44:0/758886875 conn(0x7f7ed0062690 msgr2=0x7f7ed0136c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.373+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 shutdown_connections 2020-11-20T12:16:43.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.373+0000 7f7ed6e46700 1 -- 172.21.15.44:0/758886875 wait complete. 2020-11-20T12:16:43.390 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:16:43.390 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:16:43.390 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:16:43.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.524+0000 7f23c8d45700 1 Processor -- start 2020-11-20T12:16:43.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.524+0000 7f23c8d45700 1 -- start start 2020-11-20T12:16:43.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c8d45700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f23c40a84b0 0x7f23c40a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c8d45700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40a3250 0x7f23c4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c8d45700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c40a93c0 con 0x7f23c4068f60 2020-11-20T12:16:43.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c8d45700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c40a95f0 con 0x7f23c40a84b0 2020-11-20T12:16:43.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c8d45700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40a9ea0 con 0x7f23c40a3250 2020-11-20T12:16:43.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f23c40a84b0 0x7f23c40a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f23c40a84b0 0x7f23c40a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41852/0 (socket says 172.21.15.44:41852) 2020-11-20T12:16:43.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 -- 172.21.15.44:0/3811602007 learned_addr learned my addr 172.21.15.44:0/3811602007 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:43.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c2d9d700 1 --2- 172.21.15.44:0/3811602007 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40a3250 0x7f23c4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 -- 172.21.15.44:0/3811602007 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40a3250 msgr2=0x7f23c4137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:43.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 --2- 172.21.15.44:0/3811602007 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40a3250 0x7f23c4137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 -- 172.21.15.44:0/3811602007 >> v1:172.21.15.79:6789/0 conn(0x7f23c4068f60 legacy=0x7f23c40a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 -- 172.21.15.44:0/3811602007 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4069340 con 0x7f23c40a84b0 2020-11-20T12:16:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.525+0000 7f23c1d9b700 1 --2- 172.21.15.44:0/3811602007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c40a9910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f23b8006fd0 tx=0x7f23b8006e50).ready entity=mon.0 client_cookie=ccdd02e9d063f845 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c159a700 1 -- 172.21.15.44:0/3811602007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8006030 con 0x7f23c40a84b0 2020-11-20T12:16:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c159a700 1 -- 172.21.15.44:0/3811602007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b8006190 con 0x7f23c40a84b0 2020-11-20T12:16:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c159a700 1 -- 172.21.15.44:0/3811602007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8006510 con 0x7f23c40a84b0 2020-11-20T12:16:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 -- 172.21.15.44:0/3811602007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 msgr2=0x7f23c40a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 --2- 172.21.15.44:0/3811602007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c40a9910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f23b8006fd0 tx=0x7f23b8006e50).stop 2020-11-20T12:16:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 -- 172.21.15.44:0/3811602007 shutdown_connections 2020-11-20T12:16:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 --2- 172.21.15.44:0/3811602007 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40a3250 0x7f23c4137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 --2- 172.21.15.44:0/3811602007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c40a9910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 -- 172.21.15.44:0/3811602007 >> 172.21.15.44:0/3811602007 conn(0x7f23c40ad6d0 msgr2=0x7f23c40adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 -- 172.21.15.44:0/3811602007 shutdown_connections 2020-11-20T12:16:43.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.526+0000 7f23c8d45700 1 -- 172.21.15.44:0/3811602007 wait complete. 2020-11-20T12:16:43.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.527+0000 7f23c8d45700 1 Processor -- start 2020-11-20T12:16:43.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.527+0000 7f23c8d45700 1 -- start start 2020-11-20T12:16:43.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c8d45700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4068f60 0x7f23c4136230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c8d45700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c40a3250 0x7f23c40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c8d45700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40657a0 0x7f23c4065c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c8d45700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40ad200 con 0x7f23c4068f60 2020-11-20T12:16:43.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c8d45700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40678a0 con 0x7f23c40a3250 2020-11-20T12:16:43.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c8d45700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40ad490 con 0x7f23c40657a0 2020-11-20T12:16:43.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4068f60 0x7f23c4136230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4068f60 0x7f23c4136230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41858/0 (socket says 172.21.15.44:41858) 2020-11-20T12:16:43.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c259c700 1 -- 172.21.15.44:0/2386356630 learned_addr learned my addr 172.21.15.44:0/2386356630 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:43.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c1d9b700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c40a3250 0x7f23c40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c2d9d700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40657a0 0x7f23c4065c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c259c700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40657a0 msgr2=0x7f23c4065c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.528+0000 7f23c259c700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f23c40657a0 0x7f23c4065c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23c259c700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c40a3250 msgr2=0x7f23c40acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23c259c700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c40a3250 0x7f23c40acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23c259c700 1 -- 172.21.15.44:0/2386356630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4067de0 con 0x7f23c4068f60 2020-11-20T12:16:43.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23c259c700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4068f60 0x7f23c4136230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f23b4001e70 tx=0x7f23b4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4007020 con 0x7f23c4068f60 2020-11-20T12:16:43.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4007180 con 0x7f23c4068f60 2020-11-20T12:16:43.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c4068080 con 0x7f23c4068f60 2020-11-20T12:16:43.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b40064f0 con 0x7f23c4068f60 2020-11-20T12:16:43.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.529+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c4138870 con 0x7f23c4068f60 2020-11-20T12:16:43.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.530+0000 7f23c8d45700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c4131450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.530+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4069670 con 0x7f23c40a84b0 2020-11-20T12:16:43.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.530+0000 7f23c1d9b700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c4131450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.530+0000 7f23c1d9b700 1 -- 172.21.15.44:0/2386356630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23c40a2ea0 con 0x7f23c40a84b0 2020-11-20T12:16:43.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.530+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f23b4010070 con 0x7f23c4068f60 2020-11-20T12:16:43.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.530+0000 7f23c1d9b700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c4131450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f23b8006e90 tx=0x7f23b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2398056d20 0x7f23980591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f23b406f3f0 con 0x7f23c4068f60 2020-11-20T12:16:43.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8023030 con 0x7f23c40a84b0 2020-11-20T12:16:43.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23c2d9d700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2398056d20 0x7f23980591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f23b8005a30 con 0x7f23c40a84b0 2020-11-20T12:16:43.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 msgr2=0x7f23c4131450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c4131450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f23b8006e90 tx=0x7f23b8015000).stop 2020-11-20T12:16:43.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 msgr2=0x7f23c4131450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:43.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23b27fc700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c40a84b0 0x7f23c4131450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.531+0000 7f23c2d9d700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2398056d20 0x7f23980591e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f23c40a36d0 tx=0x7f23bc0091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.540+0000 7f23c8d45700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4131940 0x7f23c40ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.540+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40ad340 con 0x7f23c4131940 2020-11-20T12:16:43.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.540+0000 7f23c1d9b700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4131940 0x7f23c40ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.540+0000 7f23c1d9b700 1 -- 172.21.15.44:0/2386356630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f23c40a2ca0 con 0x7f23c4131940 2020-11-20T12:16:43.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.540+0000 7f23c1d9b700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4131940 0x7f23c40ab4e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f23c405ef20 tx=0x7f23b8021070).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.540+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8033060 con 0x7f23c4131940 2020-11-20T12:16:43.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4609 (secure 0 0 0) 0x7f23b8005820 con 0x7f23c4131940 2020-11-20T12:16:43.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4131940 msgr2=0x7f23c40ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23b27fc700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4131940 0x7f23c40ab4e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f23c405ef20 tx=0x7f23b8021070).stop 2020-11-20T12:16:43.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23b27fc700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4131940 msgr2=0x7f23c40ab4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:43.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23b27fc700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4131940 0x7f23c40ab4e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2398056d20 msgr2=0x7f23980591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2398056d20 0x7f23980591e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f23c40a36d0 tx=0x7f23bc0091d0).stop 2020-11-20T12:16:43.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4068f60 msgr2=0x7f23c4136230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 --2- 172.21.15.44:0/2386356630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4068f60 0x7f23c4136230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f23b4001e70 tx=0x7f23b4007c50).stop 2020-11-20T12:16:43.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c2d9d700 1 -- 172.21.15.44:0/2386356630 reap_dead start 2020-11-20T12:16:43.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c2d9d700 1 -- 172.21.15.44:0/2386356630 reap_dead start 2020-11-20T12:16:43.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 shutdown_connections 2020-11-20T12:16:43.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 >> 172.21.15.44:0/2386356630 conn(0x7f23c40ad6d0 msgr2=0x7f23c4137a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 shutdown_connections 2020-11-20T12:16:43.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.541+0000 7f23c8d45700 1 -- 172.21.15.44:0/2386356630 wait complete. 2020-11-20T12:16:43.556 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:16:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 130, 2020-11-20T12:16:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:16:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 13, 2020-11-20T12:16:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:16:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:16:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:16:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:43.594 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:43.596 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:16:43.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.735+0000 7fdcb7e12700 1 Processor -- start 2020-11-20T12:16:43.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.736+0000 7fdcb7e12700 1 -- start start 2020-11-20T12:16:43.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.736+0000 7fdcb7e12700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.736+0000 7fdcb7e12700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdcb013f800 0x7fdcb0141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.736+0000 7fdcb7e12700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb00a6260 con 0x7fdcb0060330 2020-11-20T12:16:43.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.736+0000 7fdcb7e12700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb00a52e0 con 0x7fdcb013f800 2020-11-20T12:16:43.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.736+0000 7fdcb7e12700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00a6490 con 0x7fdcb013a990 2020-11-20T12:16:43.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb63af700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdcb013f800 0x7fdcb0141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb013adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53308/0 (socket says 172.21.15.44:53308) 2020-11-20T12:16:43.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 -- 172.21.15.44:0/425475993 learned_addr learned my addr 172.21.15.44:0/425475993 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:43.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb63af700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdcb013f800 0x7fdcb0141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:33924/0 (socket says 172.21.15.44:33924) 2020-11-20T12:16:43.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 -- 172.21.15.44:0/425475993 >> v2:172.21.15.79:3300/0 conn(0x7fdcb013f800 msgr2=0x7fdcb0141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:43.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 --2- 172.21.15.44:0/425475993 >> v2:172.21.15.79:3300/0 conn(0x7fdcb013f800 0x7fdcb0141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 -- 172.21.15.44:0/425475993 >> v1:172.21.15.79:6789/0 conn(0x7fdcb0060330 legacy=0x7fdcb00a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 -- 172.21.15.44:0/425475993 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb00a6730 con 0x7fdcb013a990 2020-11-20T12:16:43.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb53ad700 1 --2- 172.21.15.44:0/425475993 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb013adf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fdca0006fd0 tx=0x7fdca0006eb0).ready entity=mon.2 client_cookie=4a3b879aeb9ad68c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb4bac700 1 -- 172.21.15.44:0/425475993 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0009810 con 0x7fdcb013a990 2020-11-20T12:16:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb4bac700 1 -- 172.21.15.44:0/425475993 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0009970 con 0x7fdcb013a990 2020-11-20T12:16:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.737+0000 7fdcb4bac700 1 -- 172.21.15.44:0/425475993 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0009cf0 con 0x7fdcb013a990 2020-11-20T12:16:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 -- 172.21.15.44:0/425475993 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 msgr2=0x7fdcb013adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 --2- 172.21.15.44:0/425475993 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb013adf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fdca0006fd0 tx=0x7fdca0006eb0).stop 2020-11-20T12:16:43.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 -- 172.21.15.44:0/425475993 shutdown_connections 2020-11-20T12:16:43.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 --2- 172.21.15.44:0/425475993 >> v2:172.21.15.79:3300/0 conn(0x7fdcb013f800 0x7fdcb0141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 --2- 172.21.15.44:0/425475993 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb013adf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 -- 172.21.15.44:0/425475993 >> 172.21.15.44:0/425475993 conn(0x7fdcb0062690 msgr2=0x7fdcb00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 -- 172.21.15.44:0/425475993 shutdown_connections 2020-11-20T12:16:43.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.738+0000 7fdcb7e12700 1 -- 172.21.15.44:0/425475993 wait complete. 2020-11-20T12:16:43.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.739+0000 7fdcb7e12700 1 Processor -- start 2020-11-20T12:16:43.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.739+0000 7fdcb7e12700 1 -- start start 2020-11-20T12:16:43.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb7e12700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0060330 0x7fdcb0139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb7e12700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb00ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb7e12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcb013f800 0x7fdcb0135a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb7e12700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0137850 con 0x7fdcb013f800 2020-11-20T12:16:43.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb7e12700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0136c10 con 0x7fdcb0060330 2020-11-20T12:16:43.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb7e12700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0136e70 con 0x7fdcb013a990 2020-11-20T12:16:43.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb63af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcb013f800 0x7fdcb0135a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb53ad700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb00ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb63af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcb013f800 0x7fdcb0135a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41880/0 (socket says 172.21.15.44:41880) 2020-11-20T12:16:43.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb5bae700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0060330 0x7fdcb0139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb63af700 1 -- 172.21.15.44:0/474515475 learned_addr learned my addr 172.21.15.44:0/474515475 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:43.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb53ad700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0060330 msgr2=0x7fdcb0139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb53ad700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0060330 0x7fdcb0139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb53ad700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcb013f800 msgr2=0x7fdcb0135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb53ad700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcb013f800 0x7fdcb0135a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb53ad700 1 -- 172.21.15.44:0/474515475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0137110 con 0x7fdcb013a990 2020-11-20T12:16:43.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.740+0000 7fdcb63af700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdcb013f800 0x7fdcb0135a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:43.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.741+0000 7fdcb53ad700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb00ab800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fdca0012000 tx=0x7fdca0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.741+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0011d50 con 0x7fdcb013a990 2020-11-20T12:16:43.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.741+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca001e460 con 0x7fdcb013a990 2020-11-20T12:16:43.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.741+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca001e7e0 con 0x7fdcb013a990 2020-11-20T12:16:43.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.741+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb01373b0 con 0x7fdcb013a990 2020-11-20T12:16:43.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.741+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb00abeb0 con 0x7fdcb013a990 2020-11-20T12:16:43.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.742+0000 7fdcb7e12700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0135f80 0x7fdcb012e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.742+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0136360 con 0x7fdcb0135f80 2020-11-20T12:16:43.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.742+0000 7fdcb5bae700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0135f80 0x7fdcb012e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.742+0000 7fdcb5bae700 1 -- 172.21.15.44:0/474515475 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdcb0061650 con 0x7fdcb0135f80 2020-11-20T12:16:43.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.742+0000 7fdcb5bae700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0135f80 0x7fdcb012e0f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdcac00f820 tx=0x7fdcac000b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.742+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fdca001a040 con 0x7fdcb013a990 2020-11-20T12:16:43.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdc88056d20 0x7fdc880591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fdca0071320 con 0x7fdcb013a990 2020-11-20T12:16:43.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdcb63af700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdc88056d20 0x7fdc880591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00d6b0 con 0x7fdcb0135f80 2020-11-20T12:16:43.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fdcac00d810 con 0x7fdcb0135f80 2020-11-20T12:16:43.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0135f80 msgr2=0x7fdcb012e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0135f80 0x7fdcb012e0f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdcac00f820 tx=0x7fdcac000b80).stop 2020-11-20T12:16:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0135f80 msgr2=0x7fdcb012e0f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdc9dffb700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb0135f80 0x7fdcb012e0f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.743+0000 7fdcb63af700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdc88056d20 0x7fdc880591e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fdca80115c0 tx=0x7fdca80091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.752+0000 7fdcb7e12700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb012e5e0 0x7fdcb0133150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.752+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb013f5e0 con 0x7fdcb012e5e0 2020-11-20T12:16:43.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.752+0000 7fdcb5bae700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb012e5e0 0x7fdcb0133150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.752+0000 7fdcb5bae700 1 -- 172.21.15.44:0/474515475 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdcb00446d0 con 0x7fdcb012e5e0 2020-11-20T12:16:43.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.752+0000 7fdcb5bae700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb012e5e0 0x7fdcb0133150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fdcac00f820 tx=0x7fdcac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac014cb0 con 0x7fdcb012e5e0 2020-11-20T12:16:43.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7fdcac014e10 con 0x7fdcb012e5e0 2020-11-20T12:16:43.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb012e5e0 msgr2=0x7fdcb0133150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdc9dffb700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb012e5e0 0x7fdcb0133150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fdcac00f820 tx=0x7fdcac016000).stop 2020-11-20T12:16:43.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdc9dffb700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb012e5e0 msgr2=0x7fdcb0133150 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:43.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdc9dffb700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcb012e5e0 0x7fdcb0133150 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdc88056d20 msgr2=0x7fdc880591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fdc88056d20 0x7fdc880591e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fdca80115c0 tx=0x7fdca80091d0).stop 2020-11-20T12:16:43.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 msgr2=0x7fdcb00ab800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 --2- 172.21.15.44:0/474515475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fdcb013a990 0x7fdcb00ab800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fdca0012000 tx=0x7fdca0015000).stop 2020-11-20T12:16:43.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb63af700 1 -- 172.21.15.44:0/474515475 reap_dead start 2020-11-20T12:16:43.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb63af700 1 -- 172.21.15.44:0/474515475 reap_dead start 2020-11-20T12:16:43.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 shutdown_connections 2020-11-20T12:16:43.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 >> 172.21.15.44:0/474515475 conn(0x7fdcb0062690 msgr2=0x7fdcb00a8500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 shutdown_connections 2020-11-20T12:16:43.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.753+0000 7fdcb7e12700 1 -- 172.21.15.44:0/474515475 wait complete. 2020-11-20T12:16:43.766 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:16:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 130, 2020-11-20T12:16:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:16:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 13, 2020-11-20T12:16:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:43.799 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:43.800 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:16:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.940+0000 7f7848144700 1 Processor -- start 2020-11-20T12:16:43.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.940+0000 7f7848144700 1 -- start start 2020-11-20T12:16:43.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7848144700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7840064190 0x7f7840137bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7848144700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7840061ea0 con 0x7f78400aa500 2020-11-20T12:16:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7848144700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7840062140 con 0x7f78400a99e0 2020-11-20T12:16:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7848144700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7840061cb0 con 0x7f7840064190 2020-11-20T12:16:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7845ee0700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f78400aa500 0x7f7840064b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45636/0 (socket says 172.21.15.44:45636) 2020-11-20T12:16:43.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7845ee0700 1 -- 172.21.15.44:0/107773062 learned_addr learned my addr 172.21.15.44:0/107773062 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:16:43.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f78466e1700 1 --2- 172.21.15.44:0/107773062 >> v2:172.21.15.79:3300/0 conn(0x7f7840064190 0x7f7840137bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359111193 0 0) 0x7f7840061ea0 con 0x7f78400aa500 2020-11-20T12:16:43.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7828003240 con 0x7f78400aa500 2020-11-20T12:16:43.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1003105044 0 0) 0x7f7828003240 con 0x7f78400aa500 2020-11-20T12:16:43.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 >> v2:172.21.15.79:3300/0 conn(0x7f7840064190 msgr2=0x7f7840137bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:43.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 --2- 172.21.15.44:0/107773062 >> v2:172.21.15.79:3300/0 conn(0x7f7840064190 0x7f7840137bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 >> v1:172.21.15.79:6789/0 conn(0x7f78400a99e0 legacy=0x7f78401374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400aa8e0 con 0x7f78400aa500 2020-11-20T12:16:43.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f783c0014e0 con 0x7f78400aa500 2020-11-20T12:16:43.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 >> v1:172.21.15.44:6789/0 conn(0x7f78400aa500 legacy=0x7f7840064b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7828003730 0x7f7828003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f78280044e0 0x7f7828004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f782800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7828004260 con 0x7f7828004e50 2020-11-20T12:16:43.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782800bc20 con 0x7f7828003730 2020-11-20T12:16:43.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.941+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782800bd60 con 0x7f78280044e0 2020-11-20T12:16:43.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f782800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f7845ee0700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7828003730 0x7f7828003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78456df700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f78280044e0 0x7f7828004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 -- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f78280044e0 msgr2=0x7f7828004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:43.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f78280044e0 0x7f7828004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 -- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7828003730 msgr2=0x7f7828003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:43.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7828003730 0x7f7828003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 -- 172.21.15.44:0/107773062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828003240 con 0x7f7828004e50 2020-11-20T12:16:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78456df700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f78280044e0 0x7f7828004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 -- 172.21.15.44:0/107773062 reap_dead start 2020-11-20T12:16:43.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f78466e1700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f782800b730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f7838009ea0 tx=0x7f78380091d0).ready entity=mon.0 client_cookie=79e4dc3cd8dca5e9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78380095c0 con 0x7f7828004e50 2020-11-20T12:16:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783800ecf0 con 0x7f7828004e50 2020-11-20T12:16:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.942+0000 7f7844ede700 1 -- 172.21.15.44:0/107773062 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838012640 con 0x7f7828004e50 2020-11-20T12:16:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.943+0000 7f7848144700 1 -- 172.21.15.44:0/107773062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 msgr2=0x7f782800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.943+0000 7f7848144700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f782800b730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f7838009ea0 tx=0x7f78380091d0).stop 2020-11-20T12:16:43.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.943+0000 7f7848144700 1 -- 172.21.15.44:0/107773062 shutdown_connections 2020-11-20T12:16:43.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.943+0000 7f7848144700 1 --2- 172.21.15.44:0/107773062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f782800b730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.943+0000 7f7848144700 1 -- 172.21.15.44:0/107773062 >> 172.21.15.44:0/107773062 conn(0x7f78400af040 msgr2=0x7f78400af450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.943+0000 7f7848144700 1 -- 172.21.15.44:0/107773062 shutdown_connections 2020-11-20T12:16:43.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.943+0000 7f7848144700 1 -- 172.21.15.44:0/107773062 wait complete. 2020-11-20T12:16:43.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.944+0000 7f7848144700 1 Processor -- start 2020-11-20T12:16:43.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.944+0000 7f7848144700 1 -- start start 2020-11-20T12:16:43.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.944+0000 7f7848144700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f784013c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.944+0000 7f7848144700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7840136a10 0x7f7840130c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7848144700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840131160 0x7f784014d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7848144700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784013bc00 con 0x7f7828004e50 2020-11-20T12:16:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7848144700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78401315e0 con 0x7f7840136a10 2020-11-20T12:16:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7848144700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400ad8f0 con 0x7f7840131160 2020-11-20T12:16:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f784013c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f784013c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41900/0 (socket says 172.21.15.44:41900) 2020-11-20T12:16:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 -- 172.21.15.44:0/1032617461 learned_addr learned my addr 172.21.15.44:0/1032617461 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:16:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f78466e1700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840131160 0x7f784014d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f78456df700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7840136a10 0x7f7840130c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840131160 msgr2=0x7f784014d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840131160 0x7f784014d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7840136a10 msgr2=0x7f7840130c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7840136a10 0x7f7840130c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 -- 172.21.15.44:0/1032617461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840136570 con 0x7f7828004e50 2020-11-20T12:16:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7845ee0700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f784013c130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f783c001610 tx=0x7f7840062030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c00f8d0 con 0x7f7828004e50 2020-11-20T12:16:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783c00fa30 con 0x7f7828004e50 2020-11-20T12:16:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78400ace50 con 0x7f7828004e50 2020-11-20T12:16:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.945+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c00fdb0 con 0x7f7828004e50 2020-11-20T12:16:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.946+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7840135830 con 0x7f7828004e50 2020-11-20T12:16:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.946+0000 7f7848144700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840136df0 0x7f784014dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.946+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784013c7e0 con 0x7f7840136df0 2020-11-20T12:16:43.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f78456df700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840136df0 0x7f784014dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f783c01b030 con 0x7f7828004e50 2020-11-20T12:16:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f78456df700 1 -- 172.21.15.44:0/1032617461 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78401363f0 con 0x7f7840136df0 2020-11-20T12:16:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f782dffb700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7818056d20 0x7f78180591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f783c06f920 con 0x7f7828004e50 2020-11-20T12:16:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f78456df700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840136df0 0x7f784014dbc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f78400a9de0 tx=0x7f783000c040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f78466e1700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7818056d20 0x7f78180591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.947+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830013640 con 0x7f7840136df0 2020-11-20T12:16:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.948+0000 7f78466e1700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7818056d20 0x7f78180591e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f78280043a0 tx=0x7f7838019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.948+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f7830010040 con 0x7f7840136df0 2020-11-20T12:16:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.948+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840136df0 msgr2=0x7f784014dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.948+0000 7f782dffb700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840136df0 0x7f784014dbc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f78400a9de0 tx=0x7f783000c040).stop 2020-11-20T12:16:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.948+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840136df0 msgr2=0x7f784014dbc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.948+0000 7f782dffb700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7840136df0 0x7f784014dbc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.957+0000 7f7848144700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f784014e0b0 0x7f78400625e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.957+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400ad730 con 0x7f784014e0b0 2020-11-20T12:16:43.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.957+0000 7f78456df700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f784014e0b0 0x7f78400625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.957+0000 7f78456df700 1 -- 172.21.15.44:0/1032617461 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7840134830 con 0x7f784014e0b0 2020-11-20T12:16:43.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.957+0000 7f78456df700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f784014e0b0 0x7f78400625e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f7830018000 tx=0x7f783001b000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.957+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300233f0 con 0x7f784014e0b0 2020-11-20T12:16:43.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f7830013430 con 0x7f784014e0b0 2020-11-20T12:16:43.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f784014e0b0 msgr2=0x7f78400625e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f782dffb700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f784014e0b0 0x7f78400625e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f7830018000 tx=0x7f783001b000).stop 2020-11-20T12:16:43.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f782dffb700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f784014e0b0 msgr2=0x7f78400625e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:16:43.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f782dffb700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f784014e0b0 0x7f78400625e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7818056d20 msgr2=0x7f78180591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7818056d20 0x7f78180591e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f78280043a0 tx=0x7f7838019000).stop 2020-11-20T12:16:43.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 msgr2=0x7f784013c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 --2- 172.21.15.44:0/1032617461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7828004e50 0x7f784013c130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f783c001610 tx=0x7f7840062030).stop 2020-11-20T12:16:43.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f78466e1700 1 -- 172.21.15.44:0/1032617461 reap_dead start 2020-11-20T12:16:43.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f78466e1700 1 -- 172.21.15.44:0/1032617461 reap_dead start 2020-11-20T12:16:43.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 shutdown_connections 2020-11-20T12:16:43.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 >> 172.21.15.44:0/1032617461 conn(0x7f78400af040 msgr2=0x7f7840064ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 shutdown_connections 2020-11-20T12:16:43.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:43.958+0000 7f7848144700 1 -- 172.21.15.44:0/1032617461 wait complete. 2020-11-20T12:16:43.974 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:16:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:16:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 130, 2020-11-20T12:16:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:16:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:16:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:16:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:16:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 13, 2020-11-20T12:16:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:16:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:16:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:16:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:16:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:16:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:16:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:16:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:16:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:16:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:16:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:16:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:16:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:16:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:16:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:16:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:16:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:16:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:16:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:16:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:16:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:16:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:16:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:16:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:16:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:16:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:16:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:16:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:16:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:16:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:16:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:16:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:16:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:16:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:16:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:16:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:16:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:16:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:16:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:16:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:16:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:16:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:16:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:16:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:16:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:16:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:16:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:16:43.999 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:16:43.999 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:16:43.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:43.999 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:43.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:44.000 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:16:44.000 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:16:44.000 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:16:44.000 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:16:44.000 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:16:44.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:44.001 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:44.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:44.001 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:16:44.001 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:16:44.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:44.002 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:44.002 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:44.002 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:44.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:44.003 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:44.003 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:16:44.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:44.003 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:44.003 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:44.004 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:16:44.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:44.004 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:44.004 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:16:44.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:44.005 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:44.005 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:44.005 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:44.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:44.005 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:44.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:16:44.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:44.006 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:44.006 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:44.006 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:16:44.007 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:44.007 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:16:44.007 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:16:44.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:16:44.007 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:16:44.008 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:16:44.008 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:16:44.008 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:16:44.008 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:16:44.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:16:44.009 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:16:44.009 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:16:44.010 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:16:44.010 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2020-11-20T12:16:44.011 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:16:44.011 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:16:44.011 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:16:44.011 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:16:44.022 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:16:44.023 INFO:tasks.mon_thrash:Sending STOP to mon a 2020-11-20T12:16:44.023 INFO:tasks.ceph.mon.a:Sent signal 19 2020-11-20T12:16:44.023 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2020-11-20T12:16:59.024 INFO:tasks.mon_thrash:Sending CONT to mon a 2020-11-20T12:16:59.025 INFO:tasks.ceph.mon.a:Sent signal 18 2020-11-20T12:16:59.025 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:16:59.026 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:16:59.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.158+0000 7feec1b93700 1 Processor -- start 2020-11-20T12:16:59.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.158+0000 7feec1b93700 1 -- start start 2020-11-20T12:16:59.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.158+0000 7feec1b93700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 0x7feebc13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:59.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.158+0000 7feec1b93700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:59.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.158+0000 7feec1b93700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc0a6260 con 0x7feebc060330 2020-11-20T12:16:59.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feec1b93700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0a52e0 con 0x7feebc13a990 2020-11-20T12:16:59.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feec1b93700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0a6490 con 0x7feebc13f800 2020-11-20T12:16:59.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feebb7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7feebc060330 0x7feebc0a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45662/0 (socket says 172.21.15.44:45662) 2020-11-20T12:16:59.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feeb3fff700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 msgr2=0x7feebc13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:59.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 learned_addr learned my addr 172.21.15.44:0/753840540 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:16:59.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feeb3fff700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 0x7feebc13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:59.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:59.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:16:59.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:59.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:59.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.159+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:16:59.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.360+0000 7feeb3fff700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 msgr2=0x7feebc13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.360+0000 7feeb3fff700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 0x7feebc13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:16:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.360+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.360+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:16:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.360+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:59.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.360+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:59.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.360+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:16:59.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.761+0000 7feeb3fff700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 msgr2=0x7feebc13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:16:59.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.761+0000 7feeb3fff700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 0x7feebc13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:16:59.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.761+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:59.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.761+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:16:59.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.761+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:16:59.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.761+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:16:59.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:16:59.761+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:17:00.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:00.562+0000 7feeb3fff700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 msgr2=0x7feebc13ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:00.562+0000 7feeb3fff700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 0x7feebc13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:00.562+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:00.562+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:00.562+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:00.562+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:17:00.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:00.562+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:17:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 msgr2=0x7feebc141bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feebc13f800 0x7feebc141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 msgr2=0x7feebc13ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feebc13a990 0x7feebc13ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 >> v1:172.21.15.44:6789/0 conn(0x7feebc060330 legacy=0x7feebc0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc0a6730 con 0x7feea0005e30 2020-11-20T12:17:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0a52e0 con 0x7feea00033b0 2020-11-20T12:17:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feeb3fff700 1 -- 172.21.15.44:0/753840540 >> v1:172.21.15.79:6789/0 conn(0x7feea0005e30 legacy=0x7feea00082f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:17:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:17:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:17:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:17:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.159+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:02.363 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.360+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.360+0000 7feeb3fff700 1 -- 172.21.15.44:0/753840540 >> v1:172.21.15.79:6789/0 conn(0x7feea0005e30 legacy=0x7feea00082f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:17:02.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.360+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:17:02.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.360+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:17:02.364 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.360+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:17:02.365 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.360+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:17:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.761+0000 7feeb3fff700 1 -- 172.21.15.44:0/753840540 >> v1:172.21.15.79:6789/0 conn(0x7feea0005e30 legacy=0x7feea00082f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:17:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.761+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.761+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:17:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.761+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:17:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.761+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:17:02.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:02.761+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:17:03.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:03.562+0000 7feeb3fff700 1 -- 172.21.15.44:0/753840540 >> v1:172.21.15.79:6789/0 conn(0x7feea0005e30 legacy=0x7feea00082f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:17:03.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:03.562+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:03.562+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:17:03.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:03.562+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:17:03.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:03.562+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:17:03.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:03.562+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:17:05.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 msgr2=0x7feea0005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:05.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea00033b0 0x7feea0005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 >> v1:172.21.15.79:6789/0 conn(0x7feea0005e30 legacy=0x7feea00082f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 reap_dead start 2020-11-20T12:17:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea0008ac0 0x7feea0005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008f40 0x7feea00082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feea0009380 0x7feea0009b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0a52e0 con 0x7feea0008ac0 2020-11-20T12:17:05.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0a6490 con 0x7feea0009380 2020-11-20T12:17:05.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb9ffb700 1 -- 172.21.15.44:0/753840540 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea0009970 con 0x7feea0008f40 2020-11-20T12:17:05.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.44:3300/0 conn(0x7feea0008ac0 0x7feea0005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebbfff700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feea0009380 msgr2=0x7feea0009b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:17:05.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebbfff700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feea0009380 0x7feea0009b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:05.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feeb3fff700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008f40 0x7feea00082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008f40 msgr2=0x7feea00082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008f40 0x7feea00082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feea0009380 msgr2=0x7feea0009b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:05.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feea0009380 0x7feea0009b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebb7fe700 1 -- 172.21.15.44:0/753840540 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea000aca0 con 0x7feea0008ac0 2020-11-20T12:17:05.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.159+0000 7feebb7fe700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008ac0 0x7feea0005830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7feea0008900 tx=0x7feeac009040).ready entity=mon.0 client_cookie=61cf5cf4e283a8bf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.160+0000 7feebaffd700 1 -- 172.21.15.44:0/753840540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac016690 con 0x7feea0008ac0 2020-11-20T12:17:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.160+0000 7feebaffd700 1 -- 172.21.15.44:0/753840540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac0167f0 con 0x7feea0008ac0 2020-11-20T12:17:05.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.160+0000 7feebaffd700 1 -- 172.21.15.44:0/753840540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac016b70 con 0x7feea0008ac0 2020-11-20T12:17:05.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.160+0000 7feec1b93700 1 -- 172.21.15.44:0/753840540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008ac0 msgr2=0x7feea0005830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.160+0000 7feec1b93700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008ac0 0x7feea0005830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7feea0008900 tx=0x7feeac009040).stop 2020-11-20T12:17:05.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.161+0000 7feec1b93700 1 -- 172.21.15.44:0/753840540 shutdown_connections 2020-11-20T12:17:05.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.161+0000 7feec1b93700 1 --2- 172.21.15.44:0/753840540 >> v2:172.21.15.79:3300/0 conn(0x7feea0009380 0x7feea0009b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.161+0000 7feec1b93700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008f40 0x7feea00082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.161+0000 7feec1b93700 1 --2- 172.21.15.44:0/753840540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008ac0 0x7feea0005830 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.161+0000 7feec1b93700 1 -- 172.21.15.44:0/753840540 >> 172.21.15.44:0/753840540 conn(0x7feebc062690 msgr2=0x7feebc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:05.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.161+0000 7feec1b93700 1 -- 172.21.15.44:0/753840540 shutdown_connections 2020-11-20T12:17:05.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.161+0000 7feec1b93700 1 -- 172.21.15.44:0/753840540 wait complete. 2020-11-20T12:17:05.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 Processor -- start 2020-11-20T12:17:05.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 -- start start 2020-11-20T12:17:05.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008ac0 0x7feebc131ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008f40 0x7feebc0ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea0009380 0x7feebc0acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0ad0f0 con 0x7feea0008f40 2020-11-20T12:17:05.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc134c80 con 0x7feea0009380 2020-11-20T12:17:05.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feec1b93700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0abd30 con 0x7feea0008ac0 2020-11-20T12:17:05.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feebbfff700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea0009380 msgr2=0x7feebc0acc00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:17:05.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feeb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008f40 0x7feebc0ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feebbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea0009380 0x7feebc0acc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:05.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feeb3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008f40 0x7feebc0ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41962/0 (socket says 172.21.15.44:41962) 2020-11-20T12:17:05.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feeb3fff700 1 -- 172.21.15.44:0/463270610 learned_addr learned my addr 172.21.15.44:0/463270610 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.162+0000 7feebb7fe700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008ac0 0x7feebc131ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb3fff700 1 -- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008ac0 msgr2=0x7feebc131ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb3fff700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008ac0 0x7feebc131ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb3fff700 1 -- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea0009380 msgr2=0x7feebc0acc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:17:05.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb3fff700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea0009380 0x7feebc0acc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb3fff700 1 -- 172.21.15.44:0/463270610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0abfd0 con 0x7feea0008f40 2020-11-20T12:17:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb3fff700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008f40 0x7feebc0ac710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7feea4003ae0 tx=0x7feea4009510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb37fe700 1 -- 172.21.15.44:0/463270610 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4010070 con 0x7feea0008f40 2020-11-20T12:17:05.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb37fe700 1 -- 172.21.15.44:0/463270610 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea40074d0 con 0x7feea0008f40 2020-11-20T12:17:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feeb37fe700 1 -- 172.21.15.44:0/463270610 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4005ae0 con 0x7feea0008f40 2020-11-20T12:17:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc0ac270 con 0x7feea0008f40 2020-11-20T12:17:05.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.163+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc137960 con 0x7feea0008f40 2020-11-20T12:17:05.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.164+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc0ac400 con 0x7feea0008f40 2020-11-20T12:17:05.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.165+0000 7feeb37fe700 1 -- 172.21.15.44:0/463270610 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7feea4007a60 con 0x7feea0008f40 2020-11-20T12:17:05.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.165+0000 7feeb37fe700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee98056cd0 0x7fee98059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.165+0000 7feeb37fe700 1 -- 172.21.15.44:0/463270610 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7feea4070e30 con 0x7feea0008f40 2020-11-20T12:17:05.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.165+0000 7feebb7fe700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee98056cd0 0x7fee98059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.166+0000 7feebb7fe700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee98056cd0 0x7fee98059190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7feea0008900 tx=0x7feeac009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.169+0000 7feeb37fe700 1 -- 172.21.15.44:0/463270610 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7feea40460a0 con 0x7feea0008f40 2020-11-20T12:17:05.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.450+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feebc061650 con 0x7feea0008f40 2020-11-20T12:17:05.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.450+0000 7feeb37fe700 1 -- 172.21.15.44:0/463270610 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7feea40496c0 con 0x7feea0008f40 2020-11-20T12:17:05.454 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":134,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:17:05.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee98056cd0 msgr2=0x7fee98059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee98056cd0 0x7fee98059190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7feea0008900 tx=0x7feeac009040).stop 2020-11-20T12:17:05.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008f40 msgr2=0x7feebc0ac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008f40 0x7feebc0ac710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7feea4003ae0 tx=0x7feea4009510).stop 2020-11-20T12:17:05.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 shutdown_connections 2020-11-20T12:17:05.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fee98056cd0 0x7fee98059190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feea0009380 0x7feebc0acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7feea0008ac0 0x7feebc131ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 --2- 172.21.15.44:0/463270610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feea0008f40 0x7feebc0ac710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 >> 172.21.15.44:0/463270610 conn(0x7feebc062690 msgr2=0x7feebc0a4540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:05.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 shutdown_connections 2020-11-20T12:17:05.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.452+0000 7feec1b93700 1 -- 172.21.15.44:0/463270610 wait complete. 2020-11-20T12:17:05.470 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:17:05.470 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:17:05.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.610+0000 7efcac943700 1 Processor -- start 2020-11-20T12:17:05.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.610+0000 7efcac943700 1 -- start start 2020-11-20T12:17:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efcac943700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efca80aa400 0x7efca80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efcac943700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca80a9770 con 0x7efca813ac60 2020-11-20T12:17:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efcac943700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca80a6da0 con 0x7efca80aa020 2020-11-20T12:17:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efcac943700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca80a56c0 con 0x7efca80aa400 2020-11-20T12:17:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efc9ffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efca80aa400 0x7efca80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efca659c700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7efca80aa020 legacy=0x7efca80a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:17:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efca6d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7efca813ac60 0x7efca80a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45722/0 (socket says 172.21.15.44:45722) 2020-11-20T12:17:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efca6d9d700 1 -- 172.21.15.44:0/1679334445 learned_addr learned my addr 172.21.15.44:0/1679334445 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:17:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efc9ffff700 1 -- 172.21.15.44:0/1679334445 >> v1:172.21.15.79:6789/0 conn(0x7efca80aa020 legacy=0x7efca80a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efc9ffff700 1 -- 172.21.15.44:0/1679334445 >> v1:172.21.15.44:6789/0 conn(0x7efca813ac60 legacy=0x7efca80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efca5d9b700 1 -- 172.21.15.44:0/1679334445 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297979744 0 0) 0x7efca0001450 con 0x7efca813ac60 2020-11-20T12:17:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.611+0000 7efc9ffff700 1 -- 172.21.15.44:0/1679334445 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80a7040 con 0x7efca80aa400 2020-11-20T12:17:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efc9ffff700 1 --2- 172.21.15.44:0/1679334445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca80a9280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efc9000dc30 tx=0x7efc90006e80).ready entity=mon.0 client_cookie=d16f40c67851b432 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efca5d9b700 1 -- 172.21.15.44:0/1679334445 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000bc20 con 0x7efca80aa400 2020-11-20T12:17:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efca5d9b700 1 -- 172.21.15.44:0/1679334445 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9000bd80 con 0x7efca80aa400 2020-11-20T12:17:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efca5d9b700 1 -- 172.21.15.44:0/1679334445 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90010680 con 0x7efca80aa400 2020-11-20T12:17:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efcac943700 1 -- 172.21.15.44:0/1679334445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 msgr2=0x7efca80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efcac943700 1 --2- 172.21.15.44:0/1679334445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca80a9280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efc9000dc30 tx=0x7efc90006e80).stop 2020-11-20T12:17:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efcac943700 1 -- 172.21.15.44:0/1679334445 shutdown_connections 2020-11-20T12:17:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efcac943700 1 --2- 172.21.15.44:0/1679334445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca80a9280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.612+0000 7efcac943700 1 -- 172.21.15.44:0/1679334445 >> 172.21.15.44:0/1679334445 conn(0x7efca8062690 msgr2=0x7efca80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.613+0000 7efcac943700 1 -- 172.21.15.44:0/1679334445 shutdown_connections 2020-11-20T12:17:05.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.613+0000 7efcac943700 1 -- 172.21.15.44:0/1679334445 wait complete. 2020-11-20T12:17:05.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.613+0000 7efcac943700 1 Processor -- start 2020-11-20T12:17:05.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.613+0000 7efcac943700 1 -- start start 2020-11-20T12:17:05.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efcac943700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca80aa020 0x7efca80a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efcac943700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca8066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efcac943700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efca813ac60 0x7efca812ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efcac943700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca8149450 con 0x7efca80aa400 2020-11-20T12:17:05.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efcac943700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca80ac720 con 0x7efca80aa020 2020-11-20T12:17:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efcac943700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca80ac9c0 con 0x7efca813ac60 2020-11-20T12:17:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca8066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efca659c700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca80aa020 msgr2=0x7efca80a7b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:17:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca8066f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41978/0 (socket says 172.21.15.44:41978) 2020-11-20T12:17:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efca659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca80aa020 0x7efca80a7b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efc9ffff700 1 -- 172.21.15.44:0/1881693175 learned_addr learned my addr 172.21.15.44:0/1881693175 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efca6d9d700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efca813ac60 0x7efca812ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efc9ffff700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efca813ac60 msgr2=0x7efca812ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.614+0000 7efc9ffff700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efca813ac60 0x7efca812ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efc9ffff700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca80aa020 msgr2=0x7efca80a7b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:17:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efc9ffff700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efca80aa020 0x7efca80a7b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efc9ffff700 1 -- 172.21.15.44:0/1881693175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8066a80 con 0x7efca80aa400 2020-11-20T12:17:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efc9ffff700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca8066f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efc90006ec0 tx=0x7efca80a6f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90020070 con 0x7efca80aa400 2020-11-20T12:17:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9001bb40 con 0x7efca80aa400 2020-11-20T12:17:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca8067fc0 con 0x7efca80aa400 2020-11-20T12:17:05.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90025410 con 0x7efca80aa400 2020-11-20T12:17:05.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.615+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca80ac4c0 con 0x7efca80aa400 2020-11-20T12:17:05.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.616+0000 7efcac943700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aac60 0x7efca812f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.616+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca8135df0 con 0x7efca80aac60 2020-11-20T12:17:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.616+0000 7efca659c700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aac60 0x7efca812f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.616+0000 7efca659c700 1 -- 172.21.15.44:0/1881693175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efca8066c10 con 0x7efca80aac60 2020-11-20T12:17:05.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.616+0000 7efca659c700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aac60 0x7efca812f360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7efc98003650 tx=0x7efc98006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.616+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7efc9002c070 con 0x7efca80aa400 2020-11-20T12:17:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc84056cd0 0x7efc84059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7efc90074470 con 0x7efca80aa400 2020-11-20T12:17:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc980050d0 con 0x7efca80aac60 2020-11-20T12:17:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7efc98005230 con 0x7efca80aac60 2020-11-20T12:17:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aac60 msgr2=0x7efca812f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efca6d9d700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc84056cd0 0x7efc84059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aac60 0x7efca812f360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7efc98003650 tx=0x7efc98006e10).stop 2020-11-20T12:17:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aac60 msgr2=0x7efca812f360 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efc9e7fc700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aac60 0x7efca812f360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.617+0000 7efca6d9d700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc84056cd0 0x7efc84059190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7efca0007fd0 tx=0x7efca0007f20).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.638 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.625+0000 7efcac943700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca8067410 0x7efca8145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.625+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca8061650 con 0x7efca8067410 2020-11-20T12:17:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.625+0000 7efca659c700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca8067410 0x7efca8145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efca659c700 1 -- 172.21.15.44:0/1881693175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efca80446d0 con 0x7efca8067410 2020-11-20T12:17:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efca659c700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca8067410 0x7efca8145f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efc98003650 tx=0x7efc98013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98013c50 con 0x7efca8067410 2020-11-20T12:17:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4597 (secure 0 0 0) 0x7efc980036d0 con 0x7efca8067410 2020-11-20T12:17:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca8067410 msgr2=0x7efca8145f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efc9e7fc700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca8067410 0x7efca8145f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efc98003650 tx=0x7efc98013000).stop 2020-11-20T12:17:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efc9e7fc700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca8067410 msgr2=0x7efca8145f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efc9e7fc700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca8067410 0x7efca8145f00 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0x7efc98003650 tx=0x7efc98013000).stop 2020-11-20T12:17:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc84056cd0 msgr2=0x7efc84059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efcac943700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc84056cd0 0x7efc84059190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7efca0007fd0 tx=0x7efca0007f20).stop 2020-11-20T12:17:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 msgr2=0x7efca8066f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efcac943700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca8066f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efc90006ec0 tx=0x7efca80a6f30).stop 2020-11-20T12:17:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efca6d9d700 1 -- 172.21.15.44:0/1881693175 reap_dead start 2020-11-20T12:17:05.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 shutdown_connections 2020-11-20T12:17:05.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efcac943700 1 --2- 172.21.15.44:0/1881693175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca80aa400 0x7efca8066f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.626+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 >> 172.21.15.44:0/1881693175 conn(0x7efca8062690 msgr2=0x7efca8131560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.627+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 shutdown_connections 2020-11-20T12:17:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.627+0000 7efcac943700 1 -- 172.21.15.44:0/1881693175 wait complete. 2020-11-20T12:17:05.644 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:05.644 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:05.645 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:05.645 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:17:05.645 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 134, 2020-11-20T12:17:05.646 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:05.646 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:05.646 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:05.646 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.647 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:17:05.647 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:05.647 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:05.648 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:05.648 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:05.648 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:05.648 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:05.648 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:05.649 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:05.649 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:05.649 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:05.649 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:05.650 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.650 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:05.650 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:05.650 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:05.651 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:05.651 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:05.651 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:05.651 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:05.652 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:05.652 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:05.652 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:05.652 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.653 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:05.653 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:05.653 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:05.653 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:05.654 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:05.654 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:05.654 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:05.654 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:05.655 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:05.655 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:05.655 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:05.655 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:05.655 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:05.656 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:05.656 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:05.656 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:05.656 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:05.657 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:05.657 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:05.657 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:05.657 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:05.658 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:05.658 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:05.658 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.658 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:05.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.659 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:05.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.659 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:05.659 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:05.660 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:05.660 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:05.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:05.661 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:05.661 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:05.662 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:05.662 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.662 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.662 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.663 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:05.663 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:05.663 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:05.664 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:05.664 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:05.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.664 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:05.665 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:05.665 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:05.665 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:05.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:05.666 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:05.667 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:05.668 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:05.668 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.668 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.669 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:05.669 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:05.669 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:05.669 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:05.670 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:05.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.671 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:05.671 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:05.671 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:05.671 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:17:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:17:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:17:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:05.684 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:05.685 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:17:05.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.819+0000 7f6ebf664700 1 Processor -- start 2020-11-20T12:17:05.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.820+0000 7f6ebf664700 1 -- start start 2020-11-20T12:17:05.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.820+0000 7f6ebf664700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6eb80aa020 0x7f6eb80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.820+0000 7f6ebf664700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb80aa400 0x7f6eb80a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.820+0000 7f6ebf664700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6eb813ac60 0x7f6eb80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.820+0000 7f6ebf664700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb80a9770 con 0x7f6eb813ac60 2020-11-20T12:17:05.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.820+0000 7f6ebf664700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb80a6da0 con 0x7f6eb80aa020 2020-11-20T12:17:05.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.820+0000 7f6ebf664700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80a7040 con 0x7f6eb80aa400 2020-11-20T12:17:05.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebd400700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f6eb80aa020 msgr2=0x7f6eb80a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:17:05.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebd400700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6eb80aa020 0x7f6eb80a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:05.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6eb813ac60 0x7f6eb80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6eb813ac60 0x7f6eb80a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41992/0 (socket says 172.21.15.44:41992) 2020-11-20T12:17:05.828 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 -- 172.21.15.44:0/4064326057 learned_addr learned my addr 172.21.15.44:0/4064326057 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:05.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebcbff700 1 --2- 172.21.15.44:0/4064326057 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb80aa400 0x7f6eb80a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 -- 172.21.15.44:0/4064326057 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb80aa400 msgr2=0x7f6eb80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:05.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 --2- 172.21.15.44:0/4064326057 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb80aa400 0x7f6eb80a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.829 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 -- 172.21.15.44:0/4064326057 >> v2:172.21.15.79:3300/0 conn(0x7f6eb80aa020 msgr2=0x7f6eb80a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:05.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 --2- 172.21.15.44:0/4064326057 >> v2:172.21.15.79:3300/0 conn(0x7f6eb80aa020 0x7f6eb80a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 -- 172.21.15.44:0/4064326057 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80a6a90 con 0x7f6eb813ac60 2020-11-20T12:17:05.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ebdc01700 1 --2- 172.21.15.44:0/4064326057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb813ac60 0x7f6eb80a3700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6eb80adec0 tx=0x7f6eb0009290).ready entity=mon.0 client_cookie=2d8575df73e62319 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.830 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ea7fff700 1 -- 172.21.15.44:0/4064326057 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000bbc0 con 0x7f6eb813ac60 2020-11-20T12:17:05.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ea7fff700 1 -- 172.21.15.44:0/4064326057 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb000bd20 con 0x7f6eb813ac60 2020-11-20T12:17:05.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.821+0000 7f6ea7fff700 1 -- 172.21.15.44:0/4064326057 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0011610 con 0x7f6eb813ac60 2020-11-20T12:17:05.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 -- 172.21.15.44:0/4064326057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb813ac60 msgr2=0x7f6eb80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 --2- 172.21.15.44:0/4064326057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb813ac60 0x7f6eb80a3700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f6eb80adec0 tx=0x7f6eb0009290).stop 2020-11-20T12:17:05.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 -- 172.21.15.44:0/4064326057 shutdown_connections 2020-11-20T12:17:05.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 --2- 172.21.15.44:0/4064326057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb813ac60 0x7f6eb80a3700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.832 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 --2- 172.21.15.44:0/4064326057 >> v2:172.21.15.79:3300/0 conn(0x7f6eb80aa020 0x7f6eb80a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 --2- 172.21.15.44:0/4064326057 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb80aa400 0x7f6eb80a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 -- 172.21.15.44:0/4064326057 >> 172.21.15.44:0/4064326057 conn(0x7f6eb8062690 msgr2=0x7f6eb80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:05.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 -- 172.21.15.44:0/4064326057 shutdown_connections 2020-11-20T12:17:05.833 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.822+0000 7f6ebf664700 1 -- 172.21.15.44:0/4064326057 wait complete. 2020-11-20T12:17:05.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.823+0000 7f6ebf664700 1 Processor -- start 2020-11-20T12:17:05.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.823+0000 7f6ebf664700 1 -- start start 2020-11-20T12:17:05.837 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebf664700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eb80aa020 0x7f6eb8067790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebf664700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80aa400 0x7f6eb8115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebf664700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb813ac60 0x7f6eb809e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebf664700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb809f490 con 0x7f6eb80aa400 2020-11-20T12:17:05.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebf664700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb809ed30 con 0x7f6eb80aa020 2020-11-20T12:17:05.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebf664700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb809ef50 con 0x7f6eb813ac60 2020-11-20T12:17:05.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebd400700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eb80aa020 msgr2=0x7f6eb8067790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:17:05.839 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebd400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eb80aa020 0x7f6eb8067790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:05.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebcbff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80aa400 0x7f6eb8115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebcbff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80aa400 0x7f6eb8115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41996/0 (socket says 172.21.15.44:41996) 2020-11-20T12:17:05.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebcbff700 1 -- 172.21.15.44:0/2434347079 learned_addr learned my addr 172.21.15.44:0/2434347079 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:05.840 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.824+0000 7f6ebdc01700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb813ac60 0x7f6eb809e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebcbff700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb813ac60 msgr2=0x7f6eb809e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebcbff700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb813ac60 0x7f6eb809e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.841 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebcbff700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eb80aa020 msgr2=0x7f6eb8067790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:17:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebcbff700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eb80aa020 0x7f6eb8067790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebcbff700 1 -- 172.21.15.44:0/2434347079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb809f1f0 con 0x7f6eb80aa400 2020-11-20T12:17:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebcbff700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80aa400 0x7f6eb8115db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f6ea800ebd0 tx=0x7f6ea8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.842 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800c9b0 con 0x7f6eb80aa400 2020-11-20T12:17:05.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea800cb10 con 0x7f6eb80aa400 2020-11-20T12:17:05.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb809daa0 con 0x7f6eb80aa400 2020-11-20T12:17:05.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800ce90 con 0x7f6eb80aa400 2020-11-20T12:17:05.843 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.825+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb809e030 con 0x7f6eb80aa400 2020-11-20T12:17:05.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.826+0000 7f6ebf664700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8133fb0 0x7f6eb8134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.826+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb809d800 con 0x7f6eb8133fb0 2020-11-20T12:17:05.844 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.826+0000 7f6ebd400700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8133fb0 0x7f6eb8134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.826+0000 7f6ebd400700 1 -- 172.21.15.44:0/2434347079 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6eb8061650 con 0x7f6eb8133fb0 2020-11-20T12:17:05.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.826+0000 7f6ebd400700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8133fb0 0x7f6eb8134390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6eb4009640 tx=0x7f6eb4006e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.826+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f6ea8012600 con 0x7f6eb80aa400 2020-11-20T12:17:05.845 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6e90056cd0 0x7f6e90059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f6ea80704c0 con 0x7f6eb80aa400 2020-11-20T12:17:05.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40050d0 con 0x7f6eb8133fb0 2020-11-20T12:17:05.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ebdc01700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6e90056cd0 0x7f6e90059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f6eb4005230 con 0x7f6eb8133fb0 2020-11-20T12:17:05.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8133fb0 msgr2=0x7f6eb8134390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8133fb0 0x7f6eb8134390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6eb4009640 tx=0x7f6eb4006e60).stop 2020-11-20T12:17:05.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8133fb0 msgr2=0x7f6eb8134390 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:05.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ea57fa700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8133fb0 0x7f6eb8134390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.827+0000 7f6ebdc01700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6e90056cd0 0x7f6e90059190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f6eb8149c90 tx=0x7f6eb0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.835+0000 7f6ebf664700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8134880 0x7f6eb8060170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:05.850 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.835+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80446d0 con 0x7f6eb8134880 2020-11-20T12:17:05.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.836+0000 7f6ebd400700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8134880 0x7f6eb8060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:05.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.836+0000 7f6ebd400700 1 -- 172.21.15.44:0/2434347079 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6eb80613b0 con 0x7f6eb8134880 2020-11-20T12:17:05.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.836+0000 7f6ebd400700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8134880 0x7f6eb8060170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f6eb4009640 tx=0x7f6eb4013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:05.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.836+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4014ce0 con 0x7f6eb8134880 2020-11-20T12:17:05.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4765 (secure 0 0 0) 0x7f6eb4003720 con 0x7f6eb8134880 2020-11-20T12:17:05.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8134880 msgr2=0x7f6eb8060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ea57fa700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8134880 0x7f6eb8060170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f6eb4009640 tx=0x7f6eb4013000).stop 2020-11-20T12:17:05.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ea57fa700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8134880 msgr2=0x7f6eb8060170 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ea57fa700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eb8134880 0x7f6eb8060170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6e90056cd0 msgr2=0x7f6e90059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6e90056cd0 0x7f6e90059190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f6eb8149c90 tx=0x7f6eb0016000).stop 2020-11-20T12:17:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80aa400 msgr2=0x7f6eb8115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 --2- 172.21.15.44:0/2434347079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eb80aa400 0x7f6eb8115db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f6ea800ebd0 tx=0x7f6ea8006df0).stop 2020-11-20T12:17:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebdc01700 1 -- 172.21.15.44:0/2434347079 reap_dead start 2020-11-20T12:17:05.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebdc01700 1 -- 172.21.15.44:0/2434347079 reap_dead start 2020-11-20T12:17:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 shutdown_connections 2020-11-20T12:17:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 >> 172.21.15.44:0/2434347079 conn(0x7f6eb8062690 msgr2=0x7f6eb80ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 shutdown_connections 2020-11-20T12:17:05.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:05.837+0000 7f6ebf664700 1 -- 172.21.15.44:0/2434347079 wait complete. 2020-11-20T12:17:05.856 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:05.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:05.856 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:17:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 134, 2020-11-20T12:17:05.857 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.858 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:17:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:05.859 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:05.860 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:05.861 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:05.862 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:05.863 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:05.865 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:05.866 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:05.867 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:05.868 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:05.869 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:05.878 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:05.879 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.880 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:05.881 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:05.882 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:05.883 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:05.884 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.885 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:05.886 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.887 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.888 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:17:05.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:05.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:17:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.891 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:05.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:05.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:17:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:05.894 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:17:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:05.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:05.896 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:05.896 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:05.897 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:17:08.310 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:08.319 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:25.898 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:17:25.898 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:17:25.899 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:17:25.899 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:17:25.906 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:17:25.907 INFO:tasks.mon_thrash:Sending STOP to mon a 2020-11-20T12:17:25.909 INFO:tasks.ceph.mon.a:Sent signal 19 2020-11-20T12:17:25.910 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2020-11-20T12:17:38.406 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:38.409 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:40.911 INFO:tasks.mon_thrash:Sending CONT to mon a 2020-11-20T12:17:40.911 INFO:tasks.ceph.mon.a:Sent signal 18 2020-11-20T12:17:40.912 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:17:40.912 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:17:41.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.047+0000 7f98908a7700 1 Processor -- start 2020-11-20T12:17:41.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.047+0000 7f98908a7700 1 -- start start 2020-11-20T12:17:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f98908a7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f988c13ec00 0x7f988c13ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f98908a7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c13b270 0x7f988c13b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f98908a7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c0ac640 con 0x7f988c0aacf0 2020-11-20T12:17:41.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f98908a7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c1350e0 con 0x7f988c13ec00 2020-11-20T12:17:41.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f98908a7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c13b000 con 0x7f988c13b270 2020-11-20T12:17:41.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f988b7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f988c0aacf0 0x7f988c0ab9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45760/0 (socket says 172.21.15.44:45760) 2020-11-20T12:17:41.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f988b7fe700 1 -- 172.21.15.44:0/618449649 learned_addr learned my addr 172.21.15.44:0/618449649 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:17:41.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f988affd700 1 --2- 172.21.15.44:0/618449649 >> v2:172.21.15.79:3300/0 conn(0x7f988c13ec00 0x7f988c13ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f988bfff700 1 --2- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c13b270 0x7f988c13b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f988a7fc700 1 -- 172.21.15.44:0/618449649 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430930310 0 0) 0x7f988c0ac640 con 0x7f988c0aacf0 2020-11-20T12:17:41.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.048+0000 7f988a7fc700 1 -- 172.21.15.44:0/618449649 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9878003240 con 0x7f988c0aacf0 2020-11-20T12:17:41.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988affd700 1 -- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c13b270 msgr2=0x7f988c13b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:41.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988affd700 1 --2- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c13b270 0x7f988c13b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988affd700 1 -- 172.21.15.44:0/618449649 >> v1:172.21.15.44:6789/0 conn(0x7f988c0aacf0 legacy=0x7f988c0ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988affd700 1 -- 172.21.15.44:0/618449649 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c13e260 con 0x7f988c13ec00 2020-11-20T12:17:41.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988affd700 1 --2- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13ec00 0x7f988c13ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f988000b9a0 tx=0x7f9880006df0).ready entity=mon.1 client_cookie=74c38482cd5516e2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988a7fc700 1 -- 172.21.15.44:0/618449649 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880009680 con 0x7f988c13ec00 2020-11-20T12:17:41.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988a7fc700 1 -- 172.21.15.44:0/618449649 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98800097e0 con 0x7f988c13ec00 2020-11-20T12:17:41.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f988a7fc700 1 -- 172.21.15.44:0/618449649 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880009b60 con 0x7f988c13ec00 2020-11-20T12:17:41.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f98908a7700 1 -- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13ec00 msgr2=0x7f988c13ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f98908a7700 1 --2- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13ec00 0x7f988c13ab10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f988000b9a0 tx=0x7f9880006df0).stop 2020-11-20T12:17:41.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f98908a7700 1 -- 172.21.15.44:0/618449649 shutdown_connections 2020-11-20T12:17:41.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f98908a7700 1 --2- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c13b270 0x7f988c13b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f98908a7700 1 --2- 172.21.15.44:0/618449649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13ec00 0x7f988c13ab10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.049+0000 7f98908a7700 1 -- 172.21.15.44:0/618449649 >> 172.21.15.44:0/618449649 conn(0x7f988c060170 msgr2=0x7f988c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.050+0000 7f98908a7700 1 -- 172.21.15.44:0/618449649 shutdown_connections 2020-11-20T12:17:41.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.050+0000 7f98908a7700 1 -- 172.21.15.44:0/618449649 wait complete. 2020-11-20T12:17:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 Processor -- start 2020-11-20T12:17:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 -- start start 2020-11-20T12:17:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c0aacf0 0x7f988c062b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13b270 0x7f988c0a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c13ec00 0x7f988c0a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0a5420 con 0x7f988c13ec00 2020-11-20T12:17:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0a29a0 con 0x7f988c13b270 2020-11-20T12:17:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f98908a7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0a2c40 con 0x7f988c0aacf0 2020-11-20T12:17:41.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f988bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c13ec00 0x7f988c0a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f988bfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c13ec00 0x7f988c0a30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42022/0 (socket says 172.21.15.44:42022) 2020-11-20T12:17:41.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.051+0000 7f988bfff700 1 -- 172.21.15.44:0/221021075 learned_addr learned my addr 172.21.15.44:0/221021075 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:41.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988affd700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13b270 0x7f988c0a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988b7fe700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c0aacf0 0x7f988c062b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988bfff700 1 -- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c0aacf0 msgr2=0x7f988c062b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988bfff700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c0aacf0 0x7f988c062b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988bfff700 1 -- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13b270 msgr2=0x7f988c0a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988bfff700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13b270 0x7f988c0a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988bfff700 1 -- 172.21.15.44:0/221021075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c1389b0 con 0x7f988c13ec00 2020-11-20T12:17:41.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f988bfff700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c13ec00 0x7f988c0a30e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f988400ef50 tx=0x7f98840091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f986b7fe700 1 -- 172.21.15.44:0/221021075 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988400cb40 con 0x7f988c13ec00 2020-11-20T12:17:41.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f986b7fe700 1 -- 172.21.15.44:0/221021075 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988400cca0 con 0x7f988c13ec00 2020-11-20T12:17:41.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c0a50b0 con 0x7f988c13ec00 2020-11-20T12:17:41.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f986b7fe700 1 -- 172.21.15.44:0/221021075 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840123f0 con 0x7f988c13ec00 2020-11-20T12:17:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.052+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c13a7e0 con 0x7f988c13ec00 2020-11-20T12:17:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.053+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c0a2e00 con 0x7f988c13ec00 2020-11-20T12:17:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.053+0000 7f986b7fe700 1 -- 172.21.15.44:0/221021075 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9884022070 con 0x7f988c13ec00 2020-11-20T12:17:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.054+0000 7f986b7fe700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9864056d20 0x7f98640591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.054+0000 7f986b7fe700 1 -- 172.21.15.44:0/221021075 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9884070900 con 0x7f988c13ec00 2020-11-20T12:17:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.054+0000 7f988b7fe700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9864056d20 0x7f98640591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.055+0000 7f988b7fe700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9864056d20 0x7f98640591e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f987c0055a0 tx=0x7f987c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.058+0000 7f986b7fe700 1 -- 172.21.15.44:0/221021075 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9884045bd0 con 0x7f988c13ec00 2020-11-20T12:17:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.332+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f988c057d80 con 0x7f988c13ec00 2020-11-20T12:17:41.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.332+0000 7f986b7fe700 1 -- 172.21.15.44:0/221021075 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f9884016020 con 0x7f988c13ec00 2020-11-20T12:17:41.336 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:17:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9864056d20 msgr2=0x7f98640591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9864056d20 0x7f98640591e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f987c0055a0 tx=0x7f987c009040).stop 2020-11-20T12:17:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c13ec00 msgr2=0x7f988c0a30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c13ec00 0x7f988c0a30e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f988400ef50 tx=0x7f98840091d0).stop 2020-11-20T12:17:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 shutdown_connections 2020-11-20T12:17:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9864056d20 0x7f98640591e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f988c13ec00 0x7f988c0a30e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f988c0aacf0 0x7f988c062b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 --2- 172.21.15.44:0/221021075 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f988c13b270 0x7f988c0a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 >> 172.21.15.44:0/221021075 conn(0x7f988c060170 msgr2=0x7f988c13d010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 shutdown_connections 2020-11-20T12:17:41.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.334+0000 7f98908a7700 1 -- 172.21.15.44:0/221021075 wait complete. 2020-11-20T12:17:41.349 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:17:41.350 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:17:41.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.489+0000 7f131cafe700 1 Processor -- start 2020-11-20T12:17:41.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.489+0000 7f131cafe700 1 -- start start 2020-11-20T12:17:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.490+0000 7f131cafe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1318068f60 0x7f13180a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.490+0000 7f131cafe700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180a8870 con 0x7f131813e040 2020-11-20T12:17:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.490+0000 7f131cafe700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180a9f00 con 0x7f1318060880 2020-11-20T12:17:41.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.490+0000 7f131cafe700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13180a8aa0 con 0x7f1318068f60 2020-11-20T12:17:41.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.490+0000 7f131659c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1318068f60 0x7f13180a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.490+0000 7f130ffff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f131813e040 0x7f1318060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45778/0 (socket says 172.21.15.44:45778) 2020-11-20T12:17:41.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.490+0000 7f130ffff700 1 -- 172.21.15.44:0/2715005596 learned_addr learned my addr 172.21.15.44:0/2715005596 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:17:41.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131659c700 1 -- 172.21.15.44:0/2715005596 >> v1:172.21.15.79:6789/0 conn(0x7f1318060880 legacy=0x7f1318145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131659c700 1 -- 172.21.15.44:0/2715005596 >> v1:172.21.15.44:6789/0 conn(0x7f131813e040 legacy=0x7f1318060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131659c700 1 -- 172.21.15.44:0/2715005596 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180a8d40 con 0x7f1318068f60 2020-11-20T12:17:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f1315d9b700 1 -- 172.21.15.44:0/2715005596 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799073164 0 0) 0x7f13180a8870 con 0x7f131813e040 2020-11-20T12:17:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131659c700 1 --2- 172.21.15.44:0/2715005596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318068f60 0x7f13180a91e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f130800b9a0 tx=0x7f1308006df0).ready entity=mon.0 client_cookie=686c895639a86837 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f1315d9b700 1 -- 172.21.15.44:0/2715005596 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308009840 con 0x7f1318068f60 2020-11-20T12:17:41.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f1315d9b700 1 -- 172.21.15.44:0/2715005596 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13080099a0 con 0x7f1318068f60 2020-11-20T12:17:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f1315d9b700 1 -- 172.21.15.44:0/2715005596 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308009d20 con 0x7f1318068f60 2020-11-20T12:17:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131cafe700 1 -- 172.21.15.44:0/2715005596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318068f60 msgr2=0x7f13180a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131cafe700 1 --2- 172.21.15.44:0/2715005596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318068f60 0x7f13180a91e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f130800b9a0 tx=0x7f1308006df0).stop 2020-11-20T12:17:41.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131cafe700 1 -- 172.21.15.44:0/2715005596 shutdown_connections 2020-11-20T12:17:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131cafe700 1 --2- 172.21.15.44:0/2715005596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318068f60 0x7f13180a91e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131cafe700 1 -- 172.21.15.44:0/2715005596 >> 172.21.15.44:0/2715005596 conn(0x7f13180ad6f0 msgr2=0x7f13180adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.491+0000 7f131cafe700 1 -- 172.21.15.44:0/2715005596 shutdown_connections 2020-11-20T12:17:41.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.492+0000 7f131cafe700 1 -- 172.21.15.44:0/2715005596 wait complete. 2020-11-20T12:17:41.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.492+0000 7f131cafe700 1 Processor -- start 2020-11-20T12:17:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131cafe700 1 -- start start 2020-11-20T12:17:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131cafe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1318060880 0x7f131814dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131cafe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1318068f60 0x7f13180ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131cafe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131813e040 0x7f13180a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131cafe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180acec0 con 0x7f131813e040 2020-11-20T12:17:41.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131cafe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131814d400 con 0x7f1318068f60 2020-11-20T12:17:41.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131cafe700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131814d660 con 0x7f1318060880 2020-11-20T12:17:41.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f1316d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131813e040 0x7f13180a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f1316d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131813e040 0x7f13180a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42036/0 (socket says 172.21.15.44:42036) 2020-11-20T12:17:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f1316d9d700 1 -- 172.21.15.44:0/891687176 learned_addr learned my addr 172.21.15.44:0/891687176 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f131659c700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1318060880 0x7f131814dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.493+0000 7f130ffff700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1318068f60 0x7f13180ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f1316d9d700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1318060880 msgr2=0x7f131814dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f1316d9d700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1318060880 0x7f131814dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f1316d9d700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1318068f60 msgr2=0x7f13180ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f1316d9d700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1318068f60 0x7f13180ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f1316d9d700 1 -- 172.21.15.44:0/891687176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131814d900 con 0x7f131813e040 2020-11-20T12:17:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f131659c700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f1318060880 0x7f131814dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f1316d9d700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131813e040 0x7f13180a28b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f131000ca60 tx=0x7f1310007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131000a800 con 0x7f131813e040 2020-11-20T12:17:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131000a960 con 0x7f131813e040 2020-11-20T12:17:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131000ace0 con 0x7f131813e040 2020-11-20T12:17:41.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13180a2da0 con 0x7f131813e040 2020-11-20T12:17:41.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.494+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318149cd0 con 0x7f131813e040 2020-11-20T12:17:41.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.495+0000 7f131cafe700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318149fb0 0x7f131814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.495+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131814d210 con 0x7f1318149fb0 2020-11-20T12:17:41.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.495+0000 7f131659c700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318149fb0 0x7f131814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.495+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f1310019030 con 0x7f131813e040 2020-11-20T12:17:41.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.495+0000 7f131659c700 1 -- 172.21.15.44:0/891687176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f131814da90 con 0x7f1318149fb0 2020-11-20T12:17:41.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f12f4056d20 0x7f12f40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f131006f920 con 0x7f131813e040 2020-11-20T12:17:41.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f131659c700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318149fb0 0x7f131814a390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f130800b9a0 tx=0x7f1308015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308011e00 con 0x7f1318149fb0 2020-11-20T12:17:41.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130ffff700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f12f4056d20 0x7f12f40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f130801f650 con 0x7f1318149fb0 2020-11-20T12:17:41.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130ffff700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f12f4056d20 0x7f12f40591e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f1318060720 tx=0x7f1300007c00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318149fb0 msgr2=0x7f131814a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318149fb0 0x7f131814a390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f130800b9a0 tx=0x7f1308015000).stop 2020-11-20T12:17:41.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318149fb0 msgr2=0x7f131814a390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:41.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.496+0000 7f130e7fc700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1318149fb0 0x7f131814a390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.505+0000 7f131cafe700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131814a880 0x7f13180a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.505+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318063f00 con 0x7f131814a880 2020-11-20T12:17:41.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.505+0000 7f131659c700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131814a880 0x7f13180a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f131659c700 1 -- 172.21.15.44:0/891687176 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f13180446d0 con 0x7f131814a880 2020-11-20T12:17:41.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f131659c700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131814a880 0x7f13180a4ac0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f1308006e30 tx=0x7f1308020000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308028660 con 0x7f131814a880 2020-11-20T12:17:41.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4608 (secure 0 0 0) 0x7f130802ee90 con 0x7f131814a880 2020-11-20T12:17:41.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131814a880 msgr2=0x7f13180a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f130e7fc700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131814a880 0x7f13180a4ac0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f1308006e30 tx=0x7f1308020000).stop 2020-11-20T12:17:41.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f130e7fc700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131814a880 msgr2=0x7f13180a4ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:41.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f130e7fc700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131814a880 0x7f13180a4ac0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f12f4056d20 msgr2=0x7f12f40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f131cafe700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f12f4056d20 0x7f12f40591e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f1318060720 tx=0x7f1300007c00).stop 2020-11-20T12:17:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.506+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131813e040 msgr2=0x7f13180a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.507+0000 7f131cafe700 1 --2- 172.21.15.44:0/891687176 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f131813e040 0x7f13180a28b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f131000ca60 tx=0x7f1310007f40).stop 2020-11-20T12:17:41.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.507+0000 7f1316d9d700 1 -- 172.21.15.44:0/891687176 reap_dead start 2020-11-20T12:17:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.507+0000 7f1316d9d700 1 -- 172.21.15.44:0/891687176 reap_dead start 2020-11-20T12:17:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.507+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 shutdown_connections 2020-11-20T12:17:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.507+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 >> 172.21.15.44:0/891687176 conn(0x7f13180ad6f0 msgr2=0x7f13180aa090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.507+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 shutdown_connections 2020-11-20T12:17:41.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.507+0000 7f131cafe700 1 -- 172.21.15.44:0/891687176 wait complete. 2020-11-20T12:17:41.518 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:41.518 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:41.518 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:41.518 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:17:41.518 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 142, 2020-11-20T12:17:41.519 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:41.519 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:41.519 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:17:41.519 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:41.520 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.520 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:17:41.520 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:41.520 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:41.520 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:41.521 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.521 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.521 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.521 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.521 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.522 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.522 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.522 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.522 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.522 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:41.523 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:41.523 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.523 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.523 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.523 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.524 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.524 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.524 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.524 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.524 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.525 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:41.525 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:41.525 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:41.525 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:41.526 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:41.526 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:41.526 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:41.526 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:41.526 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:41.527 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:41.527 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:41.527 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:41.527 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:41.527 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:41.528 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:41.528 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.528 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.528 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.528 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.529 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.529 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.529 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.529 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.530 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.530 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:41.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.530 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:41.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.530 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:41.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:41.531 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.531 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.532 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:41.532 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.533 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:41.533 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.533 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.533 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.534 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:41.534 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:41.534 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.534 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.535 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.535 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:41.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:41.536 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.536 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.536 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.537 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:41.537 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.538 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:41.538 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.538 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.539 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:41.539 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:41.539 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.539 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.540 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.540 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:41.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:41.541 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.541 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.541 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.542 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:41.542 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.543 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:41.543 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.543 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.543 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:41.544 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:41.544 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.544 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.544 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.545 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.545 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:41.545 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:41.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.546 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.546 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.546 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:41.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.546 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.547 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:17:41.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.547 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.547 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.547 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:41.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.548 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.548 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:17:41.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.548 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.549 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.549 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:41.549 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.549 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:41.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.550 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.550 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.550 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:17:41.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.551 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.551 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:41.551 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:41.553 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:17:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.701+0000 7f95c8ef3700 1 Processor -- start 2020-11-20T12:17:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.701+0000 7f95c8ef3700 1 -- start start 2020-11-20T12:17:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c8ef3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f95c4061340 0x7f95c4066a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c8ef3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f95c4066f80 0x7f95c413a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c8ef3700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c40a9350 con 0x7f95c413acb0 2020-11-20T12:17:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c8ef3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c40a95f0 con 0x7f95c4061340 2020-11-20T12:17:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c8ef3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c40a9890 con 0x7f95c4066f80 2020-11-20T12:17:41.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c37fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f95c4061340 0x7f95c4066a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c3fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f95c413acb0 0x7f95c413dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45796/0 (socket says 172.21.15.44:45796) 2020-11-20T12:17:41.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c3fff700 1 -- 172.21.15.44:0/343649283 learned_addr learned my addr 172.21.15.44:0/343649283 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:17:41.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c2ffd700 1 --2- 172.21.15.44:0/343649283 >> v2:172.21.15.79:3300/0 conn(0x7f95c4066f80 0x7f95c413a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c37fe700 1 -- 172.21.15.44:0/343649283 >> v2:172.21.15.79:3300/0 conn(0x7f95c4066f80 msgr2=0x7f95c413a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:41.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c37fe700 1 --2- 172.21.15.44:0/343649283 >> v2:172.21.15.79:3300/0 conn(0x7f95c4066f80 0x7f95c413a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c37fe700 1 -- 172.21.15.44:0/343649283 >> v1:172.21.15.44:6789/0 conn(0x7f95c413acb0 legacy=0x7f95c413dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c27fc700 1 -- 172.21.15.44:0/343649283 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528458111 0 0) 0x7f95c40a9350 con 0x7f95c413acb0 2020-11-20T12:17:41.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c37fe700 1 -- 172.21.15.44:0/343649283 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4061720 con 0x7f95c4061340 2020-11-20T12:17:41.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c2ffd700 1 --2- 172.21.15.44:0/343649283 >> v2:172.21.15.79:3300/0 conn(0x7f95c4066f80 0x7f95c413a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:41.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.702+0000 7f95c37fe700 1 --2- 172.21.15.44:0/343649283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c4061340 0x7f95c4066a90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f95c40662f0 tx=0x7f95b4006df0).ready entity=mon.0 client_cookie=534457650a7b4e34 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c27fc700 1 -- 172.21.15.44:0/343649283 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b40097b0 con 0x7f95c4061340 2020-11-20T12:17:41.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c27fc700 1 -- 172.21.15.44:0/343649283 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b4009910 con 0x7f95c4061340 2020-11-20T12:17:41.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c27fc700 1 -- 172.21.15.44:0/343649283 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4009c90 con 0x7f95c4061340 2020-11-20T12:17:41.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 -- 172.21.15.44:0/343649283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c4061340 msgr2=0x7f95c4066a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/343649283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c4061340 0x7f95c4066a90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f95c40662f0 tx=0x7f95b4006df0).stop 2020-11-20T12:17:41.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 -- 172.21.15.44:0/343649283 shutdown_connections 2020-11-20T12:17:41.720 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/343649283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c4061340 0x7f95c4066a90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/343649283 >> v2:172.21.15.79:3300/0 conn(0x7f95c4066f80 0x7f95c413a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 -- 172.21.15.44:0/343649283 >> 172.21.15.44:0/343649283 conn(0x7f95c40a9fa0 msgr2=0x7f95c40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 -- 172.21.15.44:0/343649283 shutdown_connections 2020-11-20T12:17:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.703+0000 7f95c8ef3700 1 -- 172.21.15.44:0/343649283 wait complete. 2020-11-20T12:17:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.704+0000 7f95c8ef3700 1 Processor -- start 2020-11-20T12:17:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.704+0000 7f95c8ef3700 1 -- start start 2020-11-20T12:17:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c8ef3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4061340 0x7f95c4135ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c8ef3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f95c4066f80 0x7f95c414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c8ef3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c413acb0 0x7f95c4132a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c8ef3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c413c7f0 con 0x7f95c413acb0 2020-11-20T12:17:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c8ef3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c413df90 con 0x7f95c4061340 2020-11-20T12:17:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c8ef3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c413dcc0 con 0x7f95c4066f80 2020-11-20T12:17:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c413acb0 0x7f95c4132a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c413acb0 0x7f95c4132a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42054/0 (socket says 172.21.15.44:42054) 2020-11-20T12:17:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c3fff700 1 -- 172.21.15.44:0/1486265945 learned_addr learned my addr 172.21.15.44:0/1486265945 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4061340 0x7f95c4135ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.705+0000 7f95c2ffd700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f95c4066f80 0x7f95c414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c3fff700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f95c4066f80 msgr2=0x7f95c414d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c3fff700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f95c4066f80 0x7f95c414d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c3fff700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4061340 msgr2=0x7f95c4135ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c3fff700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4061340 0x7f95c4135ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c3fff700 1 -- 172.21.15.44:0/1486265945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c413e430 con 0x7f95c413acb0 2020-11-20T12:17:41.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c3fff700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c413acb0 0x7f95c4132a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f95bc00cae0 tx=0x7f95bc007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc005b80 con 0x7f95c413acb0 2020-11-20T12:17:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc005ce0 con 0x7f95c413acb0 2020-11-20T12:17:41.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c41364e0 con 0x7f95c413acb0 2020-11-20T12:17:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc007340 con 0x7f95c413acb0 2020-11-20T12:17:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.706+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c413c5c0 con 0x7f95c413acb0 2020-11-20T12:17:41.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.707+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4132f00 0x7f95c413bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.707+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4069630 con 0x7f95c4132f00 2020-11-20T12:17:41.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.707+0000 7f95c37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4132f00 0x7f95c413bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.707+0000 7f95c37fe700 1 -- 172.21.15.44:0/1486265945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95c413c0b0 con 0x7f95c4132f00 2020-11-20T12:17:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.707+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f95bc010030 con 0x7f95c413acb0 2020-11-20T12:17:41.732 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.707+0000 7f95c37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4132f00 0x7f95c413bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f95c40662f0 tx=0x7f95b4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f959c056d20 0x7f959c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f95bc06fdc0 con 0x7f95c413acb0 2020-11-20T12:17:41.733 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4011dd0 con 0x7f95c4132f00 2020-11-20T12:17:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95c2ffd700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f959c056d20 0x7f959c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f95b401f440 con 0x7f95c4132f00 2020-11-20T12:17:41.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4132f00 msgr2=0x7f95c413bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4132f00 0x7f95c413bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f95c40662f0 tx=0x7f95b4015000).stop 2020-11-20T12:17:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4132f00 msgr2=0x7f95c413bbc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95a37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4132f00 0x7f95c413bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.708+0000 7f95c2ffd700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f959c056d20 0x7f959c0591e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f95c4135790 tx=0x7f95b8008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.716+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4139c60 0x7f95c413a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.716+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c413e150 con 0x7f95c4139c60 2020-11-20T12:17:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.716+0000 7f95c37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4139c60 0x7f95c413a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.716+0000 7f95c37fe700 1 -- 172.21.15.44:0/1486265945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95c4057d80 con 0x7f95c4139c60 2020-11-20T12:17:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.716+0000 7f95c37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4139c60 0x7f95c413a070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f95b4006e30 tx=0x7f95b4021070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4029ba0 con 0x7f95c4139c60 2020-11-20T12:17:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f95b4011bc0 con 0x7f95c4139c60 2020-11-20T12:17:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4139c60 msgr2=0x7f95c413a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95a37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4139c60 0x7f95c413a070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f95b4006e30 tx=0x7f95b4021070).stop 2020-11-20T12:17:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95a37fe700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4139c60 msgr2=0x7f95c413a070 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95a37fe700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95c4139c60 0x7f95c413a070 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f959c056d20 msgr2=0x7f959c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f959c056d20 0x7f959c0591e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f95c4135790 tx=0x7f95b8008040).stop 2020-11-20T12:17:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c413acb0 msgr2=0x7f95c4132a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c413acb0 0x7f95c4132a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f95bc00cae0 tx=0x7f95bc007f70).stop 2020-11-20T12:17:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c3fff700 1 -- 172.21.15.44:0/1486265945 reap_dead start 2020-11-20T12:17:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 shutdown_connections 2020-11-20T12:17:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 --2- 172.21.15.44:0/1486265945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95c413acb0 0x7f95c4132a10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 >> 172.21.15.44:0/1486265945 conn(0x7f95c40a9fa0 msgr2=0x7f95c40accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 shutdown_connections 2020-11-20T12:17:41.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.717+0000 7f95c8ef3700 1 -- 172.21.15.44:0/1486265945 wait complete. 2020-11-20T12:17:41.748 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:41.748 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:41.748 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:41.749 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:17:41.749 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 142, 2020-11-20T12:17:41.749 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:41.750 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:41.751 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:17:41.751 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:41.751 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.752 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:17:41.752 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:41.752 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:41.753 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:41.753 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.753 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.754 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.754 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.755 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.755 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.755 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.756 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.756 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:41.756 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:41.757 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.757 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.758 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.758 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.759 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.759 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.759 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.760 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.760 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:41.761 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:41.761 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:41.761 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:41.762 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:41.762 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:41.762 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:41.763 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:41.763 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:41.763 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:41.764 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:41.764 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:41.764 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:41.765 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:41.765 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:41.766 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.766 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.766 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.767 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.767 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.767 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.768 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.768 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.769 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:41.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.769 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:41.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.770 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:41.770 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:41.771 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.771 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.772 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:41.772 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.774 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:41.774 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.774 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.775 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.776 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:41.776 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:41.776 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.777 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.777 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:41.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:41.779 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.779 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.780 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:41.780 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.781 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:41.782 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.782 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.782 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.783 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:41.784 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:41.784 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.784 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.784 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.786 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:41.786 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:41.786 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:41.788 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.789 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:41.790 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.790 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.791 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:41.791 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:41.792 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.792 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.792 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.793 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.794 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:41.794 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:41.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.795 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.795 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.795 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:41.796 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.796 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.796 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:41.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.797 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.797 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.798 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:41.798 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.799 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.799 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:41.800 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:41.800 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:17:41.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.937+0000 7f90de7e8700 1 Processor -- start 2020-11-20T12:17:41.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90de7e8700 1 -- start start 2020-11-20T12:17:41.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90de7e8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f90d80a74e0 0x7f90d80a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90de7e8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d80a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90de7e8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d80a6bc0 con 0x7f90d80a95e0 2020-11-20T12:17:41.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90de7e8700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d8136d90 con 0x7f90d80a74e0 2020-11-20T12:17:41.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90de7e8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80a8c50 con 0x7f90d80a4e30 2020-11-20T12:17:41.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90dd7e6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f90d80a74e0 0x7f90d80a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90dd7e6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f90d80a74e0 0x7f90d80a6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42062/0 (socket says 172.21.15.44:42062) 2020-11-20T12:17:41.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.938+0000 7f90dd7e6700 1 -- 172.21.15.44:0/1190382965 learned_addr learned my addr 172.21.15.44:0/1190382965 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:41.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90dcfe5700 1 --2- 172.21.15.44:0/1190382965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d80a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90dd7e6700 1 -- 172.21.15.44:0/1190382965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 msgr2=0x7f90d80a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:41.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/1190382965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d80a90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90dd7e6700 1 -- 172.21.15.44:0/1190382965 >> v1:172.21.15.79:6789/0 conn(0x7f90d80a95e0 legacy=0x7f90d8133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90dd7e6700 1 -- 172.21.15.44:0/1190382965 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d80a78c0 con 0x7f90d80a74e0 2020-11-20T12:17:41.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/1190382965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d80a6520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f90d4006fd0 tx=0x7f90d4006eb0).ready entity=mon.0 client_cookie=e3755f02cf8f79ac server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90c7fff700 1 -- 172.21.15.44:0/1190382965 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4009810 con 0x7f90d80a74e0 2020-11-20T12:17:41.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90c7fff700 1 -- 172.21.15.44:0/1190382965 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4009970 con 0x7f90d80a74e0 2020-11-20T12:17:41.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.939+0000 7f90c7fff700 1 -- 172.21.15.44:0/1190382965 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4009cf0 con 0x7f90d80a74e0 2020-11-20T12:17:41.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 -- 172.21.15.44:0/1190382965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 msgr2=0x7f90d80a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 --2- 172.21.15.44:0/1190382965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d80a6520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f90d4006fd0 tx=0x7f90d4006eb0).stop 2020-11-20T12:17:41.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 -- 172.21.15.44:0/1190382965 shutdown_connections 2020-11-20T12:17:41.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 --2- 172.21.15.44:0/1190382965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d80a6520 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 --2- 172.21.15.44:0/1190382965 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d80a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 -- 172.21.15.44:0/1190382965 >> 172.21.15.44:0/1190382965 conn(0x7f90d80ad520 msgr2=0x7f90d80ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 -- 172.21.15.44:0/1190382965 shutdown_connections 2020-11-20T12:17:41.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.940+0000 7f90de7e8700 1 -- 172.21.15.44:0/1190382965 wait complete. 2020-11-20T12:17:41.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.941+0000 7f90de7e8700 1 Processor -- start 2020-11-20T12:17:41.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.941+0000 7f90de7e8700 1 -- start start 2020-11-20T12:17:41.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90de7e8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d8135750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90de7e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d8138510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90de7e8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90d80a95e0 0x7f90d8138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90de7e8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8138ef0 con 0x7f90d80a74e0 2020-11-20T12:17:41.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90de7e8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8139580 con 0x7f90d80a95e0 2020-11-20T12:17:41.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90de7e8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8139820 con 0x7f90d80a4e30 2020-11-20T12:17:41.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d8138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d8138510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42070/0 (socket says 172.21.15.44:42070) 2020-11-20T12:17:41.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 -- 172.21.15.44:0/484965066 learned_addr learned my addr 172.21.15.44:0/484965066 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:41.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90ddfe7700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90d80a95e0 0x7f90d8138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d8135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 msgr2=0x7f90d8135750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d8135750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90d80a95e0 msgr2=0x7f90d8138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90d80a95e0 0x7f90d8138a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 -- 172.21.15.44:0/484965066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d8139200 con 0x7f90d80a74e0 2020-11-20T12:17:41.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d80a4e30 0x7f90d8135750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:41.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.942+0000 7f90dcfe5700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d8138510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f90c800eba0 tx=0x7f90c8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.943+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800ca60 con 0x7f90d80a74e0 2020-11-20T12:17:41.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.943+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c800cbc0 con 0x7f90d80a74e0 2020-11-20T12:17:41.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.943+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80123f0 con 0x7f90d80a74e0 2020-11-20T12:17:41.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.943+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d8133560 con 0x7f90d80a74e0 2020-11-20T12:17:41.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.943+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d813aff0 con 0x7f90d80a74e0 2020-11-20T12:17:41.957 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90de7e8700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125270 0x7f90d8125650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d805cf40 con 0x7f90d8125270 2020-11-20T12:17:41.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125270 0x7f90d8125650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.958 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f90c8012760 con 0x7f90d80a74e0 2020-11-20T12:17:41.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90dd7e6700 1 -- 172.21.15.44:0/484965066 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90d81399e0 con 0x7f90d8125270 2020-11-20T12:17:41.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125270 0x7f90d8125650 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f90d400beb0 tx=0x7f90d4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90c57fa700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f90b0056d50 0x7f90b0059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f90c80705c0 con 0x7f90d80a74e0 2020-11-20T12:17:41.959 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4011d50 con 0x7f90d8125270 2020-11-20T12:17:41.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.944+0000 7f90ddfe7700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f90b0056d50 0x7f90b0059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.945+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f90d401e430 con 0x7f90d8125270 2020-11-20T12:17:41.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.945+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125270 msgr2=0x7f90d8125650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.945+0000 7f90c57fa700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125270 0x7f90d8125650 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f90d400beb0 tx=0x7f90d4015000).stop 2020-11-20T12:17:41.960 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.945+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125270 msgr2=0x7f90d8125650 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.945+0000 7f90c57fa700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125270 0x7f90d8125650 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.961 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.945+0000 7f90ddfe7700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f90b0056d50 0x7f90b0059210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f90d813a8c0 tx=0x7f90d0009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.953+0000 7f90de7e8700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125b40 0x7f90d80ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.953+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d813b240 con 0x7f90d8125b40 2020-11-20T12:17:41.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.953+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125b40 0x7f90d80ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.953+0000 7f90dd7e6700 1 -- 172.21.15.44:0/484965066 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f90d8057d80 con 0x7f90d8125b40 2020-11-20T12:17:41.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.953+0000 7f90dd7e6700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125b40 0x7f90d80ac300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f90d400b1b0 tx=0x7f90d401f000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d402db30 con 0x7f90d8125b40 2020-11-20T12:17:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f90d4027450 con 0x7f90d8125b40 2020-11-20T12:17:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125b40 msgr2=0x7f90d80ac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90c57fa700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125b40 0x7f90d80ac300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f90d400b1b0 tx=0x7f90d401f000).stop 2020-11-20T12:17:41.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90c57fa700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125b40 msgr2=0x7f90d80ac300 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90c57fa700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f90d8125b40 0x7f90d80ac300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f90b0056d50 msgr2=0x7f90b0059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f90b0056d50 0x7f90b0059210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f90d813a8c0 tx=0x7f90d0009040).stop 2020-11-20T12:17:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 msgr2=0x7f90d8138510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 --2- 172.21.15.44:0/484965066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90d80a74e0 0x7f90d8138510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f90c800eba0 tx=0x7f90c8006df0).stop 2020-11-20T12:17:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90ddfe7700 1 -- 172.21.15.44:0/484965066 reap_dead start 2020-11-20T12:17:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90ddfe7700 1 -- 172.21.15.44:0/484965066 reap_dead start 2020-11-20T12:17:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 shutdown_connections 2020-11-20T12:17:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 >> 172.21.15.44:0/484965066 conn(0x7f90d80ad520 msgr2=0x7f90d8134480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 shutdown_connections 2020-11-20T12:17:41.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:41.954+0000 7f90de7e8700 1 -- 172.21.15.44:0/484965066 wait complete. 2020-11-20T12:17:41.967 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:41.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:41.967 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:41.968 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:17:41.968 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 142, 2020-11-20T12:17:41.968 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:41.968 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:41.968 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:17:41.969 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:41.969 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.969 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:17:41.969 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:41.969 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:41.970 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:41.970 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.970 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.970 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.970 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.971 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.971 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.971 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.971 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.971 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.972 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:41.972 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:41.972 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.972 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.972 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.973 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.973 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.973 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.973 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.973 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.974 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.974 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:41.974 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:41.974 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:41.975 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:41.975 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:41.975 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:41.975 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:41.975 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:41.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:41.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:41.976 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:41.976 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:41.976 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:41.976 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:41.977 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:41.977 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:41.977 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:41.977 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:41.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:41.978 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:41.978 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:41.978 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:41.979 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:41.979 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.979 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:41.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.979 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:41.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.980 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:41.980 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:41.980 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.980 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.981 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:41.981 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.982 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:41.982 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.983 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.983 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:41.983 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:41.983 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.984 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.984 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.984 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:41.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:41.985 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.985 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.986 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:41.986 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.987 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:41.988 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.988 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.989 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:41.989 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:41.989 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.989 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.990 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.990 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:41.990 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:41.991 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:41.991 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:41.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:41.991 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:41.992 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:41.992 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:41.992 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:41.993 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.993 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.993 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:41.993 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:41.994 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:41.994 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:41.994 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:41.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.994 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:41.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:41.995 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:41.995 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:41.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.995 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.996 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.996 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:41.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.996 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.996 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:17:41.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.997 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.997 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.997 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:41.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.998 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.998 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:17:41.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:41.998 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:41.998 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:41.999 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:41.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:41.999 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:41.999 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:41.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:42.000 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:42.000 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:42.000 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:42.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:42.000 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:42.001 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:17:42.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:42.001 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:42.001 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:42.001 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:17:42.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:42.002 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:42.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:42.002 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:42.002 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:42.004 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:17:42.004 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:17:42.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.137+0000 7f742326e700 1 Processor -- start 2020-11-20T12:17:42.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.137+0000 7f742326e700 1 -- start start 2020-11-20T12:17:42.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742326e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa400 0x7f741c0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742326e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f741c13ac60 0x7f741c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742326e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c0a9770 con 0x7f741c0aa020 2020-11-20T12:17:42.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742326e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c0a56c0 con 0x7f741c13ac60 2020-11-20T12:17:42.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742326e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0a6da0 con 0x7f741c0aa400 2020-11-20T12:17:42.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742180b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f741c13ac60 0x7f741c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742180b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f741c13ac60 0x7f741c0a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42084/0 (socket says 172.21.15.44:42084) 2020-11-20T12:17:42.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742180b700 1 -- 172.21.15.44:0/1243575395 learned_addr learned my addr 172.21.15.44:0/1243575395 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:42.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f7420809700 1 --2- 172.21.15.44:0/1243575395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa400 0x7f741c0a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.138+0000 7f742180b700 1 -- 172.21.15.44:0/1243575395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa400 msgr2=0x7f741c0a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:42.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742180b700 1 --2- 172.21.15.44:0/1243575395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa400 0x7f741c0a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742180b700 1 -- 172.21.15.44:0/1243575395 >> v1:172.21.15.79:6789/0 conn(0x7f741c0aa020 legacy=0x7f741c0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742180b700 1 -- 172.21.15.44:0/1243575395 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0a7040 con 0x7f741c13ac60 2020-11-20T12:17:42.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742180b700 1 --2- 172.21.15.44:0/1243575395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c13ac60 0x7f741c0a3700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f7414009d50 tx=0x7f7414009290).ready entity=mon.0 client_cookie=2678b97b65d03950 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f740bfff700 1 -- 172.21.15.44:0/1243575395 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741400bbc0 con 0x7f741c13ac60 2020-11-20T12:17:42.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f740bfff700 1 -- 172.21.15.44:0/1243575395 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741400bd20 con 0x7f741c13ac60 2020-11-20T12:17:42.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f740bfff700 1 -- 172.21.15.44:0/1243575395 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414011610 con 0x7f741c13ac60 2020-11-20T12:17:42.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 -- 172.21.15.44:0/1243575395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c13ac60 msgr2=0x7f741c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 --2- 172.21.15.44:0/1243575395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c13ac60 0x7f741c0a3700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f7414009d50 tx=0x7f7414009290).stop 2020-11-20T12:17:42.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 -- 172.21.15.44:0/1243575395 shutdown_connections 2020-11-20T12:17:42.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 --2- 172.21.15.44:0/1243575395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c13ac60 0x7f741c0a3700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 --2- 172.21.15.44:0/1243575395 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa400 0x7f741c0a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 -- 172.21.15.44:0/1243575395 >> 172.21.15.44:0/1243575395 conn(0x7f741c062690 msgr2=0x7f741c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:42.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 -- 172.21.15.44:0/1243575395 shutdown_connections 2020-11-20T12:17:42.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.139+0000 7f742326e700 1 -- 172.21.15.44:0/1243575395 wait complete. 2020-11-20T12:17:42.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 Processor -- start 2020-11-20T12:17:42.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 -- start start 2020-11-20T12:17:42.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa020 0x7f741c067e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c0aa400 0x7f741c139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f741c13ac60 0x7f741c0aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0aca00 con 0x7f741c0aa400 2020-11-20T12:17:42.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0ac020 con 0x7f741c13ac60 2020-11-20T12:17:42.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f742326e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0ac2c0 con 0x7f741c0aa020 2020-11-20T12:17:42.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.141+0000 7f7420809700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c0aa400 0x7f741c139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c0aa400 0x7f741c139510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42088/0 (socket says 172.21.15.44:42088) 2020-11-20T12:17:42.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 -- 172.21.15.44:0/3080170590 learned_addr learned my addr 172.21.15.44:0/3080170590 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:42.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f742100a700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa020 0x7f741c067e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f742180b700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f741c13ac60 0x7f741c0aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 -- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa020 msgr2=0x7f741c067e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa020 0x7f741c067e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 -- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f741c13ac60 msgr2=0x7f741c0aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f741c13ac60 0x7f741c0aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 -- 172.21.15.44:0/3080170590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0ac560 con 0x7f741c0aa400 2020-11-20T12:17:42.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f7420809700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c0aa400 0x7f741c139510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f740c00eba0 tx=0x7f740c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f74097fa700 1 -- 172.21.15.44:0/3080170590 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00c8e0 con 0x7f741c0aa400 2020-11-20T12:17:42.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f74097fa700 1 -- 172.21.15.44:0/3080170590 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c00ca40 con 0x7f741c0aa400 2020-11-20T12:17:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c0655b0 con 0x7f741c0aa400 2020-11-20T12:17:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f74097fa700 1 -- 172.21.15.44:0/3080170590 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00cdc0 con 0x7f741c0aa400 2020-11-20T12:17:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.142+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c0ab550 con 0x7f741c0aa400 2020-11-20T12:17:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.143+0000 7f74097fa700 1 -- 172.21.15.44:0/3080170590 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f740c012600 con 0x7f741c0aa400 2020-11-20T12:17:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.143+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c061650 con 0x7f741c0aa400 2020-11-20T12:17:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.144+0000 7f74097fa700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f73f4056d20 0x7f73f40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.144+0000 7f74097fa700 1 -- 172.21.15.44:0/3080170590 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f740c070420 con 0x7f741c0aa400 2020-11-20T12:17:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.144+0000 7f742100a700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f73f4056d20 0x7f73f40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.144+0000 7f742100a700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f73f4056d20 0x7f73f40591e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f741c0ad8c0 tx=0x7f7418009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.148+0000 7f74097fa700 1 -- 172.21.15.44:0/3080170590 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f740c0456f0 con 0x7f741c0aa400 2020-11-20T12:17:42.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.421+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f741c0446d0 con 0x7f741c0aa400 2020-11-20T12:17:42.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.422+0000 7f74097fa700 1 -- 172.21.15.44:0/3080170590 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f740c048d10 con 0x7f741c0aa400 2020-11-20T12:17:42.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f73f4056d20 msgr2=0x7f73f40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f73f4056d20 0x7f73f40591e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f741c0ad8c0 tx=0x7f7418009040).stop 2020-11-20T12:17:42.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c0aa400 msgr2=0x7f741c139510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c0aa400 0x7f741c139510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f740c00eba0 tx=0x7f740c006df0).stop 2020-11-20T12:17:42.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 shutdown_connections 2020-11-20T12:17:42.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f73f4056d20 0x7f73f40591e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f741c13ac60 0x7f741c0aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f741c0aa020 0x7f741c067e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 --2- 172.21.15.44:0/3080170590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f741c0aa400 0x7f741c139510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 >> 172.21.15.44:0/3080170590 conn(0x7f741c062690 msgr2=0x7f741c13bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 shutdown_connections 2020-11-20T12:17:42.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:42.424+0000 7f742326e700 1 -- 172.21.15.44:0/3080170590 wait complete. 2020-11-20T12:17:42.439 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:17:52.440 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:17:52.441 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:17:52.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.571+0000 7f9bfe693700 1 Processor -- start 2020-11-20T12:17:52.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.571+0000 7f9bfe693700 1 -- start start 2020-11-20T12:17:52.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bfe693700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9bf80aa020 0x7f9bf80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:52.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bfe693700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf80aa400 0x7f9bf80a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:52.585 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bfe693700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80a9770 con 0x7f9bf813ac60 2020-11-20T12:17:52.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bfe693700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80a6da0 con 0x7f9bf80aa020 2020-11-20T12:17:52.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bfe693700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80a7040 con 0x7f9bf80aa400 2020-11-20T12:17:52.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bfcc30700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9bf813ac60 0x7f9bf80a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45852/0 (socket says 172.21.15.44:45852) 2020-11-20T12:17:52.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bfcc30700 1 -- 172.21.15.44:0/12495169 learned_addr learned my addr 172.21.15.44:0/12495169 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:17:52.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bf7fff700 1 --2- 172.21.15.44:0/12495169 >> v2:172.21.15.79:3300/0 conn(0x7f9bf80aa020 0x7f9bf80a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:52.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bf77fe700 1 --2- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf80aa400 0x7f9bf80a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:52.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bf6ffd700 1 -- 172.21.15.44:0/12495169 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451979953 0 0) 0x7f9bf0001450 con 0x7f9bf813ac60 2020-11-20T12:17:52.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.572+0000 7f9bf6ffd700 1 -- 172.21.15.44:0/12495169 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be4003240 con 0x7f9bf813ac60 2020-11-20T12:17:52.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf7fff700 1 -- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf80aa400 msgr2=0x7f9bf80a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:52.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf7fff700 1 --2- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf80aa400 0x7f9bf80a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf7fff700 1 -- 172.21.15.44:0/12495169 >> v1:172.21.15.44:6789/0 conn(0x7f9bf813ac60 legacy=0x7f9bf80a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:52.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf7fff700 1 -- 172.21.15.44:0/12495169 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80a6a90 con 0x7f9bf80aa020 2020-11-20T12:17:52.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf77fe700 1 --2- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf80aa400 0x7f9bf80a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:52.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf7fff700 1 --2- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa020 0x7f9bf80a9280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9be800b9a0 tx=0x7f9be8006df0).ready entity=mon.1 client_cookie=7018a694845fdfa7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:52.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf6ffd700 1 -- 172.21.15.44:0/12495169 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8009680 con 0x7f9bf80aa020 2020-11-20T12:17:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf6ffd700 1 -- 172.21.15.44:0/12495169 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be80097e0 con 0x7f9bf80aa020 2020-11-20T12:17:52.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bf6ffd700 1 -- 172.21.15.44:0/12495169 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8009b60 con 0x7f9bf80aa020 2020-11-20T12:17:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 -- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa020 msgr2=0x7f9bf80a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:52.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 --2- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa020 0x7f9bf80a9280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9be800b9a0 tx=0x7f9be8006df0).stop 2020-11-20T12:17:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 -- 172.21.15.44:0/12495169 shutdown_connections 2020-11-20T12:17:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 --2- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa020 0x7f9bf80a9280 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 --2- 172.21.15.44:0/12495169 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf80aa400 0x7f9bf80a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 -- 172.21.15.44:0/12495169 >> 172.21.15.44:0/12495169 conn(0x7f9bf8062690 msgr2=0x7f9bf80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:52.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 -- 172.21.15.44:0/12495169 shutdown_connections 2020-11-20T12:17:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.573+0000 7f9bfe693700 1 -- 172.21.15.44:0/12495169 wait complete. 2020-11-20T12:17:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.574+0000 7f9bfe693700 1 Processor -- start 2020-11-20T12:17:52.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bfe693700 1 -- start start 2020-11-20T12:17:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bfe693700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80aa020 0x7f9bf80ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bfe693700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa400 0x7f9bf809d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:52.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bfe693700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf813ac60 0x7f9bf8148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bfe693700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80ac030 con 0x7f9bf80aa020 2020-11-20T12:17:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bfe693700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80ac2d0 con 0x7f9bf80aa400 2020-11-20T12:17:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bfe693700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80ac570 con 0x7f9bf813ac60 2020-11-20T12:17:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80aa020 0x7f9bf80ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bf7fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80aa020 0x7f9bf80ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42106/0 (socket says 172.21.15.44:42106) 2020-11-20T12:17:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.575+0000 7f9bf7fff700 1 -- 172.21.15.44:0/3574474636 learned_addr learned my addr 172.21.15.44:0/3574474636 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bfcc30700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf813ac60 0x7f9bf8148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:52.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf77fe700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa400 0x7f9bf809d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:52.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf7fff700 1 -- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf813ac60 msgr2=0x7f9bf8148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:52.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf7fff700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf813ac60 0x7f9bf8148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf7fff700 1 -- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa400 msgr2=0x7f9bf809d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:52.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf7fff700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa400 0x7f9bf809d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf7fff700 1 -- 172.21.15.44:0/3574474636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80677e0 con 0x7f9bf80aa020 2020-11-20T12:17:52.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf77fe700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa400 0x7f9bf809d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:52.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bfcc30700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf813ac60 0x7f9bf8148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:52.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bf7fff700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80aa020 0x7f9bf80ab630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f9bf8061280 tx=0x7f9be8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:52.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bd7fff700 1 -- 172.21.15.44:0/3574474636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8011df0 con 0x7f9bf80aa020 2020-11-20T12:17:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bd7fff700 1 -- 172.21.15.44:0/3574474636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be801e460 con 0x7f9bf80aa020 2020-11-20T12:17:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf80abb20 con 0x7f9bf80aa020 2020-11-20T12:17:52.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bd7fff700 1 -- 172.21.15.44:0/3574474636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be801e7e0 con 0x7f9bf80aa020 2020-11-20T12:17:52.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.576+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf80aca10 con 0x7f9bf80aa020 2020-11-20T12:17:52.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.577+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf80acc30 con 0x7f9bf80aa020 2020-11-20T12:17:52.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.577+0000 7f9bd7fff700 1 -- 172.21.15.44:0/3574474636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9be8025030 con 0x7f9bf80aa020 2020-11-20T12:17:52.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.578+0000 7f9bd7fff700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9bd0056d20 0x7f9bd00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.578+0000 7f9bd7fff700 1 -- 172.21.15.44:0/3574474636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9be8071560 con 0x7f9bf80aa020 2020-11-20T12:17:52.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.578+0000 7f9bf77fe700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9bd0056d20 0x7f9bd00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:52.609 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.578+0000 7f9bf77fe700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9bd0056d20 0x7f9bd00591e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9bec006fd0 tx=0x7f9bec008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:52.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.582+0000 7f9bd7fff700 1 -- 172.21.15.44:0/3574474636 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9be80468e0 con 0x7f9bf80aa020 2020-11-20T12:17:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.855+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9bf8061650 con 0x7f9bf80aa020 2020-11-20T12:17:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.856+0000 7f9bd7fff700 1 -- 172.21.15.44:0/3574474636 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f9be8023070 con 0x7f9bf80aa020 2020-11-20T12:17:52.859 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:17:52.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9bd0056d20 msgr2=0x7f9bd00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9bd0056d20 0x7f9bd00591e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9bec006fd0 tx=0x7f9bec008040).stop 2020-11-20T12:17:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80aa020 msgr2=0x7f9bf80ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80aa020 0x7f9bf80ab630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f9bf8061280 tx=0x7f9be8015000).stop 2020-11-20T12:17:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 shutdown_connections 2020-11-20T12:17:52.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9bd0056d20 0x7f9bd00591e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9bf813ac60 0x7f9bf8148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf80aa020 0x7f9bf80ab630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.857+0000 7f9bfe693700 1 --2- 172.21.15.44:0/3574474636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9bf80aa400 0x7f9bf809d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:52.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.858+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 >> 172.21.15.44:0/3574474636 conn(0x7f9bf8062690 msgr2=0x7f9bf80a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:52.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.858+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 shutdown_connections 2020-11-20T12:17:52.864 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:52.858+0000 7f9bfe693700 1 -- 172.21.15.44:0/3574474636 wait complete. 2020-11-20T12:17:52.876 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:17:52.877 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:17:52.879 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:17:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.017+0000 7fef300fe700 1 Processor -- start 2020-11-20T12:17:53.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.017+0000 7fef300fe700 1 -- start start 2020-11-20T12:17:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.017+0000 7fef300fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fef2813ab60 0x7fef280a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.017+0000 7fef300fe700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef280aaa00 con 0x7fef280b1d50 2020-11-20T12:17:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.017+0000 7fef300fe700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef280a9610 con 0x7fef280b1970 2020-11-20T12:17:53.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef300fe700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2813afd0 con 0x7fef2813ab60 2020-11-20T12:17:53.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2e69b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fef2813ab60 0x7fef280a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2e69b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fef2813ab60 0x7fef280a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42118/0 (socket says 172.21.15.44:42118) 2020-11-20T12:17:53.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2e69b700 1 -- 172.21.15.44:0/3833104474 learned_addr learned my addr 172.21.15.44:0/3833104474 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2e69b700 1 -- 172.21.15.44:0/3833104474 >> v1:172.21.15.79:6789/0 conn(0x7fef280b1970 legacy=0x7fef2813b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2e69b700 1 -- 172.21.15.44:0/3833104474 >> v1:172.21.15.44:6789/0 conn(0x7fef280b1d50 legacy=0x7fef28143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2e69b700 1 -- 172.21.15.44:0/3833104474 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280a98b0 con 0x7fef2813ab60 2020-11-20T12:17:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2e69b700 1 --2- 172.21.15.44:0/3833104474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef2813ab60 0x7fef280a86a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fef20009d50 tx=0x7fef20009290).ready entity=mon.0 client_cookie=25bb5a72477124f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2ce98700 1 -- 172.21.15.44:0/3833104474 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2000bc80 con 0x7fef2813ab60 2020-11-20T12:17:53.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2ce98700 1 -- 172.21.15.44:0/3833104474 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2000bde0 con 0x7fef2813ab60 2020-11-20T12:17:53.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.018+0000 7fef2ce98700 1 -- 172.21.15.44:0/3833104474 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20011610 con 0x7fef2813ab60 2020-11-20T12:17:53.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.019+0000 7fef300fe700 1 -- 172.21.15.44:0/3833104474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef2813ab60 msgr2=0x7fef280a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.019+0000 7fef300fe700 1 --2- 172.21.15.44:0/3833104474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef2813ab60 0x7fef280a86a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fef20009d50 tx=0x7fef20009290).stop 2020-11-20T12:17:53.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.019+0000 7fef300fe700 1 -- 172.21.15.44:0/3833104474 shutdown_connections 2020-11-20T12:17:53.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.019+0000 7fef300fe700 1 --2- 172.21.15.44:0/3833104474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef2813ab60 0x7fef280a86a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.019+0000 7fef300fe700 1 -- 172.21.15.44:0/3833104474 >> 172.21.15.44:0/3833104474 conn(0x7fef28060c40 msgr2=0x7fef280b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.019+0000 7fef300fe700 1 -- 172.21.15.44:0/3833104474 shutdown_connections 2020-11-20T12:17:53.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.019+0000 7fef300fe700 1 -- 172.21.15.44:0/3833104474 wait complete. 2020-11-20T12:17:53.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.020+0000 7fef300fe700 1 Processor -- start 2020-11-20T12:17:53.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.020+0000 7fef300fe700 1 -- start start 2020-11-20T12:17:53.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.020+0000 7fef300fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef280b1970 0x7fef280a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.020+0000 7fef300fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280b1d50 0x7fef2809e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef300fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef2813ab60 0x7fef2814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef300fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2809ea30 con 0x7fef280b1d50 2020-11-20T12:17:53.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef300fe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280a8330 con 0x7fef2813ab60 2020-11-20T12:17:53.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef300fe700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2809f0d0 con 0x7fef280b1970 2020-11-20T12:17:53.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280b1d50 0x7fef2809e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280b1d50 0x7fef2809e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42122/0 (socket says 172.21.15.44:42122) 2020-11-20T12:17:53.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 -- 172.21.15.44:0/3806060054 learned_addr learned my addr 172.21.15.44:0/3806060054 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:53.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2de9a700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef280b1970 0x7fef280a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2e69b700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef2813ab60 0x7fef2814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef280b1970 msgr2=0x7fef280a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fef280b1970 0x7fef280a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef2813ab60 msgr2=0x7fef2814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef2813ab60 0x7fef2814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 -- 172.21.15.44:0/3806060054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2809f370 con 0x7fef280b1d50 2020-11-20T12:17:53.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2e69b700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef2813ab60 0x7fef2814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:53.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef2d699700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280b1d50 0x7fef2809e540 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fef18001e70 tx=0x7fef18007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18007020 con 0x7fef280b1d50 2020-11-20T12:17:53.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef18007180 con 0x7fef280b1d50 2020-11-20T12:17:53.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2814d980 con 0x7fef280b1d50 2020-11-20T12:17:53.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.021+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800e560 con 0x7fef280b1d50 2020-11-20T12:17:53.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.022+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2809f810 con 0x7fef280b1d50 2020-11-20T12:17:53.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.022+0000 7fef300fe700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280a3b20 0x7fef280a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.022+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2809eb70 con 0x7fef280a3b20 2020-11-20T12:17:53.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef2de9a700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280a3b20 0x7fef280a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fef18010070 con 0x7fef280b1d50 2020-11-20T12:17:53.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef2de9a700 1 -- 172.21.15.44:0/3806060054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef2809f500 con 0x7fef280a3b20 2020-11-20T12:17:53.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef2de9a700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280a3b20 0x7fef280a3f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fef24007fd0 tx=0x7fef24009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef00056d20 0x7fef000591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fef1806f4d0 con 0x7fef280b1d50 2020-11-20T12:17:53.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef240105e0 con 0x7fef280a3b20 2020-11-20T12:17:53.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fef24010740 con 0x7fef280a3b20 2020-11-20T12:17:53.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280a3b20 msgr2=0x7fef280a3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280a3b20 0x7fef280a3f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fef24007fd0 tx=0x7fef24009040).stop 2020-11-20T12:17:53.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280a3b20 msgr2=0x7fef280a3f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:53.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.023+0000 7fef15ffb700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280a3b20 0x7fef280a3f00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.024+0000 7fef2e69b700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef00056d20 0x7fef000591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.024+0000 7fef2e69b700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef00056d20 0x7fef000591e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fef20009fd0 tx=0x7fef20016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.031+0000 7fef300fe700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef28149f40 0x7fef2814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280619a0 con 0x7fef28149f40 2020-11-20T12:17:53.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef2de9a700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef28149f40 0x7fef2814a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef2de9a700 1 -- 172.21.15.44:0/3806060054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef2813a940 con 0x7fef28149f40 2020-11-20T12:17:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef2de9a700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef28149f40 0x7fef2814a350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fef24014000 tx=0x7fef24017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2401d040 con 0x7fef28149f40 2020-11-20T12:17:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4609 (secure 0 0 0) 0x7fef240103d0 con 0x7fef28149f40 2020-11-20T12:17:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef28149f40 msgr2=0x7fef2814a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.032+0000 7fef15ffb700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef28149f40 0x7fef2814a350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fef24014000 tx=0x7fef24017000).stop 2020-11-20T12:17:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef15ffb700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef28149f40 msgr2=0x7fef2814a350 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef15ffb700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef28149f40 0x7fef2814a350 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef00056d20 msgr2=0x7fef000591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fef00056d20 0x7fef000591e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fef20009fd0 tx=0x7fef20016000).stop 2020-11-20T12:17:53.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280b1d50 msgr2=0x7fef2809e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280b1d50 0x7fef2809e540 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fef18001e70 tx=0x7fef18007c50).stop 2020-11-20T12:17:53.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef2e69b700 1 -- 172.21.15.44:0/3806060054 reap_dead start 2020-11-20T12:17:53.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 shutdown_connections 2020-11-20T12:17:53.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 --2- 172.21.15.44:0/3806060054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef280b1d50 0x7fef2809e540 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 >> 172.21.15.44:0/3806060054 conn(0x7fef28060c40 msgr2=0x7fef2813bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 shutdown_connections 2020-11-20T12:17:53.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.033+0000 7fef300fe700 1 -- 172.21.15.44:0/3806060054 wait complete. 2020-11-20T12:17:53.062 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:53.062 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:53.063 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:53.063 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:17:53.063 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 142, 2020-11-20T12:17:53.063 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:53.063 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:53.064 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:17:53.064 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:53.064 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.064 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:17:53.064 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:53.065 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:53.065 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:53.065 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.065 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.065 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.066 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.066 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.066 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.066 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.066 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.067 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.067 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:53.067 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:53.067 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.067 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.068 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.068 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.068 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.068 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.068 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.069 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.069 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.069 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:53.069 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:53.070 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:53.070 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:53.070 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:53.070 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:53.070 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:53.071 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:53.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:53.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:53.071 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:53.071 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:53.072 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:53.072 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:53.072 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:53.072 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.072 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.073 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.073 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.073 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.073 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.073 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.074 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.074 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.074 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:53.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.074 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:53.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.075 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:53.075 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:53.075 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.075 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.076 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.076 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:53.076 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.077 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.077 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:53.077 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.078 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.078 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.078 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:53.078 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:53.079 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.079 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.079 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.080 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:53.080 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:53.080 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.080 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.082 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:53.082 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.082 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.083 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:53.083 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.083 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.084 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:53.084 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:53.084 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.084 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.084 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.085 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:53.085 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:53.085 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.086 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.086 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.086 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:53.086 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.087 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:53.087 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.088 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.088 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.088 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:53.088 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:53.089 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.089 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.089 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.089 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.090 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:53.090 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:53.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.090 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.091 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.091 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:53.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.091 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.091 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:17:53.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.092 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.092 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.092 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:53.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:17:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.094 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:53.094 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.094 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.094 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:53.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.095 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.095 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.095 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:17:53.095 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.095 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.096 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:53.096 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:53.097 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:17:53.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.233+0000 7f4d16ecd700 1 Processor -- start 2020-11-20T12:17:53.236 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.233+0000 7f4d16ecd700 1 -- start start 2020-11-20T12:17:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.233+0000 7f4d16ecd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d100abc70 0x7f4d100aac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.233+0000 7f4d16ecd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4d100a2710 0x7f4d1013eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.233+0000 7f4d16ecd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100ab100 con 0x7f4d100671c0 2020-11-20T12:17:53.239 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.233+0000 7f4d16ecd700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100a9f20 con 0x7f4d100a2710 2020-11-20T12:17:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.233+0000 7f4d16ecd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100ac960 con 0x7f4d100abc70 2020-11-20T12:17:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4d100a2710 0x7f4d1013eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4d100a2710 0x7f4d1013eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42132/0 (socket says 172.21.15.44:42132) 2020-11-20T12:17:53.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 -- 172.21.15.44:0/787777484 learned_addr learned my addr 172.21.15.44:0/787777484 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d14c69700 1 --2- 172.21.15.44:0/787777484 >> v2:172.21.15.79:3300/0 conn(0x7f4d100abc70 0x7f4d100aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 -- 172.21.15.44:0/787777484 >> v2:172.21.15.79:3300/0 conn(0x7f4d100abc70 msgr2=0x7f4d100aac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 --2- 172.21.15.44:0/787777484 >> v2:172.21.15.79:3300/0 conn(0x7f4d100abc70 0x7f4d100aac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 -- 172.21.15.44:0/787777484 >> v1:172.21.15.79:6789/0 conn(0x7f4d100671c0 legacy=0x7f4d10067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 -- 172.21.15.44:0/787777484 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100ab390 con 0x7f4d100a2710 2020-11-20T12:17:53.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0ffff700 1 --2- 172.21.15.44:0/787777484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100a2710 0x7f4d1013eb10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f4d1005c330 tx=0x7f4cfc006eb0).ready entity=mon.0 client_cookie=dae94772a701f148 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0f7fe700 1 -- 172.21.15.44:0/787777484 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc009960 con 0x7f4d100a2710 2020-11-20T12:17:53.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0f7fe700 1 -- 172.21.15.44:0/787777484 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc009ac0 con 0x7f4d100a2710 2020-11-20T12:17:53.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.234+0000 7f4d0f7fe700 1 -- 172.21.15.44:0/787777484 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc009e40 con 0x7f4d100a2710 2020-11-20T12:17:53.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 -- 172.21.15.44:0/787777484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100a2710 msgr2=0x7f4d1013eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 --2- 172.21.15.44:0/787777484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100a2710 0x7f4d1013eb10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f4d1005c330 tx=0x7f4cfc006eb0).stop 2020-11-20T12:17:53.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 -- 172.21.15.44:0/787777484 shutdown_connections 2020-11-20T12:17:53.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 --2- 172.21.15.44:0/787777484 >> v2:172.21.15.79:3300/0 conn(0x7f4d100abc70 0x7f4d100aac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 --2- 172.21.15.44:0/787777484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100a2710 0x7f4d1013eb10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 -- 172.21.15.44:0/787777484 >> 172.21.15.44:0/787777484 conn(0x7f4d10060170 msgr2=0x7f4d10060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 -- 172.21.15.44:0/787777484 shutdown_connections 2020-11-20T12:17:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.235+0000 7f4d16ecd700 1 -- 172.21.15.44:0/787777484 wait complete. 2020-11-20T12:17:53.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 Processor -- start 2020-11-20T12:17:53.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 -- start start 2020-11-20T12:17:53.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d100671c0 0x7f4d1013e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d100a2710 0x7f4d1013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100abc70 0x7f4d101332f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1013bc00 con 0x7f4d100abc70 2020-11-20T12:17:53.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1013b2e0 con 0x7f4d100671c0 2020-11-20T12:17:53.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.236+0000 7f4d16ecd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d101373f0 con 0x7f4d100a2710 2020-11-20T12:17:53.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100abc70 0x7f4d101332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100abc70 0x7f4d101332f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42140/0 (socket says 172.21.15.44:42140) 2020-11-20T12:17:53.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 -- 172.21.15.44:0/532334636 learned_addr learned my addr 172.21.15.44:0/532334636 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:53.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d0ffff700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d100a2710 0x7f4d1013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d14c69700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d100671c0 0x7f4d1013e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d100a2710 msgr2=0x7f4d1013adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4d100a2710 0x7f4d1013adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d100671c0 msgr2=0x7f4d1013e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d100671c0 0x7f4d1013e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 -- 172.21.15.44:0/532334636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10137690 con 0x7f4d100abc70 2020-11-20T12:17:53.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d14c69700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d100671c0 0x7f4d1013e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:53.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d1546a700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100abc70 0x7f4d101332f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f4d0800ca60 tx=0x7f4d08007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800a770 con 0x7f4d100abc70 2020-11-20T12:17:53.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800a8d0 con 0x7f4d100abc70 2020-11-20T12:17:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10139eb0 con 0x7f4d100abc70 2020-11-20T12:17:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800ac50 con 0x7f4d100abc70 2020-11-20T12:17:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.237+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d10135300 con 0x7f4d100abc70 2020-11-20T12:17:53.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.238+0000 7f4d16ecd700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d10137a60 0x7f4d101337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.238+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1005c430 con 0x7f4d10137a60 2020-11-20T12:17:53.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.239+0000 7f4d14c69700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d10137a60 0x7f4d101337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.239+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4d08019030 con 0x7f4d100abc70 2020-11-20T12:17:53.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.239+0000 7f4d14c69700 1 -- 172.21.15.44:0/532334636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d1013a040 con 0x7f4d10137a60 2020-11-20T12:17:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.239+0000 7f4d14c69700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d10137a60 0x7f4d101337e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f4d0400e720 tx=0x7f4d04008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.239+0000 7f4d0cff9700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ce8056d20 0x7f4ce80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.239+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4d0806f8b0 con 0x7f4d100abc70 2020-11-20T12:17:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.239+0000 7f4d0ffff700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ce8056d20 0x7f4ce80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.240+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d040133f0 con 0x7f4d10137a60 2020-11-20T12:17:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.240+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f4d04013550 con 0x7f4d10137a60 2020-11-20T12:17:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.240+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d10137a60 msgr2=0x7f4d101337e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.240+0000 7f4d0cff9700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d10137a60 0x7f4d101337e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f4d0400e720 tx=0x7f4d04008040).stop 2020-11-20T12:17:53.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.240+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d10137a60 msgr2=0x7f4d101337e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:53.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.240+0000 7f4d0cff9700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d10137a60 0x7f4d101337e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.240+0000 7f4d0ffff700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ce8056d20 0x7f4ce80591e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4cfc00b1b0 tx=0x7f4cfc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.250+0000 7f4d16ecd700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d1014d880 0x7f4d1014dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.250+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10137240 con 0x7f4d1014d880 2020-11-20T12:17:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.250+0000 7f4d14c69700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d1014d880 0x7f4d1014dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.259 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.250+0000 7f4d14c69700 1 -- 172.21.15.44:0/532334636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4d10061650 con 0x7f4d1014d880 2020-11-20T12:17:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.250+0000 7f4d14c69700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d1014d880 0x7f4d1014dc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f4d04014000 tx=0x7f4d04017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d040223e0 con 0x7f4d1014d880 2020-11-20T12:17:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7f4d04022540 con 0x7f4d1014d880 2020-11-20T12:17:53.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d1014d880 msgr2=0x7f4d1014dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d0cff9700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d1014d880 0x7f4d1014dc60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f4d04014000 tx=0x7f4d04017000).stop 2020-11-20T12:17:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d0cff9700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d1014d880 msgr2=0x7f4d1014dc60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:53.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d0cff9700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d1014d880 0x7f4d1014dc60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ce8056d20 msgr2=0x7f4ce80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4ce8056d20 0x7f4ce80591e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4cfc00b1b0 tx=0x7f4cfc015000).stop 2020-11-20T12:17:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100abc70 msgr2=0x7f4d101332f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 --2- 172.21.15.44:0/532334636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d100abc70 0x7f4d101332f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f4d0800ca60 tx=0x7f4d08007f40).stop 2020-11-20T12:17:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d1546a700 1 -- 172.21.15.44:0/532334636 reap_dead start 2020-11-20T12:17:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d1546a700 1 -- 172.21.15.44:0/532334636 reap_dead start 2020-11-20T12:17:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 shutdown_connections 2020-11-20T12:17:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 >> 172.21.15.44:0/532334636 conn(0x7f4d10060170 msgr2=0x7f4d101346c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 shutdown_connections 2020-11-20T12:17:53.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.251+0000 7f4d16ecd700 1 -- 172.21.15.44:0/532334636 wait complete. 2020-11-20T12:17:53.264 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:17:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 142, 2020-11-20T12:17:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:17:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:17:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:53.298 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:53.300 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:17:53.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.436+0000 7f691fa16700 1 Processor -- start 2020-11-20T12:17:53.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.436+0000 7f691fa16700 1 -- start start 2020-11-20T12:17:53.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691fa16700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f69180aad00 0x7f69180a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691fa16700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f69180a86d0 0x7f691813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691fa16700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69181385f0 con 0x7f69180ab0e0 2020-11-20T12:17:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691fa16700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691813e250 con 0x7f69180a86d0 2020-11-20T12:17:53.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691fa16700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691813e4f0 con 0x7f69180aad00 2020-11-20T12:17:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691dfb3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f69180a86d0 0x7f691813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691cfb1700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f69180ab0e0 0x7f69180abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:45898/0 (socket says 172.21.15.44:45898) 2020-11-20T12:17:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691cfb1700 1 -- 172.21.15.44:0/3564061807 learned_addr learned my addr 172.21.15.44:0/3564061807 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:17:53.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.437+0000 7f691d7b2700 1 --2- 172.21.15.44:0/3564061807 >> v2:172.21.15.79:3300/0 conn(0x7f69180aad00 0x7f69180a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691dfb3700 1 -- 172.21.15.44:0/3564061807 >> v2:172.21.15.79:3300/0 conn(0x7f69180aad00 msgr2=0x7f69180a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691dfb3700 1 --2- 172.21.15.44:0/3564061807 >> v2:172.21.15.79:3300/0 conn(0x7f69180aad00 0x7f69180a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691dfb3700 1 -- 172.21.15.44:0/3564061807 >> v1:172.21.15.44:6789/0 conn(0x7f69180ab0e0 legacy=0x7f69180abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691dfb3700 1 -- 172.21.15.44:0/3564061807 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69180ac340 con 0x7f69180a86d0 2020-11-20T12:17:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691dfb3700 1 --2- 172.21.15.44:0/3564061807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691813ec30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6918060f50 tx=0x7f69100091d0).ready entity=mon.0 client_cookie=e201104fdfab1f70 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f6907fff700 1 -- 172.21.15.44:0/3564061807 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000ba70 con 0x7f69180a86d0 2020-11-20T12:17:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f6907fff700 1 -- 172.21.15.44:0/3564061807 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691000bbd0 con 0x7f69180a86d0 2020-11-20T12:17:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f6907fff700 1 -- 172.21.15.44:0/3564061807 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910011400 con 0x7f69180a86d0 2020-11-20T12:17:53.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691fa16700 1 -- 172.21.15.44:0/3564061807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 msgr2=0x7f691813ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691fa16700 1 --2- 172.21.15.44:0/3564061807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691813ec30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f6918060f50 tx=0x7f69100091d0).stop 2020-11-20T12:17:53.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.438+0000 7f691fa16700 1 -- 172.21.15.44:0/3564061807 shutdown_connections 2020-11-20T12:17:53.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.439+0000 7f691fa16700 1 --2- 172.21.15.44:0/3564061807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691813ec30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.439+0000 7f691fa16700 1 --2- 172.21.15.44:0/3564061807 >> v2:172.21.15.79:3300/0 conn(0x7f69180aad00 0x7f69180a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.439+0000 7f691fa16700 1 -- 172.21.15.44:0/3564061807 >> 172.21.15.44:0/3564061807 conn(0x7f6918060170 msgr2=0x7f6918060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.439+0000 7f691fa16700 1 -- 172.21.15.44:0/3564061807 shutdown_connections 2020-11-20T12:17:53.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.439+0000 7f691fa16700 1 -- 172.21.15.44:0/3564061807 wait complete. 2020-11-20T12:17:53.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 Processor -- start 2020-11-20T12:17:53.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 -- start start 2020-11-20T12:17:53.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691809e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f69180aad00 0x7f69180a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180ab0e0 0x7f69180a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69180a7770 con 0x7f69180a86d0 2020-11-20T12:17:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69180a5110 con 0x7f69180ab0e0 2020-11-20T12:17:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.440+0000 7f691fa16700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180a4040 con 0x7f69180aad00 2020-11-20T12:17:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691809e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691809e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42154/0 (socket says 172.21.15.44:42154) 2020-11-20T12:17:53.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 -- 172.21.15.44:0/1052345779 learned_addr learned my addr 172.21.15.44:0/1052345779 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:53.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691dfb3700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180ab0e0 0x7f69180a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691cfb1700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f69180aad00 0x7f69180a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f69180aad00 msgr2=0x7f69180a7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f69180aad00 0x7f69180a7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180ab0e0 msgr2=0x7f69180a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69180ab0e0 0x7f69180a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 -- 172.21.15.44:0/1052345779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69180a42e0 con 0x7f69180a86d0 2020-11-20T12:17:53.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691d7b2700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691809e010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f691400ebd0 tx=0x7f6914006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691400c8e0 con 0x7f69180a86d0 2020-11-20T12:17:53.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691400ca40 con 0x7f69180a86d0 2020-11-20T12:17:53.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691400cdc0 con 0x7f69180a86d0 2020-11-20T12:17:53.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69180a4580 con 0x7f69180a86d0 2020-11-20T12:17:53.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.441+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69180a6190 con 0x7f69180a86d0 2020-11-20T12:17:53.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.442+0000 7f691fa16700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918133470 0x7f6918133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.442+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691805c5f0 con 0x7f6918133470 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.442+0000 7f691cfb1700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918133470 0x7f6918133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f691cfb1700 1 -- 172.21.15.44:0/1052345779 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69180a4710 con 0x7f6918133470 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f691cfb1700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918133470 0x7f6918133880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f69180a8230 tx=0x7f6908009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080070d0 con 0x7f6918133470 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f6914012600 con 0x7f69180a86d0 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f68f0056dc0 0x7f68f0059280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f6914070460 con 0x7f69180a86d0 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f6908007230 con 0x7f6918133470 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918133470 msgr2=0x7f6918133880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918133470 0x7f6918133880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f69180a8230 tx=0x7f6908009040).stop 2020-11-20T12:17:53.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918133470 msgr2=0x7f6918133880 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:53.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f69057fa700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918133470 0x7f6918133880 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0x7f69180a8230 tx=0x7f6908009040).stop 2020-11-20T12:17:53.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.443+0000 7f691dfb3700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f68f0056dc0 0x7f68f0059280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.444+0000 7f691dfb3700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f68f0056dc0 0x7f68f0059280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f6910009210 tx=0x7f6910016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.456+0000 7f691fa16700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918149f50 0x7f691814a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.456+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180a5b10 con 0x7f6918149f50 2020-11-20T12:17:53.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.456+0000 7f691cfb1700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918149f50 0x7f691814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691cfb1700 1 -- 172.21.15.44:0/1052345779 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f69180a3e30 con 0x7f6918149f50 2020-11-20T12:17:53.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691cfb1700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918149f50 0x7f691814a360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f6908015000 tx=0x7f6908016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080119b0 con 0x7f6918149f50 2020-11-20T12:17:53.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7f69080055d0 con 0x7f6918149f50 2020-11-20T12:17:53.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918149f50 msgr2=0x7f691814a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f69057fa700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918149f50 0x7f691814a360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f6908015000 tx=0x7f6908016000).stop 2020-11-20T12:17:53.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f69057fa700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918149f50 msgr2=0x7f691814a360 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:17:53.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f69057fa700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6918149f50 0x7f691814a360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f68f0056dc0 msgr2=0x7f68f0059280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691fa16700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f68f0056dc0 0x7f68f0059280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f6910009210 tx=0x7f6910016000).stop 2020-11-20T12:17:53.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 msgr2=0x7f691809e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691fa16700 1 --2- 172.21.15.44:0/1052345779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69180a86d0 0x7f691809e010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f691400ebd0 tx=0x7f6914006df0).stop 2020-11-20T12:17:53.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691dfb3700 1 -- 172.21.15.44:0/1052345779 reap_dead start 2020-11-20T12:17:53.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.457+0000 7f691dfb3700 1 -- 172.21.15.44:0/1052345779 reap_dead start 2020-11-20T12:17:53.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.458+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 shutdown_connections 2020-11-20T12:17:53.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.458+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 >> 172.21.15.44:0/1052345779 conn(0x7f6918060170 msgr2=0x7f69180ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.458+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 shutdown_connections 2020-11-20T12:17:53.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.458+0000 7f691fa16700 1 -- 172.21.15.44:0/1052345779 wait complete. 2020-11-20T12:17:53.468 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:17:53.468 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:53.468 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:53.468 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:17:53.469 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 142, 2020-11-20T12:17:53.469 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:17:53.469 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:17:53.469 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:17:53.469 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:17:53.470 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.470 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 12, 2020-11-20T12:17:53.470 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:53.470 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:17:53.471 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:17:53.471 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.471 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.471 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.471 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.472 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.472 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.472 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.472 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.472 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.473 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:17:53.473 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:17:53.473 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.473 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.473 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.474 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.474 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.474 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.474 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.474 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.475 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.475 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:17:53.475 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:17:53.475 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:17:53.476 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:17:53.476 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:17:53.476 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:17:53.476 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:53.476 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:17:53.477 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:17:53.477 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:17:53.477 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:17:53.477 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:17:53.477 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:17:53.478 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:17:53.478 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:17:53.478 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:17:53.478 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:17:53.479 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:17:53.479 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:17:53.479 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:17:53.479 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:17:53.479 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:17:53.480 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:17:53.480 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.480 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:17:53.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.480 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:17:53.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.481 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:17:53.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:17:53.481 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.481 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.482 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.482 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:17:53.482 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.483 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.483 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:17:53.483 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.484 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.484 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:17:53.484 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:17:53.484 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.485 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.485 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.485 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:17:53.486 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:17:53.486 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.486 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.487 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:17:53.487 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.488 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:17:53.488 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.488 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.489 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:17:53.489 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:17:53.489 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.489 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.489 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.490 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:17:53.490 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:17:53.491 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:17:53.491 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:17:53.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:17:53.491 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:17:53.492 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:17:53.492 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:17:53.493 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:17:53.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.493 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.493 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:17:53.494 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:17:53.494 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:17:53.494 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:17:53.494 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:17:53.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.495 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.495 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:17:53.495 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:17:53.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.496 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.496 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.496 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:53.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.496 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.497 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:17:53.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.497 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.497 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.497 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:17:53.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.498 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.498 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:17:53.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.498 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.499 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.499 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:53.499 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.499 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:17:53.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.500 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.500 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.500 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:17:53.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.501 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:17:53.501 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:17:53.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:17:53.501 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:17:53.501 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:17:53.502 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:17:53.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:17:53.502 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:17:53.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:17:53.502 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:17:53.503 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:17:53.504 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:17:53.504 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:17:53.504 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:17:53.504 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:17:53.505 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:17:53.505 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:17:53.516 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:17:53.516 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:17:53.516 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:17:53.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.690+0000 7f52be5b7700 1 Processor -- start 2020-11-20T12:17:53.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.690+0000 7f52be5b7700 1 -- start start 2020-11-20T12:17:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52be5b7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a6600 0x7f52b80a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52be5b7700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f52b80a8af0 0x7f52b80a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52be5b7700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b80a8f60 con 0x7f52b80a9880 2020-11-20T12:17:53.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52be5b7700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b80a71b0 con 0x7f52b80a8af0 2020-11-20T12:17:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52be5b7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80a6b80 con 0x7f52b80a6600 2020-11-20T12:17:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52bd5b5700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f52b80a9880 legacy=0x7f52b80a7630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:17:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52bddb6700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f52b80a8af0 msgr2=0x7f52b80a7d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:17:53.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52bddb6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f52b80a8af0 0x7f52b80a7d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52bcdb4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a6600 0x7f52b80a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52bcdb4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a6600 0x7f52b80a4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53630/0 (socket says 172.21.15.44:53630) 2020-11-20T12:17:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.691+0000 7f52bcdb4700 1 -- 172.21.15.44:0/4287421674 learned_addr learned my addr 172.21.15.44:0/4287421674 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52bcdb4700 1 -- 172.21.15.44:0/4287421674 >> v2:172.21.15.44:3300/0 conn(0x7f52b80a8af0 msgr2=0x7f52b80a7d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:53.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52bcdb4700 1 --2- 172.21.15.44:0/4287421674 >> v2:172.21.15.44:3300/0 conn(0x7f52b80a8af0 0x7f52b80a7d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52bcdb4700 1 -- 172.21.15.44:0/4287421674 >> v1:172.21.15.44:6789/0 conn(0x7f52b80a9880 legacy=0x7f52b80a7630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52bcdb4700 1 -- 172.21.15.44:0/4287421674 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b80a6e20 con 0x7f52b80a6600 2020-11-20T12:17:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52bcdb4700 1 --2- 172.21.15.44:0/4287421674 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a6600 0x7f52b80a4ef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f52a8006fd0 tx=0x7f52a8006eb0).ready entity=mon.2 client_cookie=de08ac169e97052 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52a7fff700 1 -- 172.21.15.44:0/4287421674 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8009920 con 0x7f52b80a6600 2020-11-20T12:17:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52a7fff700 1 -- 172.21.15.44:0/4287421674 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a8009a80 con 0x7f52b80a6600 2020-11-20T12:17:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52a7fff700 1 -- 172.21.15.44:0/4287421674 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8009e00 con 0x7f52b80a6600 2020-11-20T12:17:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52be5b7700 1 -- 172.21.15.44:0/4287421674 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a6600 msgr2=0x7f52b80a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52be5b7700 1 --2- 172.21.15.44:0/4287421674 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a6600 0x7f52b80a4ef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f52a8006fd0 tx=0x7f52a8006eb0).stop 2020-11-20T12:17:53.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52be5b7700 1 -- 172.21.15.44:0/4287421674 shutdown_connections 2020-11-20T12:17:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52be5b7700 1 --2- 172.21.15.44:0/4287421674 >> v2:172.21.15.44:3300/0 conn(0x7f52b80a8af0 0x7f52b80a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52be5b7700 1 --2- 172.21.15.44:0/4287421674 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a6600 0x7f52b80a4ef0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.692+0000 7f52be5b7700 1 -- 172.21.15.44:0/4287421674 >> 172.21.15.44:0/4287421674 conn(0x7f52b8061b60 msgr2=0x7f52b8061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.693+0000 7f52be5b7700 1 -- 172.21.15.44:0/4287421674 shutdown_connections 2020-11-20T12:17:53.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.693+0000 7f52be5b7700 1 -- 172.21.15.44:0/4287421674 wait complete. 2020-11-20T12:17:53.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.693+0000 7f52be5b7700 1 Processor -- start 2020-11-20T12:17:53.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52be5b7700 1 -- start start 2020-11-20T12:17:53.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52be5b7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80a6600 0x7f52b809f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52be5b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b80a8af0 0x7f52b80a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52be5b7700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a9880 0x7f52b814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52be5b7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b814a880 con 0x7f52b80a6600 2020-11-20T12:17:53.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52be5b7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b814aaa0 con 0x7f52b80a8af0 2020-11-20T12:17:53.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52be5b7700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b814ad40 con 0x7f52b80a9880 2020-11-20T12:17:53.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52bd5b5700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80a6600 msgr2=0x7f52b809f7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:17:53.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.694+0000 7f52bd5b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80a6600 0x7f52b809f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:53.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a9880 0x7f52b814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bcdb4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b80a8af0 0x7f52b80a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a9880 0x7f52b814a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53638/0 (socket says 172.21.15.44:53638) 2020-11-20T12:17:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 -- 172.21.15.44:0/743711013 learned_addr learned my addr 172.21.15.44:0/743711013 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:53.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 -- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b80a8af0 msgr2=0x7f52b80a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b80a8af0 0x7f52b80a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 -- 172.21.15.44:0/743711013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80a6600 msgr2=0x7f52b809f7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:17:53.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80a6600 0x7f52b809f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:53.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 -- 172.21.15.44:0/743711013 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b814afe0 con 0x7f52b80a9880 2020-11-20T12:17:53.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bcdb4700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b80a8af0 0x7f52b80a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:53.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.695+0000 7f52bddb6700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a9880 0x7f52b814a390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f52b0009f00 tx=0x7f52b0009120).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.696+0000 7f52a57fa700 1 -- 172.21.15.44:0/743711013 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0003c60 con 0x7f52b80a9880 2020-11-20T12:17:53.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.696+0000 7f52a57fa700 1 -- 172.21.15.44:0/743711013 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b0003dc0 con 0x7f52b80a9880 2020-11-20T12:17:53.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.696+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b8146ea0 con 0x7f52b80a9880 2020-11-20T12:17:53.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.696+0000 7f52a57fa700 1 -- 172.21.15.44:0/743711013 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000e890 con 0x7f52b80a9880 2020-11-20T12:17:53.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.696+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b81473e0 con 0x7f52b80a9880 2020-11-20T12:17:53.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.697+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b80aa430 con 0x7f52b80a9880 2020-11-20T12:17:53.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.697+0000 7f52a57fa700 1 -- 172.21.15.44:0/743711013 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f52b000ec40 con 0x7f52b80a9880 2020-11-20T12:17:53.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.697+0000 7f52a57fa700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5290056cd0 0x7f5290059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.697+0000 7f52a57fa700 1 -- 172.21.15.44:0/743711013 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f52b0013020 con 0x7f52b80a9880 2020-11-20T12:17:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.698+0000 7f52bd5b5700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5290056cd0 0x7f5290059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.698+0000 7f52bd5b5700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5290056cd0 0x7f5290059190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f52b40036a0 tx=0x7f52b4006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.701+0000 7f52a57fa700 1 -- 172.21.15.44:0/743711013 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f52b0078080 con 0x7f52b80a9880 2020-11-20T12:17:53.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.981+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f52b80a9330 con 0x7f52b80a9880 2020-11-20T12:17:53.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.982+0000 7f52a57fa700 1 -- 172.21.15.44:0/743711013 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f52b0047040 con 0x7f52b80a9880 2020-11-20T12:17:53.985 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:17:53.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5290056cd0 msgr2=0x7f5290059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5290056cd0 0x7f5290059190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f52b40036a0 tx=0x7f52b4006ae0).stop 2020-11-20T12:17:53.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a9880 msgr2=0x7f52b814a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a9880 0x7f52b814a390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f52b0009f00 tx=0x7f52b0009120).stop 2020-11-20T12:17:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 shutdown_connections 2020-11-20T12:17:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5290056cd0 0x7f5290059190 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f52b80a9880 0x7f52b814a390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52b80a6600 0x7f52b809f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 --2- 172.21.15.44:0/743711013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52b80a8af0 0x7f52b80a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.983+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 >> 172.21.15.44:0/743711013 conn(0x7f52b8061b60 msgr2=0x7f52b80ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.984+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 shutdown_connections 2020-11-20T12:17:53.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:53.984+0000 7f52be5b7700 1 -- 172.21.15.44:0/743711013 wait complete. 2020-11-20T12:17:56.999 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:17:57.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.134+0000 7efc41676700 1 Processor -- start 2020-11-20T12:17:57.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.134+0000 7efc41676700 1 -- start start 2020-11-20T12:17:57.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc41676700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efc3c0a8b60 0x7efc3c137e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:57.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc41676700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a4a20 0x7efc3c141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:57.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc41676700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c1350d0 con 0x7efc3c0a8780 2020-11-20T12:17:57.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc41676700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c0a4ec0 con 0x7efc3c0a8b60 2020-11-20T12:17:57.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc41676700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0a5160 con 0x7efc3c0a4a20 2020-11-20T12:17:57.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc33fff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7efc3c0a8b60 msgr2=0x7efc3c137e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:17:57.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc33fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efc3c0a8b60 0x7efc3c137e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:57.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc3affd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7efc3c0a8780 0x7efc3c0aa0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:48828/0 (socket says 172.21.15.44:48828) 2020-11-20T12:17:57.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc3b7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a4a20 0x7efc3c141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:57.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc3affd700 1 -- 172.21.15.44:0/377715483 learned_addr learned my addr 172.21.15.44:0/377715483 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:17:57.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc3a7fc700 1 -- 172.21.15.44:0/377715483 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321727357 0 0) 0x7efc3c1350d0 con 0x7efc3c0a8780 2020-11-20T12:17:57.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.135+0000 7efc3a7fc700 1 -- 172.21.15.44:0/377715483 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1c003240 con 0x7efc3c0a8780 2020-11-20T12:17:57.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3b7fe700 1 -- 172.21.15.44:0/377715483 >> v2:172.21.15.44:3300/0 conn(0x7efc3c0a8b60 msgr2=0x7efc3c137e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:17:57.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3b7fe700 1 --2- 172.21.15.44:0/377715483 >> v2:172.21.15.44:3300/0 conn(0x7efc3c0a8b60 0x7efc3c137e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:57.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3b7fe700 1 -- 172.21.15.44:0/377715483 >> v1:172.21.15.79:6789/0 conn(0x7efc3c0a8780 legacy=0x7efc3c0aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:57.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3b7fe700 1 -- 172.21.15.44:0/377715483 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0a4270 con 0x7efc3c0a4a20 2020-11-20T12:17:57.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3b7fe700 1 --2- 172.21.15.44:0/377715483 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a4a20 0x7efc3c141ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efc34009c90 tx=0x7efc340091d0).ready entity=mon.2 client_cookie=af6e48a0250d9f63 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:57.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3a7fc700 1 -- 172.21.15.44:0/377715483 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400ba70 con 0x7efc3c0a4a20 2020-11-20T12:17:57.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3a7fc700 1 -- 172.21.15.44:0/377715483 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3400bbd0 con 0x7efc3c0a4a20 2020-11-20T12:17:57.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc3a7fc700 1 -- 172.21.15.44:0/377715483 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34011400 con 0x7efc3c0a4a20 2020-11-20T12:17:57.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc41676700 1 -- 172.21.15.44:0/377715483 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a4a20 msgr2=0x7efc3c141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:57.160 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.136+0000 7efc41676700 1 --2- 172.21.15.44:0/377715483 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a4a20 0x7efc3c141ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efc34009c90 tx=0x7efc340091d0).stop 2020-11-20T12:17:57.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.137+0000 7efc41676700 1 -- 172.21.15.44:0/377715483 shutdown_connections 2020-11-20T12:17:57.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.137+0000 7efc41676700 1 --2- 172.21.15.44:0/377715483 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a4a20 0x7efc3c141ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:57.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.137+0000 7efc41676700 1 --2- 172.21.15.44:0/377715483 >> v2:172.21.15.44:3300/0 conn(0x7efc3c0a8b60 0x7efc3c137e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:57.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.137+0000 7efc41676700 1 -- 172.21.15.44:0/377715483 >> 172.21.15.44:0/377715483 conn(0x7efc3c064810 msgr2=0x7efc3c0619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:57.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.137+0000 7efc41676700 1 -- 172.21.15.44:0/377715483 shutdown_connections 2020-11-20T12:17:57.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.137+0000 7efc41676700 1 -- 172.21.15.44:0/377715483 wait complete. 2020-11-20T12:17:57.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 Processor -- start 2020-11-20T12:17:57.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 -- start start 2020-11-20T12:17:57.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc3c0a4a20 0x7efc3c0a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a8780 0x7efc3c0a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:57.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc3c0a8b60 0x7efc3c0a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:57.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c14aae0 con 0x7efc3c0a8b60 2020-11-20T12:17:57.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0a39e0 con 0x7efc3c0a4a20 2020-11-20T12:17:57.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.138+0000 7efc41676700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c14a100 con 0x7efc3c0a8780 2020-11-20T12:17:57.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc3b7fe700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc3c0a8b60 msgr2=0x7efc3c0a2d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:17:57.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc3b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc3c0a8b60 0x7efc3c0a2d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:17:57.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc3affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc3c0a4a20 0x7efc3c0a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:57.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a8780 0x7efc3c0a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:57.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc3affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc3c0a4a20 0x7efc3c0a78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:34260/0 (socket says 172.21.15.44:34260) 2020-11-20T12:17:57.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a8780 0x7efc3c0a2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53650/0 (socket says 172.21.15.44:53650) 2020-11-20T12:17:57.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc3affd700 1 -- 172.21.15.44:0/1544163047 learned_addr learned my addr 172.21.15.44:0/1544163047 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:17:57.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 -- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc3c0a4a20 msgr2=0x7efc3c0a78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:57.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc3c0a4a20 0x7efc3c0a78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:57.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 -- 172.21.15.44:0/1544163047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc3c0a8b60 msgr2=0x7efc3c0a2d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:17:57.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc3c0a8b60 0x7efc3c0a2d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:57.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 -- 172.21.15.44:0/1544163047 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c14a3a0 con 0x7efc3c0a8780 2020-11-20T12:17:57.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc3affd700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc3c0a4a20 0x7efc3c0a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:57.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.139+0000 7efc33fff700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a8780 0x7efc3c0a2890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efc240096b0 tx=0x7efc24006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:57.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.140+0000 7efc32ffd700 1 -- 172.21.15.44:0/1544163047 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24006170 con 0x7efc3c0a8780 2020-11-20T12:17:57.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.140+0000 7efc32ffd700 1 -- 172.21.15.44:0/1544163047 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc240062f0 con 0x7efc3c0a8780 2020-11-20T12:17:57.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.140+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c14a640 con 0x7efc3c0a8780 2020-11-20T12:17:57.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.140+0000 7efc32ffd700 1 -- 172.21.15.44:0/1544163047 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240038e0 con 0x7efc3c0a8780 2020-11-20T12:17:57.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.140+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c09f0c0 con 0x7efc3c0a8780 2020-11-20T12:17:57.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.141+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c14a7d0 con 0x7efc3c0a8780 2020-11-20T12:17:57.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.141+0000 7efc32ffd700 1 -- 172.21.15.44:0/1544163047 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7efc2400f070 con 0x7efc3c0a8780 2020-11-20T12:17:57.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.141+0000 7efc32ffd700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc18056cd0 0x7efc18059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:57.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.141+0000 7efc32ffd700 1 -- 172.21.15.44:0/1544163047 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7efc2406f890 con 0x7efc3c0a8780 2020-11-20T12:17:57.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.142+0000 7efc3affd700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc18056cd0 0x7efc18059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:57.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.142+0000 7efc3affd700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc18056cd0 0x7efc18059190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7efc2c001ca0 tx=0x7efc2c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:57.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.146+0000 7efc32ffd700 1 -- 172.21.15.44:0/1544163047 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efc24044b60 con 0x7efc3c0a8780 2020-11-20T12:17:57.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.434+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efc3c057d80 con 0x7efc3c0a8780 2020-11-20T12:17:57.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.434+0000 7efc32ffd700 1 -- 172.21.15.44:0/1544163047 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7efc24048180 con 0x7efc3c0a8780 2020-11-20T12:17:57.438 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:17:57.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc18056cd0 msgr2=0x7efc18059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc18056cd0 0x7efc18059190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7efc2c001ca0 tx=0x7efc2c009040).stop 2020-11-20T12:17:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a8780 msgr2=0x7efc3c0a2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a8780 0x7efc3c0a2890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efc240096b0 tx=0x7efc24006df0).stop 2020-11-20T12:17:57.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 shutdown_connections 2020-11-20T12:17:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efc18056cd0 0x7efc18059190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc3c0a8b60 0x7efc3c0a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc3c0a4a20 0x7efc3c0a78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 --2- 172.21.15.44:0/1544163047 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc3c0a8780 0x7efc3c0a2890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:57.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 >> 172.21.15.44:0/1544163047 conn(0x7efc3c064810 msgr2=0x7efc3c135cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 shutdown_connections 2020-11-20T12:17:57.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:17:57.436+0000 7efc41676700 1 -- 172.21.15.44:0/1544163047 wait complete. 2020-11-20T12:18:00.453 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:00.619 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.616+0000 7f9d6a145700 1 Processor -- start 2020-11-20T12:18:00.620 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.616+0000 7f9d6a145700 1 -- start start 2020-11-20T12:18:00.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d6a145700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d6413e6a0 0x7f9d6413eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d6a145700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9d64137650 0x7f9d64137ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.622 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d6a145700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d640a9020 con 0x7f9d640661e0 2020-11-20T12:18:00.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d6a145700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d640a9290 con 0x7f9d64137650 2020-11-20T12:18:00.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d6a145700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6413e200 con 0x7f9d6413e6a0 2020-11-20T12:18:00.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d637fe700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f9d640661e0 legacy=0x7f9d640a8910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:18:00.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d63fff700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f9d64137650 msgr2=0x7f9d64137ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:00.623 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d63fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9d64137650 0x7f9d64137ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:00.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d6413e6a0 0x7f9d6413eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d6413e6a0 0x7f9d6413eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53660/0 (socket says 172.21.15.44:53660) 2020-11-20T12:18:00.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.617+0000 7f9d5bfff700 1 -- 172.21.15.44:0/1913803760 learned_addr learned my addr 172.21.15.44:0/1913803760 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:00.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d5bfff700 1 -- 172.21.15.44:0/1913803760 >> v2:172.21.15.44:3300/0 conn(0x7f9d64137650 msgr2=0x7f9d64137ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:00.624 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/1913803760 >> v2:172.21.15.44:3300/0 conn(0x7f9d64137650 0x7f9d64137ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:00.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d5bfff700 1 -- 172.21.15.44:0/1913803760 >> v1:172.21.15.44:6789/0 conn(0x7f9d640661e0 legacy=0x7f9d640a8910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:00.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d5bfff700 1 -- 172.21.15.44:0/1913803760 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6413eff0 con 0x7f9d6413e6a0 2020-11-20T12:18:00.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/1913803760 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d6413e6a0 0x7f9d6413eb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9d4c006fd0 tx=0x7f9d4c006eb0).ready entity=mon.2 client_cookie=bca3621c2271c06d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:00.625 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d62ffd700 1 -- 172.21.15.44:0/1913803760 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c009890 con 0x7f9d6413e6a0 2020-11-20T12:18:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d62ffd700 1 -- 172.21.15.44:0/1913803760 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0099f0 con 0x7f9d6413e6a0 2020-11-20T12:18:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d62ffd700 1 -- 172.21.15.44:0/1913803760 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c009d70 con 0x7f9d6413e6a0 2020-11-20T12:18:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d6a145700 1 -- 172.21.15.44:0/1913803760 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d6413e6a0 msgr2=0x7f9d6413eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.618+0000 7f9d6a145700 1 --2- 172.21.15.44:0/1913803760 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d6413e6a0 0x7f9d6413eb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f9d4c006fd0 tx=0x7f9d4c006eb0).stop 2020-11-20T12:18:00.626 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.619+0000 7f9d6a145700 1 -- 172.21.15.44:0/1913803760 shutdown_connections 2020-11-20T12:18:00.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.619+0000 7f9d6a145700 1 --2- 172.21.15.44:0/1913803760 >> v2:172.21.15.44:3300/0 conn(0x7f9d64137650 0x7f9d64137ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:00.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.619+0000 7f9d6a145700 1 --2- 172.21.15.44:0/1913803760 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d6413e6a0 0x7f9d6413eb00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.619+0000 7f9d6a145700 1 -- 172.21.15.44:0/1913803760 >> 172.21.15.44:0/1913803760 conn(0x7f9d640657f0 msgr2=0x7f9d64062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:00.627 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.619+0000 7f9d6a145700 1 -- 172.21.15.44:0/1913803760 shutdown_connections 2020-11-20T12:18:00.628 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.619+0000 7f9d6a145700 1 -- 172.21.15.44:0/1913803760 wait complete. 2020-11-20T12:18:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.620+0000 7f9d6a145700 1 Processor -- start 2020-11-20T12:18:00.630 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.620+0000 7f9d6a145700 1 -- start start 2020-11-20T12:18:00.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d6a145700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d640661e0 0x7f9d64136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d6a145700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d64137650 0x7f9d6413d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d6a145700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d64135d50 0x7f9d64139a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d6a145700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6414afc0 con 0x7f9d640661e0 2020-11-20T12:18:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d6a145700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d64136170 con 0x7f9d64135d50 2020-11-20T12:18:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d6a145700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d64136aa0 con 0x7f9d64137650 2020-11-20T12:18:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d637fe700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d640661e0 msgr2=0x7f9d64136e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d637fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d640661e0 0x7f9d64136e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:00.632 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d64137650 0x7f9d6413d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d64137650 0x7f9d6413d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53666/0 (socket says 172.21.15.44:53666) 2020-11-20T12:18:00.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 -- 172.21.15.44:0/3124907063 learned_addr learned my addr 172.21.15.44:0/3124907063 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:00.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d63fff700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d64135d50 0x7f9d64139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.633 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 -- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d64135d50 msgr2=0x7f9d64139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d64135d50 0x7f9d64139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 -- 172.21.15.44:0/3124907063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d640661e0 msgr2=0x7f9d64136e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:00.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d640661e0 0x7f9d64136e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:00.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.621+0000 7f9d5bfff700 1 -- 172.21.15.44:0/3124907063 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d64136360 con 0x7f9d64137650 2020-11-20T12:18:00.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.622+0000 7f9d5bfff700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d64137650 0x7f9d6413d7d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9d4c010000 tx=0x7f9d4c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.622+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/3124907063 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c01e460 con 0x7f9d64137650 2020-11-20T12:18:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.622+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/3124907063 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c01e5c0 con 0x7f9d64137650 2020-11-20T12:18:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.622+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d64136600 con 0x7f9d64137650 2020-11-20T12:18:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.622+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/3124907063 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c01e980 con 0x7f9d64137650 2020-11-20T12:18:00.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.622+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6413dd40 con 0x7f9d64137650 2020-11-20T12:18:00.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.623+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d64065c00 con 0x7f9d64137650 2020-11-20T12:18:00.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.623+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/3124907063 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9d4c01a040 con 0x7f9d64137650 2020-11-20T12:18:00.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.624+0000 7f9d5b7fe700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d38056cd0 0x7f9d38059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.624+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/3124907063 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9d4c071780 con 0x7f9d64137650 2020-11-20T12:18:00.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.624+0000 7f9d637fe700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d38056cd0 0x7f9d38059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.624+0000 7f9d637fe700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d38056cd0 0x7f9d38059190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9d6413d2d0 tx=0x7f9d54006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:00.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.628+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/3124907063 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9d4c046b00 con 0x7f9d64137650 2020-11-20T12:18:00.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.889+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9d640665c0 con 0x7f9d64137650 2020-11-20T12:18:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.890+0000 7f9d5b7fe700 1 -- 172.21.15.44:0/3124907063 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f9d4c023020 con 0x7f9d64137650 2020-11-20T12:18:00.894 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:00.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d38056cd0 msgr2=0x7f9d38059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d38056cd0 0x7f9d38059190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9d6413d2d0 tx=0x7f9d54006ae0).stop 2020-11-20T12:18:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d64137650 msgr2=0x7f9d6413d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d64137650 0x7f9d6413d7d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f9d4c010000 tx=0x7f9d4c015000).stop 2020-11-20T12:18:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 shutdown_connections 2020-11-20T12:18:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9d38056cd0 0x7f9d38059190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d64135d50 0x7f9d64139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d640661e0 0x7f9d64136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 --2- 172.21.15.44:0/3124907063 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9d64137650 0x7f9d6413d7d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 >> 172.21.15.44:0/3124907063 conn(0x7f9d640657f0 msgr2=0x7f9d64137ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:00.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 shutdown_connections 2020-11-20T12:18:00.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:00.891+0000 7f9d6a145700 1 -- 172.21.15.44:0/3124907063 wait complete. 2020-11-20T12:18:03.907 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:04.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.040+0000 7fe885355700 1 Processor -- start 2020-11-20T12:18:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.040+0000 7fe885355700 1 -- start start 2020-11-20T12:18:04.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe885355700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 0x7fe8800adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:04.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe885355700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8800ab630 con 0x7fe88013b770 2020-11-20T12:18:04.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe885355700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8800ab110 con 0x7fe8800a9510 2020-11-20T12:18:04.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe885355700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800ab3b0 con 0x7fe8800a9130 2020-11-20T12:18:04.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe87f7fe700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fe88013b770 legacy=0x7fe880145420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:18:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe87effd700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 msgr2=0x7fe8800adc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe87effd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 0x7fe8800adc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe87e7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe8800a9510 0x7fe88013b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:48858/0 (socket says 172.21.15.44:48858) 2020-11-20T12:18:04.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.041+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 learned_addr learned my addr 172.21.15.44:0/325626668 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:04.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.242+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 msgr2=0x7fe8800adc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:04.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.242+0000 7fe87f7fe700 1 -- 172.21.15.44:0/325626668 >> v1:172.21.15.44:6789/0 conn(0x7fe88013b770 legacy=0x7fe880145420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:18:04.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.242+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 0x7fe8800adc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:18:04.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.643+0000 7fe87f7fe700 1 -- 172.21.15.44:0/325626668 >> v1:172.21.15.44:6789/0 conn(0x7fe88013b770 legacy=0x7fe880145420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:18:04.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.643+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 msgr2=0x7fe8800adc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:04.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:04.643+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 0x7fe8800adc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:18:05.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:05.444+0000 7fe87f7fe700 1 -- 172.21.15.44:0/325626668 >> v1:172.21.15.44:6789/0 conn(0x7fe88013b770 legacy=0x7fe880145420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:18:05.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:05.444+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 msgr2=0x7fe8800adc90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:05.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:05.444+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 0x7fe8800adc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:18:07.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 msgr2=0x7fe8800adc90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.44:3300/0 conn(0x7fe8800a9130 0x7fe8800adc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 -- 172.21.15.44:0/325626668 >> v1:172.21.15.79:6789/0 conn(0x7fe8800a9510 legacy=0x7fe88013b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 -- 172.21.15.44:0/325626668 >> v1:172.21.15.44:6789/0 conn(0x7fe88013b770 legacy=0x7fe880145420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:07.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 -- 172.21.15.44:0/325626668 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800ab3b0 con 0x7fe8600033b0 2020-11-20T12:18:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87cff9700 1 -- 172.21.15.44:0/325626668 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe860008960 con 0x7fe860005dd0 2020-11-20T12:18:07.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 msgr2=0x7fe860008290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 msgr2=0x7fe860005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:18:07.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 msgr2=0x7fe860008290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 msgr2=0x7fe860005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:07.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:07.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:07.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.041+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:07.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 msgr2=0x7fe860005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:18:07.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 msgr2=0x7fe860005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:07.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 msgr2=0x7fe860008290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:07.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 msgr2=0x7fe860008290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:07.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:07.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:07.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:18:07.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.242+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:18:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 msgr2=0x7fe860008290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 msgr2=0x7fe860008290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 msgr2=0x7fe860005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:18:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87effd700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 msgr2=0x7fe860005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:18:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:07.643+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:18:08.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.444+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.444+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.444+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 msgr2=0x7fe860005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:08.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.444+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.444+0000 7fe87e7fc700 1 -- 172.21.15.44:0/325626668 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8800ad7f0 con 0x7fe860005dd0 2020-11-20T12:18:08.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.444+0000 7fe87effd700 1 --2- 172.21.15.44:0/325626668 >> v2:172.21.15.79:3300/0 conn(0x7fe8600033b0 0x7fe860005830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:08.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.444+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe874007a90 tx=0x7fe874009040).ready entity=mon.2 client_cookie=510904e68a1eb7d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe87dffb700 1 -- 172.21.15.44:0/325626668 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874014550 con 0x7fe860005dd0 2020-11-20T12:18:08.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe87dffb700 1 -- 172.21.15.44:0/325626668 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8740146b0 con 0x7fe860005dd0 2020-11-20T12:18:08.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe87dffb700 1 -- 172.21.15.44:0/325626668 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874014a30 con 0x7fe860005dd0 2020-11-20T12:18:08.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe885355700 1 -- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 msgr2=0x7fe860008290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe885355700 1 --2- 172.21.15.44:0/325626668 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe860005dd0 0x7fe860008290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe874007a90 tx=0x7fe874009040).stop 2020-11-20T12:18:08.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe87f7fe700 1 -- 172.21.15.44:0/325626668 reap_dead start 2020-11-20T12:18:08.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe885355700 1 -- 172.21.15.44:0/325626668 shutdown_connections 2020-11-20T12:18:08.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe885355700 1 -- 172.21.15.44:0/325626668 >> 172.21.15.44:0/325626668 conn(0x7fe880060170 msgr2=0x7fe880060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:08.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe885355700 1 -- 172.21.15.44:0/325626668 shutdown_connections 2020-11-20T12:18:08.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.445+0000 7fe885355700 1 -- 172.21.15.44:0/325626668 wait complete. 2020-11-20T12:18:08.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 Processor -- start 2020-11-20T12:18:08.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 -- start start 2020-11-20T12:18:08.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe860005dd0 0x7fe880141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe88009d550 0x7fe8800a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8800a7120 0x7fe88013dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800b17b0 con 0x7fe8800a7120 2020-11-20T12:18:08.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88009ed60 con 0x7fe860005dd0 2020-11-20T12:18:08.455 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe885355700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88009f000 con 0x7fe88009d550 2020-11-20T12:18:08.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe87f7fe700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8800a7120 msgr2=0x7fe88013dd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:08.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe87f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8800a7120 0x7fe88013dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:08.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe87e7fc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe88009d550 0x7fe8800a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe87effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe860005dd0 0x7fe880141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe87e7fc700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe88009d550 0x7fe8800a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53706/0 (socket says 172.21.15.44:53706) 2020-11-20T12:18:08.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.447+0000 7fe87e7fc700 1 -- 172.21.15.44:0/1837823351 learned_addr learned my addr 172.21.15.44:0/1837823351 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:08.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe87effd700 1 -- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe88009d550 msgr2=0x7fe8800a6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe87effd700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe88009d550 0x7fe8800a6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe87effd700 1 -- 172.21.15.44:0/1837823351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8800a7120 msgr2=0x7fe88013dd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:08.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe87effd700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8800a7120 0x7fe88013dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:08.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe87effd700 1 -- 172.21.15.44:0/1837823351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88009f2a0 con 0x7fe860005dd0 2020-11-20T12:18:08.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe88009d550 0x7fe8800a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:08.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe87effd700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe860005dd0 0x7fe880141430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe8700036a0 tx=0x7fe870008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe866ffd700 1 -- 172.21.15.44:0/1837823351 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870011d20 con 0x7fe860005dd0 2020-11-20T12:18:08.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe866ffd700 1 -- 172.21.15.44:0/1837823351 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870011e80 con 0x7fe860005dd0 2020-11-20T12:18:08.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8801386c0 con 0x7fe860005dd0 2020-11-20T12:18:08.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe866ffd700 1 -- 172.21.15.44:0/1837823351 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870005300 con 0x7fe860005dd0 2020-11-20T12:18:08.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.448+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88013e280 con 0x7fe860005dd0 2020-11-20T12:18:08.461 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.449+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88009f430 con 0x7fe860005dd0 2020-11-20T12:18:08.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.450+0000 7fe866ffd700 1 -- 172.21.15.44:0/1837823351 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe87001b020 con 0x7fe860005dd0 2020-11-20T12:18:08.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.450+0000 7fe866ffd700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe854056cd0 0x7fe854059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.450+0000 7fe866ffd700 1 -- 172.21.15.44:0/1837823351 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe870070b30 con 0x7fe860005dd0 2020-11-20T12:18:08.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.450+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe854056cd0 0x7fe854059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.451+0000 7fe87e7fc700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe854056cd0 0x7fe854059190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fe874004560 tx=0x7fe874009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.454+0000 7fe866ffd700 1 -- 172.21.15.44:0/1837823351 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe870045e00 con 0x7fe860005dd0 2020-11-20T12:18:08.518 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:08.530 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.722+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe8800a83c0 con 0x7fe860005dd0 2020-11-20T12:18:08.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.723+0000 7fe866ffd700 1 -- 172.21.15.44:0/1837823351 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fe870049420 con 0x7fe860005dd0 2020-11-20T12:18:08.728 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":144,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:08.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe854056cd0 msgr2=0x7fe854059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe854056cd0 0x7fe854059190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fe874004560 tx=0x7fe874009040).stop 2020-11-20T12:18:08.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe860005dd0 msgr2=0x7fe880141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe860005dd0 0x7fe880141430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe8700036a0 tx=0x7fe870008040).stop 2020-11-20T12:18:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 shutdown_connections 2020-11-20T12:18:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe854056cd0 0x7fe854059190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8800a7120 0x7fe88013dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe860005dd0 0x7fe880141430 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 --2- 172.21.15.44:0/1837823351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe88009d550 0x7fe8800a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 >> 172.21.15.44:0/1837823351 conn(0x7fe880060170 msgr2=0x7fe8800acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 shutdown_connections 2020-11-20T12:18:08.731 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.725+0000 7fe885355700 1 -- 172.21.15.44:0/1837823351 wait complete. 2020-11-20T12:18:08.740 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:18:08.740 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:18:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.879+0000 7f60be1f1700 1 Processor -- start 2020-11-20T12:18:08.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.879+0000 7f60be1f1700 1 -- start start 2020-11-20T12:18:08.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60be1f1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f60b80a4e10 0x7f60b8138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60be1f1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f60b80a74d0 0x7f60b8068130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60be1f1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b80a6d90 con 0x7f60b8061340 2020-11-20T12:18:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60be1f1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b80a5900 con 0x7f60b80a4e10 2020-11-20T12:18:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60be1f1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b80a6ba0 con 0x7f60b80a74d0 2020-11-20T12:18:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60b6ffd700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f60b80a4e10 msgr2=0x7f60b8138190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60b77fe700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f60b8061340 legacy=0x7f60b8137a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:18:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.880+0000 7f60b6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f60b80a4e10 0x7f60b8138190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:08.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f60b80a74d0 0x7f60b8068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f60b80a74d0 0x7f60b8068130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:34328/0 (socket says 172.21.15.44:34328) 2020-11-20T12:18:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 -- 172.21.15.44:0/3930919310 learned_addr learned my addr 172.21.15.44:0/3930919310 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 -- 172.21.15.44:0/3930919310 >> v2:172.21.15.44:3300/0 conn(0x7f60b80a4e10 msgr2=0x7f60b8138190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:08.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 --2- 172.21.15.44:0/3930919310 >> v2:172.21.15.44:3300/0 conn(0x7f60b80a4e10 0x7f60b8138190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 -- 172.21.15.44:0/3930919310 >> v1:172.21.15.44:6789/0 conn(0x7f60b8061340 legacy=0x7f60b8137a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 -- 172.21.15.44:0/3930919310 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b80a7030 con 0x7f60b80a74d0 2020-11-20T12:18:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b7fff700 1 --2- 172.21.15.44:0/3930919310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 0x7f60b8068130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f60b80680c0 tx=0x7f60b0009290).ready entity=mon.1 client_cookie=ab5324a311cabd0b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b67fc700 1 -- 172.21.15.44:0/3930919310 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b000bb80 con 0x7f60b80a74d0 2020-11-20T12:18:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b67fc700 1 -- 172.21.15.44:0/3930919310 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b000bce0 con 0x7f60b80a74d0 2020-11-20T12:18:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.881+0000 7f60b67fc700 1 -- 172.21.15.44:0/3930919310 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0011610 con 0x7f60b80a74d0 2020-11-20T12:18:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 -- 172.21.15.44:0/3930919310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 msgr2=0x7f60b8068130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 --2- 172.21.15.44:0/3930919310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 0x7f60b8068130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f60b80680c0 tx=0x7f60b0009290).stop 2020-11-20T12:18:08.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 -- 172.21.15.44:0/3930919310 shutdown_connections 2020-11-20T12:18:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 --2- 172.21.15.44:0/3930919310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 0x7f60b8068130 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 --2- 172.21.15.44:0/3930919310 >> v2:172.21.15.44:3300/0 conn(0x7f60b80a4e10 0x7f60b8138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 -- 172.21.15.44:0/3930919310 >> 172.21.15.44:0/3930919310 conn(0x7f60b80ad6e0 msgr2=0x7f60b80adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 -- 172.21.15.44:0/3930919310 shutdown_connections 2020-11-20T12:18:08.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.882+0000 7f60be1f1700 1 -- 172.21.15.44:0/3930919310 wait complete. 2020-11-20T12:18:08.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.883+0000 7f60be1f1700 1 Processor -- start 2020-11-20T12:18:08.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.883+0000 7f60be1f1700 1 -- start start 2020-11-20T12:18:08.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60be1f1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60b8061340 0x7f60b80674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60be1f1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60b80a4e10 0x7f60b8140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60be1f1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 0x7f60b8066150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60be1f1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80ad230 con 0x7f60b80a4e10 2020-11-20T12:18:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60be1f1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b8066b60 con 0x7f60b80a74d0 2020-11-20T12:18:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60be1f1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b8066d90 con 0x7f60b8061340 2020-11-20T12:18:08.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60b6ffd700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60b80a4e10 msgr2=0x7f60b8140230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60b6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60b80a4e10 0x7f60b8140230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60b77fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60b8061340 0x7f60b80674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60b77fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60b8061340 0x7f60b80674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53718/0 (socket says 172.21.15.44:53718) 2020-11-20T12:18:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60b77fe700 1 -- 172.21.15.44:0/3721173433 learned_addr learned my addr 172.21.15.44:0/3721173433 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:08.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.884+0000 7f60b7fff700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 0x7f60b8066150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60b7fff700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60b8061340 msgr2=0x7f60b80674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60b7fff700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60b8061340 0x7f60b80674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60b7fff700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60b80a4e10 msgr2=0x7f60b8140230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:08.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60b7fff700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60b80a4e10 0x7f60b8140230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60b7fff700 1 -- 172.21.15.44:0/3721173433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8067030 con 0x7f60b80a74d0 2020-11-20T12:18:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60b77fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f60b8061340 0x7f60b80674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60b7fff700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 0x7f60b8066150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f60b80680c0 tx=0x7f60b0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0020070 con 0x7f60b80a74d0 2020-11-20T12:18:08.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b0011e70 con 0x7f60b80a74d0 2020-11-20T12:18:08.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b8065160 con 0x7f60b80a74d0 2020-11-20T12:18:08.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b8066800 con 0x7f60b80a74d0 2020-11-20T12:18:08.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.885+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0010ce0 con 0x7f60b80a74d0 2020-11-20T12:18:08.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.886+0000 7f60be1f1700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80619a0 0x7f60b814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.886+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80671c0 con 0x7f60b80619a0 2020-11-20T12:18:08.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.886+0000 7f60b77fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80619a0 0x7f60b814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.886+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f60b0017070 con 0x7f60b80a74d0 2020-11-20T12:18:08.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.886+0000 7f60b77fe700 1 -- 172.21.15.44:0/3721173433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60b8067e00 con 0x7f60b80619a0 2020-11-20T12:18:08.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60b77fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80619a0 0x7f60b814d3d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f60a800edd0 tx=0x7f60a8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6090056cd0 0x7f6090059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f60b0071e20 con 0x7f60b80a74d0 2020-11-20T12:18:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60b6ffd700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6090056cd0 0x7f6090059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005320 con 0x7f60b80619a0 2020-11-20T12:18:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f60a8005480 con 0x7f60b80619a0 2020-11-20T12:18:08.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80619a0 msgr2=0x7f60b814d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80619a0 0x7f60b814d3d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f60a800edd0 tx=0x7f60a8008040).stop 2020-11-20T12:18:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80619a0 msgr2=0x7f60b814d3d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60977fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80619a0 0x7f60b814d3d0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0x7f60a800edd0 tx=0x7f60a8008040).stop 2020-11-20T12:18:08.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.887+0000 7f60b6ffd700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6090056cd0 0x7f6090059190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f60ac006fd0 tx=0x7f60ac006a90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.895+0000 7f60be1f1700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b814d8c0 0x7f60b814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.895+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80446d0 con 0x7f60b814d8c0 2020-11-20T12:18:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60b77fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b814d8c0 0x7f60b814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60b77fe700 1 -- 172.21.15.44:0/3721173433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f60b80697f0 con 0x7f60b814d8c0 2020-11-20T12:18:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60b77fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b814d8c0 0x7f60b814dca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f60a800edd0 tx=0x7f60a8018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:08.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8013660 con 0x7f60b814d8c0 2020-11-20T12:18:08.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4427 (secure 0 0 0) 0x7f60a8005110 con 0x7f60b814d8c0 2020-11-20T12:18:08.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b814d8c0 msgr2=0x7f60b814dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60977fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b814d8c0 0x7f60b814dca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f60a800edd0 tx=0x7f60a8018000).stop 2020-11-20T12:18:08.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60977fe700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b814d8c0 msgr2=0x7f60b814dca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.896+0000 7f60977fe700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b814d8c0 0x7f60b814dca0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6090056cd0 msgr2=0x7f6090059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6090056cd0 0x7f6090059190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f60ac006fd0 tx=0x7f60ac006a90).stop 2020-11-20T12:18:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 msgr2=0x7f60b8066150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 --2- 172.21.15.44:0/3721173433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60b80a74d0 0x7f60b8066150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f60b80680c0 tx=0x7f60b0016000).stop 2020-11-20T12:18:08.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60b7fff700 1 -- 172.21.15.44:0/3721173433 reap_dead start 2020-11-20T12:18:08.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60b7fff700 1 -- 172.21.15.44:0/3721173433 reap_dead start 2020-11-20T12:18:08.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 shutdown_connections 2020-11-20T12:18:08.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 >> 172.21.15.44:0/3721173433 conn(0x7f60b80ad6e0 msgr2=0x7f60b8137c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:08.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 shutdown_connections 2020-11-20T12:18:08.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:08.897+0000 7f60be1f1700 1 -- 172.21.15.44:0/3721173433 wait complete. 2020-11-20T12:18:08.912 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:08.912 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:18:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 144, 2020-11-20T12:18:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:08.913 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:18:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:08.914 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:08.915 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:08.916 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:08.917 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:08.918 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:08.919 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:08.920 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:08.921 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:08.922 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:08.923 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.924 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:08.925 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:08.926 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:08.927 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:08.928 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:08.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:08.932 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:08.933 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:08.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:08.935 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.936 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:08.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:08.938 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:08.939 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:08.940 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:18:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:08.941 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:08.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:18:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:08.943 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:08.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:08.944 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:08.944 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:08.945 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:18:09.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 Processor -- start 2020-11-20T12:18:09.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 -- start start 2020-11-20T12:18:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f74380a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f74380aa400 0x7f74380a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f743813ac60 0x7f74380a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74380a9770 con 0x7f743813ac60 2020-11-20T12:18:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74380a6da0 con 0x7f74380aa400 2020-11-20T12:18:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f743da25700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380a7040 con 0x7f74380aa020 2020-11-20T12:18:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f74377fe700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f743813ac60 msgr2=0x7f74380a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.088+0000 7f74377fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f743813ac60 0x7f74380a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f74380aa400 0x7f74380a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:09.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f7436ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f74380a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:09.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f74380aa400 0x7f74380a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:34344/0 (socket says 172.21.15.44:34344) 2020-11-20T12:18:09.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 -- 172.21.15.44:0/3452839184 learned_addr learned my addr 172.21.15.44:0/3452839184 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:09.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f7436ffd700 1 --2- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f74380a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53730/0 (socket says 172.21.15.44:53730) 2020-11-20T12:18:09.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 -- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 msgr2=0x7f74380a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 --2- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f74380a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 -- 172.21.15.44:0/3452839184 >> v2:172.21.15.44:3300/0 conn(0x7f743813ac60 msgr2=0x7f74380a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:09.098 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 --2- 172.21.15.44:0/3452839184 >> v2:172.21.15.44:3300/0 conn(0x7f743813ac60 0x7f74380a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:09.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 -- 172.21.15.44:0/3452839184 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380a6a90 con 0x7f74380aa400 2020-11-20T12:18:09.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f7436ffd700 1 --2- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f74380a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:09.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.089+0000 7f742ffff700 1 --2- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 0x7f74380a4fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7438062b30 tx=0x7f7420006eb0).ready entity=mon.1 client_cookie=4aec32c086408098 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.090+0000 7f74367fc700 1 -- 172.21.15.44:0/3452839184 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420009810 con 0x7f74380aa400 2020-11-20T12:18:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.090+0000 7f74367fc700 1 -- 172.21.15.44:0/3452839184 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7420009970 con 0x7f74380aa400 2020-11-20T12:18:09.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.090+0000 7f74367fc700 1 -- 172.21.15.44:0/3452839184 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420009cf0 con 0x7f74380aa400 2020-11-20T12:18:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 -- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 msgr2=0x7f74380a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 --2- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 0x7f74380a4fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7438062b30 tx=0x7f7420006eb0).stop 2020-11-20T12:18:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 -- 172.21.15.44:0/3452839184 shutdown_connections 2020-11-20T12:18:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 --2- 172.21.15.44:0/3452839184 >> v2:172.21.15.44:3300/0 conn(0x7f743813ac60 0x7f74380a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:09.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 --2- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f74380a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:09.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 --2- 172.21.15.44:0/3452839184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 0x7f74380a4fb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:09.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 -- 172.21.15.44:0/3452839184 >> 172.21.15.44:0/3452839184 conn(0x7f7438062690 msgr2=0x7f74380a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:09.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 -- 172.21.15.44:0/3452839184 shutdown_connections 2020-11-20T12:18:09.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.091+0000 7f743da25700 1 -- 172.21.15.44:0/3452839184 wait complete. 2020-11-20T12:18:09.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.092+0000 7f743da25700 1 Processor -- start 2020-11-20T12:18:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.092+0000 7f743da25700 1 -- start start 2020-11-20T12:18:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f743da25700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f743809f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f743da25700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 0x7f74380a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f743da25700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f743813ac60 0x7f74380a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f743da25700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380a2b40 con 0x7f743813ac60 2020-11-20T12:18:09.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f743da25700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380a2de0 con 0x7f74380aa400 2020-11-20T12:18:09.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f743da25700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380a3080 con 0x7f74380aa020 2020-11-20T12:18:09.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f74377fe700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f743813ac60 msgr2=0x7f74380a17f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:18:09.107 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f74377fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f743813ac60 0x7f74380a17f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:09.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f742ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 0x7f74380a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:09.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f7436ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f743809f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:09.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f7436ffd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f743809f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53736/0 (socket says 172.21.15.44:53736) 2020-11-20T12:18:09.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f7436ffd700 1 -- 172.21.15.44:0/1149936933 learned_addr learned my addr 172.21.15.44:0/1149936933 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:09.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f742ffff700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 msgr2=0x7f743809f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:09.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f742ffff700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f743809f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:09.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f742ffff700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f743813ac60 msgr2=0x7f74380a17f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:09.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f742ffff700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f743813ac60 0x7f74380a17f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:09.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f742ffff700 1 -- 172.21.15.44:0/1149936933 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438139ce0 con 0x7f74380aa400 2020-11-20T12:18:09.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f7436ffd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f74380aa020 0x7f743809f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:09.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.093+0000 7f742ffff700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 0x7f74380a0f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f7420012000 tx=0x7f7420015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:09.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.094+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420011d50 con 0x7f74380aa400 2020-11-20T12:18:09.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.094+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742001e460 con 0x7f74380aa400 2020-11-20T12:18:09.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.094+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74380a1ce0 con 0x7f74380aa400 2020-11-20T12:18:09.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.094+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742001e800 con 0x7f74380aa400 2020-11-20T12:18:09.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.094+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74380a2140 con 0x7f74380aa400 2020-11-20T12:18:09.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f743da25700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7438134080 0x7f7438134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7438139e70 con 0x7f7438134080 2020-11-20T12:18:09.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f7436ffd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7438134080 0x7f7438134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:09.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f7436ffd700 1 -- 172.21.15.44:0/1149936933 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74380a29c0 con 0x7f7438134080 2020-11-20T12:18:09.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7420025030 con 0x7f74380aa400 2020-11-20T12:18:09.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f7436ffd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7438134080 0x7f7438134460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f742800f770 tx=0x7f7428006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:09.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f742effd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7414056cd0 0x7f7414059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.095+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f7420071480 con 0x7f74380aa400 2020-11-20T12:18:09.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428013740 con 0x7f7438134080 2020-11-20T12:18:09.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f74377fe700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7414056cd0 0x7f7414059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:09.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f74280138a0 con 0x7f7438134080 2020-11-20T12:18:09.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7438134080 msgr2=0x7f7438134460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:09.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f742effd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7438134080 0x7f7438134460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f742800f770 tx=0x7f7428006df0).stop 2020-11-20T12:18:09.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7438134080 msgr2=0x7f7438134460 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:09.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f742effd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7438134080 0x7f7438134460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:09.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.096+0000 7f74377fe700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7414056cd0 0x7f7414059190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f7430006fd0 tx=0x7f7430006d90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.104+0000 7f743da25700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f743814b5e0 0x7f743814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:09.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.104+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74381364d0 con 0x7f743814b5e0 2020-11-20T12:18:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.104+0000 7f7436ffd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f743814b5e0 0x7f743814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f7436ffd700 1 -- 172.21.15.44:0/1149936933 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f74380a9380 con 0x7f743814b5e0 2020-11-20T12:18:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f7436ffd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f743814b5e0 0x7f743814b9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f742800f770 tx=0x7f7428019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:09.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428014440 con 0x7f743814b5e0 2020-11-20T12:18:09.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4765 (secure 0 0 0) 0x7f7428013530 con 0x7f743814b5e0 2020-11-20T12:18:09.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f743814b5e0 msgr2=0x7f743814b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:09.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f742effd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f743814b5e0 0x7f743814b9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f742800f770 tx=0x7f7428019000).stop 2020-11-20T12:18:09.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f742effd700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f743814b5e0 msgr2=0x7f743814b9c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:09.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f742effd700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f743814b5e0 0x7f743814b9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7414056cd0 msgr2=0x7f7414059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f743da25700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7414056cd0 0x7f7414059190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f7430006fd0 tx=0x7f7430006d90).stop 2020-11-20T12:18:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 msgr2=0x7f74380a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f743da25700 1 --2- 172.21.15.44:0/1149936933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74380aa400 0x7f74380a0f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f7420012000 tx=0x7f7420015000).stop 2020-11-20T12:18:09.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.105+0000 7f74377fe700 1 -- 172.21.15.44:0/1149936933 reap_dead start 2020-11-20T12:18:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.106+0000 7f74377fe700 1 -- 172.21.15.44:0/1149936933 reap_dead start 2020-11-20T12:18:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.106+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 shutdown_connections 2020-11-20T12:18:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.106+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 >> 172.21.15.44:0/1149936933 conn(0x7f7438062690 msgr2=0x7f74380a8030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.106+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 shutdown_connections 2020-11-20T12:18:09.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:09.106+0000 7f743da25700 1 -- 172.21.15.44:0/1149936933 wait complete. 2020-11-20T12:18:09.121 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:09.121 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:18:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 144, 2020-11-20T12:18:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:09.122 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:18:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:09.123 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:09.124 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:09.125 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:09.126 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:09.127 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:09.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:09.129 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:09.130 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:09.131 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:09.132 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:09.133 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:09.134 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:09.135 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:09.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:09.137 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:09.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:09.139 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:09.140 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.141 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:09.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:09.145 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:09.146 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:09.147 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:09.148 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.149 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:09.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:09.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:09.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:18:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:09.153 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:18:09.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:18:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.155 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:09.156 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:18:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:18:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:09.157 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:09.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:09.159 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:09.159 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:09.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:18:29.160 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:18:29.161 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:18:29.161 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:18:29.162 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:18:29.164 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:18:29.164 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:18:29.165 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:29.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.304+0000 7f714ec0e700 1 Processor -- start 2020-11-20T12:18:29.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.304+0000 7f714ec0e700 1 -- start start 2020-11-20T12:18:29.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.305+0000 7f714ec0e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480ab0c0 0x7f71480a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.305+0000 7f714ec0e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71480a86d0 0x7f71480ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.305+0000 7f714ec0e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480ae320 con 0x7f71480aace0 2020-11-20T12:18:29.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.305+0000 7f714ec0e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71481400d0 con 0x7f71480a86d0 2020-11-20T12:18:29.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.305+0000 7f714ec0e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480a9470 con 0x7f71480ab0c0 2020-11-20T12:18:29.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f714d1ab700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71480a86d0 0x7f71480ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f7147fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480ab0c0 0x7f71480a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f714d1ab700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f71480a86d0 0x7f71480ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:34370/0 (socket says 172.21.15.44:34370) 2020-11-20T12:18:29.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f7147fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480ab0c0 0x7f71480a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:53756/0 (socket says 172.21.15.44:53756) 2020-11-20T12:18:29.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f714d1ab700 1 -- 172.21.15.44:0/1019073279 learned_addr learned my addr 172.21.15.44:0/1019073279 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:29.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f714c9aa700 1 --1- 172.21.15.44:0/1019073279 >> v1:172.21.15.79:6789/0 conn(0x7f71480aace0 0x7f7148142610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:48940/0 (socket says 172.21.15.44:48940) 2020-11-20T12:18:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f7147fff700 1 -- 172.21.15.44:0/1019073279 >> v2:172.21.15.79:3300/0 conn(0x7f71480a86d0 msgr2=0x7f71480ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f7147fff700 1 --2- 172.21.15.44:0/1019073279 >> v2:172.21.15.79:3300/0 conn(0x7f71480a86d0 0x7f71480ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f7147fff700 1 -- 172.21.15.44:0/1019073279 >> v1:172.21.15.79:6789/0 conn(0x7f71480aace0 legacy=0x7f7148142610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f71477fe700 1 -- 172.21.15.44:0/1019073279 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513080130 0 0) 0x7f71480ae320 con 0x7f71480aace0 2020-11-20T12:18:29.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f7147fff700 1 -- 172.21.15.44:0/1019073279 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480a9690 con 0x7f71480ab0c0 2020-11-20T12:18:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f714d1ab700 1 --2- 172.21.15.44:0/1019073279 >> v2:172.21.15.79:3300/0 conn(0x7f71480a86d0 0x7f71480ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f7147fff700 1 --2- 172.21.15.44:0/1019073279 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480ab0c0 0x7f71480a8f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f713400b9a0 tx=0x7f7134006df0).ready entity=mon.2 client_cookie=f0145cf45177c493 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f71477fe700 1 -- 172.21.15.44:0/1019073279 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134009840 con 0x7f71480ab0c0 2020-11-20T12:18:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f71477fe700 1 -- 172.21.15.44:0/1019073279 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340099a0 con 0x7f71480ab0c0 2020-11-20T12:18:29.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.306+0000 7f71477fe700 1 -- 172.21.15.44:0/1019073279 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134009d20 con 0x7f71480ab0c0 2020-11-20T12:18:29.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 -- 172.21.15.44:0/1019073279 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480ab0c0 msgr2=0x7f71480a8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 --2- 172.21.15.44:0/1019073279 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480ab0c0 0x7f71480a8f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f713400b9a0 tx=0x7f7134006df0).stop 2020-11-20T12:18:29.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 -- 172.21.15.44:0/1019073279 shutdown_connections 2020-11-20T12:18:29.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 --2- 172.21.15.44:0/1019073279 >> v2:172.21.15.79:3300/0 conn(0x7f71480a86d0 0x7f71480ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 --2- 172.21.15.44:0/1019073279 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480ab0c0 0x7f71480a8f80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 -- 172.21.15.44:0/1019073279 >> 172.21.15.44:0/1019073279 conn(0x7f7148060170 msgr2=0x7f7148060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:29.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 -- 172.21.15.44:0/1019073279 shutdown_connections 2020-11-20T12:18:29.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.307+0000 7f714ec0e700 1 -- 172.21.15.44:0/1019073279 wait complete. 2020-11-20T12:18:29.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.308+0000 7f714ec0e700 1 Processor -- start 2020-11-20T12:18:29.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.308+0000 7f714ec0e700 1 -- start start 2020-11-20T12:18:29.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714ec0e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71480a86d0 0x7f7148141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714ec0e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480aace0 0x7f714813ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714ec0e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 0x7f7148136e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714ec0e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148145420 con 0x7f71480ab0c0 2020-11-20T12:18:29.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714ec0e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148144100 con 0x7f71480a86d0 2020-11-20T12:18:29.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714ec0e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7148143e60 con 0x7f71480aace0 2020-11-20T12:18:29.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714d1ab700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 0x7f7148136e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714c9aa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71480a86d0 0x7f7148141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f7147fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480aace0 0x7f714813ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714d1ab700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 0x7f7148136e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42326/0 (socket says 172.21.15.44:42326) 2020-11-20T12:18:29.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.309+0000 7f714d1ab700 1 -- 172.21.15.44:0/3882507306 learned_addr learned my addr 172.21.15.44:0/3882507306 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:29.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f714d1ab700 1 -- 172.21.15.44:0/3882507306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 msgr2=0x7f7148136e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:18:29.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f714d1ab700 1 -- 172.21.15.44:0/3882507306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 msgr2=0x7f7148136e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:18:29.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f714d1ab700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 0x7f7148136e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:29.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f714d1ab700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 0x7f7148136e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:29.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7147fff700 1 -- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71480a86d0 msgr2=0x7f7148141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7147fff700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71480a86d0 0x7f7148141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7147fff700 1 -- 172.21.15.44:0/3882507306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 msgr2=0x7f7148136e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:29.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7147fff700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 0x7f7148136e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7147fff700 1 -- 172.21.15.44:0/3882507306 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148145610 con 0x7f71480aace0 2020-11-20T12:18:29.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f714c9aa700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71480a86d0 0x7f7148141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:29.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7147fff700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480aace0 0x7f714813ed30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f713400b9a0 tx=0x7f7134015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7144ff9700 1 -- 172.21.15.44:0/3882507306 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134011e50 con 0x7f71480aace0 2020-11-20T12:18:29.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7144ff9700 1 -- 172.21.15.44:0/3882507306 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713401e460 con 0x7f71480aace0 2020-11-20T12:18:29.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f7144ff9700 1 -- 172.21.15.44:0/3882507306 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713401e7e0 con 0x7f71480aace0 2020-11-20T12:18:29.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7148146510 con 0x7f71480aace0 2020-11-20T12:18:29.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.310+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7148145ab0 con 0x7f71480aace0 2020-11-20T12:18:29.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.311+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7148145ca0 con 0x7f71480aace0 2020-11-20T12:18:29.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.311+0000 7f7144ff9700 1 -- 172.21.15.44:0/3882507306 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f713401a040 con 0x7f71480aace0 2020-11-20T12:18:29.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.312+0000 7f7144ff9700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7120056d20 0x7f71200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.312+0000 7f7144ff9700 1 -- 172.21.15.44:0/3882507306 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f7134071370 con 0x7f71480aace0 2020-11-20T12:18:29.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.312+0000 7f714c9aa700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7120056d20 0x7f71200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.312+0000 7f714c9aa700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7120056d20 0x7f71200591e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f713c0011a0 tx=0x7f713c007a20).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.316+0000 7f7144ff9700 1 -- 172.21.15.44:0/3882507306 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f71340466d0 con 0x7f71480aace0 2020-11-20T12:18:29.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.583+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f71481457a0 con 0x7f71480aace0 2020-11-20T12:18:29.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.584+0000 7f7144ff9700 1 -- 172.21.15.44:0/3882507306 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f7134023080 con 0x7f71480aace0 2020-11-20T12:18:29.587 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":144,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.585+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7120056d20 msgr2=0x7f71200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.585+0000 7f714ec0e700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7120056d20 0x7f71200591e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f713c0011a0 tx=0x7f713c007a20).stop 2020-11-20T12:18:29.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480aace0 msgr2=0x7f714813ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480aace0 0x7f714813ed30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f713400b9a0 tx=0x7f7134015000).stop 2020-11-20T12:18:29.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 shutdown_connections 2020-11-20T12:18:29.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7120056d20 0x7f71200591e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71480ab0c0 0x7f7148136e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71480a86d0 0x7f7148141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 --2- 172.21.15.44:0/3882507306 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f71480aace0 0x7f714813ed30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 >> 172.21.15.44:0/3882507306 conn(0x7f7148060170 msgr2=0x7f71480b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:29.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 shutdown_connections 2020-11-20T12:18:29.591 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:29.586+0000 7f714ec0e700 1 -- 172.21.15.44:0/3882507306 wait complete. 2020-11-20T12:18:32.601 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:32.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.739+0000 7f9b6d1a8700 1 Processor -- start 2020-11-20T12:18:32.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.739+0000 7f9b6d1a8700 1 -- start start 2020-11-20T12:18:32.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6d1a8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa020 0x7f9b680a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6d1a8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9b680aa400 0x7f9b680a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6d1a8700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b680a9770 con 0x7f9b6813ac60 2020-11-20T12:18:32.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6d1a8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b680a6da0 con 0x7f9b680aa400 2020-11-20T12:18:32.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6d1a8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680a7040 con 0x7f9b680aa020 2020-11-20T12:18:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6759e700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9b6813ac60 0x7f9b680a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46096/0 (socket says 172.21.15.44:46096) 2020-11-20T12:18:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6759e700 1 -- 172.21.15.44:0/818837614 learned_addr learned my addr 172.21.15.44:0/818837614 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b6659c700 1 --2- 172.21.15.44:0/818837614 >> v2:172.21.15.79:3300/0 conn(0x7f9b680aa400 0x7f9b680a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.740+0000 7f9b66d9d700 1 --2- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa020 0x7f9b680a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b6659c700 1 -- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa020 msgr2=0x7f9b680a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b6659c700 1 --2- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa020 0x7f9b680a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b6659c700 1 -- 172.21.15.44:0/818837614 >> v1:172.21.15.44:6789/0 conn(0x7f9b6813ac60 legacy=0x7f9b680a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b6659c700 1 -- 172.21.15.44:0/818837614 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b680a6a90 con 0x7f9b680aa400 2020-11-20T12:18:32.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b66d9d700 1 --2- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa020 0x7f9b680a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b6659c700 1 --2- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b680aa400 0x7f9b680a4fb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9b5c00b9a0 tx=0x7f9b5c006df0).ready entity=mon.1 client_cookie=80d0a74c81eed14a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b65d9b700 1 -- 172.21.15.44:0/818837614 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0097b0 con 0x7f9b680aa400 2020-11-20T12:18:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b65d9b700 1 -- 172.21.15.44:0/818837614 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c009910 con 0x7f9b680aa400 2020-11-20T12:18:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.741+0000 7f9b65d9b700 1 -- 172.21.15.44:0/818837614 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c009c90 con 0x7f9b680aa400 2020-11-20T12:18:32.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b680aa400 msgr2=0x7f9b680a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b680aa400 0x7f9b680a4fb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9b5c00b9a0 tx=0x7f9b5c006df0).stop 2020-11-20T12:18:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/818837614 shutdown_connections 2020-11-20T12:18:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa020 0x7f9b680a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/818837614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b680aa400 0x7f9b680a4fb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/818837614 >> 172.21.15.44:0/818837614 conn(0x7f9b68062690 msgr2=0x7f9b680a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/818837614 shutdown_connections 2020-11-20T12:18:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.742+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/818837614 wait complete. 2020-11-20T12:18:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.743+0000 7f9b6d1a8700 1 Processor -- start 2020-11-20T12:18:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.743+0000 7f9b6d1a8700 1 -- start start 2020-11-20T12:18:32.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b6d1a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 0x7f9b680ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b6d1a8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa400 0x7f9b680673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b6d1a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6813ac60 0x7f9b680a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b6d1a8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b681412a0 con 0x7f9b680aa020 2020-11-20T12:18:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b6d1a8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680ac830 con 0x7f9b6813ac60 2020-11-20T12:18:32.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b6d1a8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680acad0 con 0x7f9b680aa400 2020-11-20T12:18:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b66d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 0x7f9b680ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b66d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 0x7f9b680ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42350/0 (socket says 172.21.15.44:42350) 2020-11-20T12:18:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b66d9d700 1 -- 172.21.15.44:0/706550180 learned_addr learned my addr 172.21.15.44:0/706550180 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b6659c700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa400 0x7f9b680673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b66d9d700 1 -- 172.21.15.44:0/706550180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 msgr2=0x7f9b680ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:18:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b66d9d700 1 -- 172.21.15.44:0/706550180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 msgr2=0x7f9b680ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:18:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b66d9d700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 0x7f9b680ad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.744+0000 7f9b66d9d700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 0x7f9b680ad3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6659c700 1 -- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6813ac60 msgr2=0x7f9b680a2ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:32.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6659c700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6813ac60 0x7f9b680a2ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6659c700 1 -- 172.21.15.44:0/706550180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 msgr2=0x7f9b680ad3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6659c700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 0x7f9b680ad3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6659c700 1 -- 172.21.15.44:0/706550180 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b680acd70 con 0x7f9b680aa400 2020-11-20T12:18:32.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6659c700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa400 0x7f9b680673b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f9b5c012000 tx=0x7f9b5c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b4effd700 1 -- 172.21.15.44:0/706550180 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c011dd0 con 0x7f9b680aa400 2020-11-20T12:18:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b4effd700 1 -- 172.21.15.44:0/706550180 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c01e460 con 0x7f9b680aa400 2020-11-20T12:18:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b4effd700 1 -- 172.21.15.44:0/706550180 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c01e7e0 con 0x7f9b680aa400 2020-11-20T12:18:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b680ac2b0 con 0x7f9b680aa400 2020-11-20T12:18:32.759 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.745+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b680a2fc0 con 0x7f9b680aa400 2020-11-20T12:18:32.760 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.746+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b68061650 con 0x7f9b680aa400 2020-11-20T12:18:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.746+0000 7f9b4effd700 1 -- 172.21.15.44:0/706550180 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f9b5c01a040 con 0x7f9b680aa400 2020-11-20T12:18:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.747+0000 7f9b4effd700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9b44056d20 0x7f9b440591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.747+0000 7f9b4effd700 1 -- 172.21.15.44:0/706550180 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9b5c071180 con 0x7f9b680aa400 2020-11-20T12:18:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.747+0000 7f9b66d9d700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9b44056d20 0x7f9b440591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.747+0000 7f9b66d9d700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9b44056d20 0x7f9b440591e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f9b5800b480 tx=0x7f9b58008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:32.751+0000 7f9b4effd700 1 -- 172.21.15.44:0/706550180 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9b5c046500 con 0x7f9b680aa400 2020-11-20T12:18:33.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.037+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9b680446d0 con 0x7f9b680aa400 2020-11-20T12:18:33.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.037+0000 7f9b4effd700 1 -- 172.21.15.44:0/706550180 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f9b5c023080 con 0x7f9b680aa400 2020-11-20T12:18:33.041 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":144,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:33.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9b44056d20 msgr2=0x7f9b440591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9b44056d20 0x7f9b440591e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f9b5800b480 tx=0x7f9b58008040).stop 2020-11-20T12:18:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa400 msgr2=0x7f9b680673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa400 0x7f9b680673b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f9b5c012000 tx=0x7f9b5c015000).stop 2020-11-20T12:18:33.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 shutdown_connections 2020-11-20T12:18:33.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f9b44056d20 0x7f9b440591e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b6813ac60 0x7f9b680a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:33.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9b680aa020 0x7f9b680ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.044 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 --2- 172.21.15.44:0/706550180 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f9b680aa400 0x7f9b680673b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 >> 172.21.15.44:0/706550180 conn(0x7f9b68062690 msgr2=0x7f9b68134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 shutdown_connections 2020-11-20T12:18:33.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:33.039+0000 7f9b6d1a8700 1 -- 172.21.15.44:0/706550180 wait complete. 2020-11-20T12:18:36.055 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:36.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.197+0000 7fbfe71c6700 1 Processor -- start 2020-11-20T12:18:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.197+0000 7fbfe71c6700 1 -- start start 2020-11-20T12:18:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe71c6700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe71c6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe71c6700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe0062fe0 con 0x7fbfe00ae670 2020-11-20T12:18:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe71c6700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe0063720 con 0x7fbfe00a46f0 2020-11-20T12:18:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe71c6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0063280 con 0x7fbfe00aa370 2020-11-20T12:18:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe61c4700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbfe00ae670 0x7fbfe0138710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46108/0 (socket says 172.21.15.44:46108) 2020-11-20T12:18:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe61c4700 1 -- 172.21.15.44:0/1258764062 learned_addr learned my addr 172.21.15.44:0/1258764062 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:36.205 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:36.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:36.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:36.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:18:36.206 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:36.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:36.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.198+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:18:36.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:18:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:36.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.399+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:18:36.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:18:36.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:36.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:36.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:18:36.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:36.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:36.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:36.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:36.800+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:18:37.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:37.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:18:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:18:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:37.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:37.601+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:18:38.605 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:38.610 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:39.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 msgr2=0x7fbfe0135770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:39.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfe00aa370 0x7fbfe0135770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 msgr2=0x7fbfe0135280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:39.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.44:3300/0 conn(0x7fbfe00a46f0 0x7fbfe0135280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 -- 172.21.15.44:0/1258764062 >> v1:172.21.15.44:6789/0 conn(0x7fbfe00ae670 legacy=0x7fbfe0138710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 -- 172.21.15.44:0/1258764062 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00aea50 con 0x7fbfc0005ea0 2020-11-20T12:18:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfcffff700 1 -- 172.21.15.44:0/1258764062 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe0063720 con 0x7fbfc0003390 2020-11-20T12:18:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.203 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfe61c4700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 msgr2=0x7fbfc0005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfe61c4700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 msgr2=0x7fbfc0005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:39.204 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.198+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:39.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.399+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.399+0000 7fbfe61c4700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 msgr2=0x7fbfc0005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:39.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.399+0000 7fbfe61c4700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 msgr2=0x7fbfc0005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:39.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.399+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:39.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.399+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:18:39.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.800+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.800+0000 7fbfe61c4700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 msgr2=0x7fbfc0005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:18:39.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.800+0000 7fbfe61c4700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 msgr2=0x7fbfc0005810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:18:39.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.800+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:39.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:39.800+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:18:40.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481135457 0 0) 0x7fbfe00aea50 con 0x7fbfc0005ea0 2020-11-20T12:18:40.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc8003240 con 0x7fbfc0005ea0 2020-11-20T12:18:40.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3240431437 0 0) 0x7fbfc8003240 con 0x7fbfc0005ea0 2020-11-20T12:18:40.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 msgr2=0x7fbfc0005810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 --2- 172.21.15.44:0/1258764062 >> v2:172.21.15.79:3300/0 conn(0x7fbfc0003390 0x7fbfc0005810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc000aca0 con 0x7fbfc0005ea0 2020-11-20T12:18:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fbfd000d1c0 con 0x7fbfc0005ea0 2020-11-20T12:18:40.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 >> v1:172.21.15.79:6789/0 conn(0x7fbfc0005ea0 legacy=0x7fbfc0008360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe69c5700 1 -- 172.21.15.44:0/1258764062 reap_dead start 2020-11-20T12:18:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8003730 0x7fbfc8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc80044e0 0x7fbfc8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc8004e50 0x7fbfc800fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0063720 con 0x7fbfc80044e0 2020-11-20T12:18:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8005360 con 0x7fbfc8004e50 2020-11-20T12:18:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8010060 con 0x7fbfc8003730 2020-11-20T12:18:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc80044e0 0x7fbfc8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8003730 0x7fbfc8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.508+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc8004e50 0x7fbfc800fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.509+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8003730 msgr2=0x7fbfc8003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.509+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8003730 0x7fbfc8003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.509+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc8004e50 msgr2=0x7fbfc800fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.509+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc8004e50 0x7fbfc800fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.509+0000 7fbfe59c3700 1 -- 172.21.15.44:0/1258764062 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8003240 con 0x7fbfc80044e0 2020-11-20T12:18:40.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.509+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc80044e0 0x7fbfc8004960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbfe00a7ce0 tx=0x7fbfd0008040).ready entity=mon.0 client_cookie=52694560b0efca86 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.510+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000d840 con 0x7fbfc80044e0 2020-11-20T12:18:40.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.510+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0016460 con 0x7fbfc80044e0 2020-11-20T12:18:40.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.510+0000 7fbfe51c2700 1 -- 172.21.15.44:0/1258764062 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00167e0 con 0x7fbfc80044e0 2020-11-20T12:18:40.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 -- 172.21.15.44:0/1258764062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc80044e0 msgr2=0x7fbfc8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc80044e0 0x7fbfc8004960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbfe00a7ce0 tx=0x7fbfd0008040).stop 2020-11-20T12:18:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 -- 172.21.15.44:0/1258764062 shutdown_connections 2020-11-20T12:18:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc8004e50 0x7fbfc800fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc80044e0 0x7fbfc8004960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/1258764062 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8003730 0x7fbfc8003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 -- 172.21.15.44:0/1258764062 >> 172.21.15.44:0/1258764062 conn(0x7fbfe00654a0 msgr2=0x7fbfe00adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 -- 172.21.15.44:0/1258764062 shutdown_connections 2020-11-20T12:18:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.511+0000 7fbfe71c6700 1 -- 172.21.15.44:0/1258764062 wait complete. 2020-11-20T12:18:40.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 Processor -- start 2020-11-20T12:18:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 -- start start 2020-11-20T12:18:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc8003730 0x7fbfe01393a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc80044e0 0x7fbfe01365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 0x7fbfe0139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe009d470 con 0x7fbfc8003730 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01376c0 con 0x7fbfc80044e0 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.513+0000 7fbfe71c6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0136aa0 con 0x7fbfc8004e50 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc8003730 0x7fbfe01393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc8003730 0x7fbfe01393a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42392/0 (socket says 172.21.15.44:42392) 2020-11-20T12:18:40.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 -- 172.21.15.44:0/2413402554 learned_addr learned my addr 172.21.15.44:0/2413402554 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:40.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 0x7fbfe0139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc80044e0 0x7fbfe01365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe69c5700 1 -- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 msgr2=0x7fbfe0139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:18:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe69c5700 1 -- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 msgr2=0x7fbfe0139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:18:40.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 0x7fbfe0139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:18:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe69c5700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 0x7fbfe0139e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 -- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 msgr2=0x7fbfe0139e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 0x7fbfe0139e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 -- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc80044e0 msgr2=0x7fbfe01365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc80044e0 0x7fbfe01365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 -- 172.21.15.44:0/2413402554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe013c9e0 con 0x7fbfc8003730 2020-11-20T12:18:40.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.514+0000 7fbfe61c4700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc8003730 0x7fbfe01393a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fbfdc007620 tx=0x7fbfdc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.515+0000 7fbfce7fc700 1 -- 172.21.15.44:0/2413402554 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0060d0 con 0x7fbfc8003730 2020-11-20T12:18:40.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.515+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe0137250 con 0x7fbfc8003730 2020-11-20T12:18:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.515+0000 7fbfce7fc700 1 -- 172.21.15.44:0/2413402554 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc006230 con 0x7fbfc8003730 2020-11-20T12:18:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.515+0000 7fbfce7fc700 1 -- 172.21.15.44:0/2413402554 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc004710 con 0x7fbfc8003730 2020-11-20T12:18:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.516+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe013b310 con 0x7fbfc8003730 2020-11-20T12:18:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.516+0000 7fbfce7fc700 1 -- 172.21.15.44:0/2413402554 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbfdc01f030 con 0x7fbfc8003730 2020-11-20T12:18:40.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.517+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0061240 con 0x7fbfc8003730 2020-11-20T12:18:40.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.517+0000 7fbfce7fc700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfb8056d50 0x7fbfb8059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.517+0000 7fbfce7fc700 1 -- 172.21.15.44:0/2413402554 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbfdc06ffa0 con 0x7fbfc8003730 2020-11-20T12:18:40.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.517+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfb8056d50 0x7fbfb8059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.518+0000 7fbfe59c3700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfb8056d50 0x7fbfb8059210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fbfe01329e0 tx=0x7fbfd0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.522+0000 7fbfce7fc700 1 -- 172.21.15.44:0/2413402554 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbfdc045320 con 0x7fbfc8003730 2020-11-20T12:18:40.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.817+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbfe013b500 con 0x7fbfc8003730 2020-11-20T12:18:40.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.817+0000 7fbfce7fc700 1 -- 172.21.15.44:0/2413402554 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fbfdc048940 con 0x7fbfc8003730 2020-11-20T12:18:40.821 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":148,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:40.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfb8056d50 msgr2=0x7fbfb8059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfb8056d50 0x7fbfb8059210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fbfe01329e0 tx=0x7fbfd0008040).stop 2020-11-20T12:18:40.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc8003730 msgr2=0x7fbfe01393a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.823 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc8003730 0x7fbfe01393a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fbfdc007620 tx=0x7fbfdc009040).stop 2020-11-20T12:18:40.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 shutdown_connections 2020-11-20T12:18:40.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfb8056d50 0x7fbfb8059210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbfc8004e50 0x7fbfe0139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc8003730 0x7fbfe01393a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 --2- 172.21.15.44:0/2413402554 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbfc80044e0 0x7fbfe01365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 >> 172.21.15.44:0/2413402554 conn(0x7fbfe00654a0 msgr2=0x7fbfe00608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:40.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 shutdown_connections 2020-11-20T12:18:40.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:40.819+0000 7fbfe71c6700 1 -- 172.21.15.44:0/2413402554 wait complete. 2020-11-20T12:18:43.835 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:43.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.984+0000 7f4c49a46700 1 Processor -- start 2020-11-20T12:18:43.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.984+0000 7f4c49a46700 1 -- start start 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c49a46700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4c440aa020 0x7f4c440a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c49a46700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4c440aa400 0x7f4c440a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c49a46700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c440a9770 con 0x7f4c4413ac60 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c49a46700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440a6da0 con 0x7f4c440aa400 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c49a46700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440a7040 con 0x7f4c440aa020 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c427fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f4c440aa400 0x7f4c440a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c437fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f4c4413ac60 0x7f4c440a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46154/0 (socket says 172.21.15.44:46154) 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c437fe700 1 -- 172.21.15.44:0/3241056152 learned_addr learned my addr 172.21.15.44:0/3241056152 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c42ffd700 1 --2- 172.21.15.44:0/3241056152 >> v2:172.21.15.79:3300/0 conn(0x7f4c440aa020 0x7f4c440a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c427fc700 1 -- 172.21.15.44:0/3241056152 >> v2:172.21.15.79:3300/0 conn(0x7f4c440aa020 msgr2=0x7f4c440a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:43.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c427fc700 1 --2- 172.21.15.44:0/3241056152 >> v2:172.21.15.79:3300/0 conn(0x7f4c440aa020 0x7f4c440a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c427fc700 1 -- 172.21.15.44:0/3241056152 >> v1:172.21.15.44:6789/0 conn(0x7f4c4413ac60 legacy=0x7f4c440a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c427fc700 1 -- 172.21.15.44:0/3241056152 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440a6a90 con 0x7f4c440aa400 2020-11-20T12:18:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.985+0000 7f4c427fc700 1 --2- 172.21.15.44:0/3241056152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c440a4fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f4c440adec0 tx=0x7f4c38006df0).ready entity=mon.0 client_cookie=1d84f75012fe0eb4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c41ffb700 1 -- 172.21.15.44:0/3241056152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380097b0 con 0x7f4c440aa400 2020-11-20T12:18:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c41ffb700 1 -- 172.21.15.44:0/3241056152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38009910 con 0x7f4c440aa400 2020-11-20T12:18:43.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c41ffb700 1 -- 172.21.15.44:0/3241056152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38009c90 con 0x7f4c440aa400 2020-11-20T12:18:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 -- 172.21.15.44:0/3241056152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 msgr2=0x7f4c440a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3241056152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c440a4fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f4c440adec0 tx=0x7f4c38006df0).stop 2020-11-20T12:18:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 -- 172.21.15.44:0/3241056152 shutdown_connections 2020-11-20T12:18:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3241056152 >> v2:172.21.15.79:3300/0 conn(0x7f4c440aa020 0x7f4c440a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3241056152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c440a4fb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 -- 172.21.15.44:0/3241056152 >> 172.21.15.44:0/3241056152 conn(0x7f4c44062690 msgr2=0x7f4c440a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:43.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 -- 172.21.15.44:0/3241056152 shutdown_connections 2020-11-20T12:18:43.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.986+0000 7f4c49a46700 1 -- 172.21.15.44:0/3241056152 wait complete. 2020-11-20T12:18:43.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.987+0000 7f4c49a46700 1 Processor -- start 2020-11-20T12:18:43.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.987+0000 7f4c49a46700 1 -- start start 2020-11-20T12:18:43.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c49a46700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4c440aa020 0x7f4c44114c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c49a46700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c4414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c49a46700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c4413ac60 0x7f4c441344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c49a46700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44067130 con 0x7f4c440aa400 2020-11-20T12:18:43.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c49a46700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440673d0 con 0x7f4c4413ac60 2020-11-20T12:18:43.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c49a46700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44067670 con 0x7f4c440aa020 2020-11-20T12:18:43.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c427fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c4414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c427fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c4414d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42408/0 (socket says 172.21.15.44:42408) 2020-11-20T12:18:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c427fc700 1 -- 172.21.15.44:0/3639807260 learned_addr learned my addr 172.21.15.44:0/3639807260 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c437fe700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c4413ac60 0x7f4c441344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.988+0000 7f4c42ffd700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4c440aa020 0x7f4c44114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c427fc700 1 -- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4c440aa020 msgr2=0x7f4c44114c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c427fc700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4c440aa020 0x7f4c44114c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c427fc700 1 -- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c4413ac60 msgr2=0x7f4c441344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c427fc700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c4413ac60 0x7f4c441344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c427fc700 1 -- 172.21.15.44:0/3639807260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44067b10 con 0x7f4c440aa400 2020-11-20T12:18:44.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c427fc700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c4414d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f4c3800c010 tx=0x7f4c38015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c2affd700 1 -- 172.21.15.44:0/3639807260 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38011dd0 con 0x7f4c440aa400 2020-11-20T12:18:44.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c2affd700 1 -- 172.21.15.44:0/3639807260 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3801e460 con 0x7f4c440aa400 2020-11-20T12:18:44.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c44068090 con 0x7f4c440aa400 2020-11-20T12:18:44.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c2affd700 1 -- 172.21.15.44:0/3639807260 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3801e820 con 0x7f4c440aa400 2020-11-20T12:18:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.989+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c440664c0 con 0x7f4c440aa400 2020-11-20T12:18:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.990+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c44067830 con 0x7f4c440aa400 2020-11-20T12:18:44.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.991+0000 7f4c2affd700 1 -- 172.21.15.44:0/3639807260 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4c38025030 con 0x7f4c440aa400 2020-11-20T12:18:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.991+0000 7f4c2affd700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c20056d20 0x7f4c200591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.991+0000 7f4c2affd700 1 -- 172.21.15.44:0/3639807260 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4c38071c20 con 0x7f4c440aa400 2020-11-20T12:18:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.992+0000 7f4c42ffd700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c20056d20 0x7f4c200591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.992+0000 7f4c42ffd700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c20056d20 0x7f4c200591e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f4c3400ba00 tx=0x7f4c34008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:43.995+0000 7f4c2affd700 1 -- 172.21.15.44:0/3639807260 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4c38047020 con 0x7f4c440aa400 2020-11-20T12:18:44.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.314+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4c44061650 con 0x7f4c440aa400 2020-11-20T12:18:44.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.315+0000 7f4c2affd700 1 -- 172.21.15.44:0/3639807260 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f4c38023070 con 0x7f4c440aa400 2020-11-20T12:18:44.318 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c20056d20 msgr2=0x7f4c200591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c20056d20 0x7f4c200591e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f4c3400ba00 tx=0x7f4c34008040).stop 2020-11-20T12:18:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 msgr2=0x7f4c4414d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c4414d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f4c3800c010 tx=0x7f4c38015000).stop 2020-11-20T12:18:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 shutdown_connections 2020-11-20T12:18:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4c20056d20 0x7f4c200591e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4c4413ac60 0x7f4c441344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4c440aa020 0x7f4c44114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 --2- 172.21.15.44:0/3639807260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c440aa400 0x7f4c4414d650 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 >> 172.21.15.44:0/3639807260 conn(0x7f4c44062690 msgr2=0x7f4c44135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 shutdown_connections 2020-11-20T12:18:44.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.317+0000 7f4c49a46700 1 -- 172.21.15.44:0/3639807260 wait complete. 2020-11-20T12:18:44.333 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:18:44.333 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:18:44.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.476+0000 7ff0e93aa700 1 Processor -- start 2020-11-20T12:18:44.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.476+0000 7ff0e93aa700 1 -- start start 2020-11-20T12:18:44.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e93aa700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0e40ab0e0 0x7ff0e40a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e93aa700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff0e40a86d0 0x7ff0e413ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e93aa700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e41385f0 con 0x7ff0e40aad00 2020-11-20T12:18:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e93aa700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e413e250 con 0x7ff0e40ab0e0 2020-11-20T12:18:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e93aa700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e413e4f0 con 0x7ff0e40a86d0 2020-11-20T12:18:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e27fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0e40ab0e0 0x7ff0e40a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e27fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0e40ab0e0 0x7ff0e40a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42418/0 (socket says 172.21.15.44:42418) 2020-11-20T12:18:44.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1736637805 learned_addr learned my addr 172.21.15.44:0/1736637805 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.477+0000 7ff0e37fe700 1 --2- 172.21.15.44:0/1736637805 >> v2:172.21.15.79:3300/0 conn(0x7ff0e40a86d0 0x7ff0e413ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1736637805 >> v2:172.21.15.79:3300/0 conn(0x7ff0e40a86d0 msgr2=0x7ff0e413ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e27fc700 1 --2- 172.21.15.44:0/1736637805 >> v2:172.21.15.79:3300/0 conn(0x7ff0e40a86d0 0x7ff0e413ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1736637805 >> v1:172.21.15.79:6789/0 conn(0x7ff0e40aad00 legacy=0x7ff0e40abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1736637805 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40ac340 con 0x7ff0e40ab0e0 2020-11-20T12:18:44.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e27fc700 1 --2- 172.21.15.44:0/1736637805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40ab0e0 0x7ff0e40a94d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff0e40a9d40 tx=0x7ff0d8006eb0).ready entity=mon.0 client_cookie=d96690d553c68db4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e1ffb700 1 -- 172.21.15.44:0/1736637805 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8009810 con 0x7ff0e40ab0e0 2020-11-20T12:18:44.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e1ffb700 1 -- 172.21.15.44:0/1736637805 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8009970 con 0x7ff0e40ab0e0 2020-11-20T12:18:44.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.478+0000 7ff0e1ffb700 1 -- 172.21.15.44:0/1736637805 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8009cf0 con 0x7ff0e40ab0e0 2020-11-20T12:18:44.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1736637805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40ab0e0 msgr2=0x7ff0e40a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 --2- 172.21.15.44:0/1736637805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40ab0e0 0x7ff0e40a94d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff0e40a9d40 tx=0x7ff0d8006eb0).stop 2020-11-20T12:18:44.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1736637805 shutdown_connections 2020-11-20T12:18:44.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 --2- 172.21.15.44:0/1736637805 >> v2:172.21.15.79:3300/0 conn(0x7ff0e40a86d0 0x7ff0e413ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 --2- 172.21.15.44:0/1736637805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40ab0e0 0x7ff0e40a94d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1736637805 >> 172.21.15.44:0/1736637805 conn(0x7ff0e4060170 msgr2=0x7ff0e4060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1736637805 shutdown_connections 2020-11-20T12:18:44.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.479+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1736637805 wait complete. 2020-11-20T12:18:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.480+0000 7ff0e93aa700 1 Processor -- start 2020-11-20T12:18:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.480+0000 7ff0e93aa700 1 -- start start 2020-11-20T12:18:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e93aa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0e40a86d0 0x7ff0e409ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e93aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40aad00 0x7ff0e40a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e93aa700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0e40ab0e0 0x7ff0e414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e93aa700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e414a4e0 con 0x7ff0e40aad00 2020-11-20T12:18:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e93aa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e414a780 con 0x7ff0e40a86d0 2020-11-20T12:18:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e93aa700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e414aa20 con 0x7ff0e40ab0e0 2020-11-20T12:18:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e27fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40aad00 0x7ff0e40a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e27fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40aad00 0x7ff0e40a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42424/0 (socket says 172.21.15.44:42424) 2020-11-20T12:18:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1394722669 learned_addr learned my addr 172.21.15.44:0/1394722669 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e2ffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0e40a86d0 0x7ff0e409ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.481+0000 7ff0e37fe700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0e40ab0e0 0x7ff0e414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0e40ab0e0 msgr2=0x7ff0e414d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e27fc700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0e40ab0e0 0x7ff0e414d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0e40a86d0 msgr2=0x7ff0e409ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e27fc700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0e40a86d0 0x7ff0e409ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e27fc700 1 -- 172.21.15.44:0/1394722669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e414aec0 con 0x7ff0e40aad00 2020-11-20T12:18:44.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e2ffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0e40a86d0 0x7ff0e409ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e27fc700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40aad00 0x7ff0e40a0430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff0d8006ef0 tx=0x7ff0d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8011e70 con 0x7ff0e40aad00 2020-11-20T12:18:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d801e460 con 0x7ff0e40aad00 2020-11-20T12:18:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d801e7e0 con 0x7ff0e40aad00 2020-11-20T12:18:44.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e40a7530 con 0x7ff0e40aad00 2020-11-20T12:18:44.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.482+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e414dce0 con 0x7ff0e40aad00 2020-11-20T12:18:44.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.483+0000 7ff0e93aa700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e414a040 0x7ff0e40a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.483+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e405c5f0 con 0x7ff0e414a040 2020-11-20T12:18:44.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.483+0000 7ff0e2ffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e414a040 0x7ff0e40a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.483+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff0d8025030 con 0x7ff0e40aad00 2020-11-20T12:18:44.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.483+0000 7ff0e2ffd700 1 -- 172.21.15.44:0/1394722669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0e414abe0 con 0x7ff0e414a040 2020-11-20T12:18:44.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0e2ffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e414a040 0x7ff0e40a3ff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff0d400e6c0 tx=0x7ff0d4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff0c0056d20 0x7ff0c00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff0d80717d0 con 0x7ff0e40aad00 2020-11-20T12:18:44.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4007030 con 0x7ff0e414a040 2020-11-20T12:18:44.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0e37fe700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff0c0056d20 0x7ff0c00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7ff0d4007190 con 0x7ff0e414a040 2020-11-20T12:18:44.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e414a040 msgr2=0x7ff0e40a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e414a040 0x7ff0e40a3ff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff0d400e6c0 tx=0x7ff0d4009040).stop 2020-11-20T12:18:44.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e414a040 msgr2=0x7ff0e40a3ff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:44.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0caffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e414a040 0x7ff0e40a3ff0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.484+0000 7ff0e37fe700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff0c0056d20 0x7ff0c00591e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff0e40a0240 tx=0x7ff0dc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.497+0000 7ff0e93aa700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40a44e0 0x7ff0e409d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.497+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4069680 con 0x7ff0e40a44e0 2020-11-20T12:18:44.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.497+0000 7ff0e2ffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40a44e0 0x7ff0e409d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.497+0000 7ff0e2ffd700 1 -- 172.21.15.44:0/1394722669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff0e40446d0 con 0x7ff0e40a44e0 2020-11-20T12:18:44.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0e2ffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40a44e0 0x7ff0e409d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff0d4014000 tx=0x7ff0d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d401d040 con 0x7ff0e40a44e0 2020-11-20T12:18:44.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4268 (secure 0 0 0) 0x7ff0d40103d0 con 0x7ff0e40a44e0 2020-11-20T12:18:44.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40a44e0 msgr2=0x7ff0e409d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0caffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40a44e0 0x7ff0e409d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff0d4014000 tx=0x7ff0d4015000).stop 2020-11-20T12:18:44.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0caffd700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40a44e0 msgr2=0x7ff0e409d4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:44.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0caffd700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40a44e0 0x7ff0e409d4e0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0x7ff0d4014000 tx=0x7ff0d4015000).stop 2020-11-20T12:18:44.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff0c0056d20 msgr2=0x7ff0c00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0e93aa700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff0c0056d20 0x7ff0c00591e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7ff0e40a0240 tx=0x7ff0dc008040).stop 2020-11-20T12:18:44.505 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40aad00 msgr2=0x7ff0e40a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0e93aa700 1 --2- 172.21.15.44:0/1394722669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0e40aad00 0x7ff0e40a0430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff0d8006ef0 tx=0x7ff0d8015000).stop 2020-11-20T12:18:44.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.498+0000 7ff0e37fe700 1 -- 172.21.15.44:0/1394722669 reap_dead start 2020-11-20T12:18:44.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.499+0000 7ff0e37fe700 1 -- 172.21.15.44:0/1394722669 reap_dead start 2020-11-20T12:18:44.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.499+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 shutdown_connections 2020-11-20T12:18:44.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.499+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 >> 172.21.15.44:0/1394722669 conn(0x7ff0e4060170 msgr2=0x7ff0e4136190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.499+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 shutdown_connections 2020-11-20T12:18:44.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.499+0000 7ff0e93aa700 1 -- 172.21.15.44:0/1394722669 wait complete. 2020-11-20T12:18:44.507 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:44.508 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:44.508 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:44.508 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:18:44.508 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 152, 2020-11-20T12:18:44.509 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:44.509 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:18:44.509 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:44.509 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:44.509 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.510 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:18:44.510 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:44.510 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:44.510 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:44.510 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.511 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.511 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.511 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.511 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.512 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.512 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.512 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.512 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.512 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:44.513 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:44.513 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.513 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.513 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.513 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.514 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.514 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.514 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.514 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.515 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:44.515 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:44.515 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:44.515 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:44.516 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:44.516 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:44.516 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:44.516 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:44.516 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:44.517 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:44.517 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:44.517 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:44.517 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:44.518 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:44.518 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:44.518 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.518 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.518 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.519 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.519 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.519 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.519 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.519 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.520 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.520 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:44.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.520 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:44.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.521 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:44.521 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:44.521 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.521 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.522 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:44.522 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.523 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:44.523 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.523 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.523 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.524 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:44.524 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:44.524 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.524 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.525 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.525 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:44.526 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:44.526 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.526 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.527 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:44.527 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.528 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:44.528 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.528 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.528 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.529 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:44.529 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:44.529 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.529 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.529 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.530 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:44.530 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:44.530 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.531 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.531 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:44.532 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.533 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:44.533 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.533 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.533 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.534 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:44.534 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:44.534 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.534 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.534 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.535 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.535 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.535 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:44.535 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:44.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.536 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.536 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.536 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:44.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.537 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:44.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.537 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.537 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.538 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:44.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.538 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.538 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:44.539 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:44.540 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:18:44.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16d227b700 1 Processor -- start 2020-11-20T12:18:44.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16d227b700 1 -- start start 2020-11-20T12:18:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16d227b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f16cc0aad20 0x7f16cc138320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16d227b700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc0ab310 con 0x7f16cc0a9460 2020-11-20T12:18:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16d227b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc0ab5b0 con 0x7f16cc13b780 2020-11-20T12:18:44.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16d227b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc13ac80 con 0x7f16cc0aad20 2020-11-20T12:18:44.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16cbfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f16cc0a9460 0x7f16cc13b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46182/0 (socket says 172.21.15.44:46182) 2020-11-20T12:18:44.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.685+0000 7f16cbfff700 1 -- 172.21.15.44:0/3450405994 learned_addr learned my addr 172.21.15.44:0/3450405994 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:44.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16d0818700 1 --2- 172.21.15.44:0/3450405994 >> v2:172.21.15.79:3300/0 conn(0x7f16cc0aad20 0x7f16cc138320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16cb7fe700 1 -- 172.21.15.44:0/3450405994 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581392455 0 0) 0x7f16cc0ab310 con 0x7f16cc0a9460 2020-11-20T12:18:44.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16cb7fe700 1 -- 172.21.15.44:0/3450405994 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16ac003240 con 0x7f16cc0a9460 2020-11-20T12:18:44.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16cb7fe700 1 -- 172.21.15.44:0/3450405994 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1975393094 0 0) 0x7f16ac003240 con 0x7f16cc0a9460 2020-11-20T12:18:44.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16d0818700 1 -- 172.21.15.44:0/3450405994 >> v1:172.21.15.79:6789/0 conn(0x7f16cc13b780 legacy=0x7f16cc137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.696 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16d0818700 1 -- 172.21.15.44:0/3450405994 >> v1:172.21.15.44:6789/0 conn(0x7f16cc0a9460 legacy=0x7f16cc13b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16d0818700 1 -- 172.21.15.44:0/3450405994 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0a9840 con 0x7f16cc0aad20 2020-11-20T12:18:44.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16d0818700 1 --2- 172.21.15.44:0/3450405994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc0aad20 0x7f16cc138320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f16c4009c90 tx=0x7f16c40091d0).ready entity=mon.1 client_cookie=6fb611cbf5be43cd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16cb7fe700 1 -- 172.21.15.44:0/3450405994 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0014e0 con 0x7f16cc0aad20 2020-11-20T12:18:44.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16cb7fe700 1 -- 172.21.15.44:0/3450405994 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c400b760 con 0x7f16cc0aad20 2020-11-20T12:18:44.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.686+0000 7f16cb7fe700 1 -- 172.21.15.44:0/3450405994 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c400bae0 con 0x7f16cc0aad20 2020-11-20T12:18:44.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.687+0000 7f16d227b700 1 -- 172.21.15.44:0/3450405994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc0aad20 msgr2=0x7f16cc138320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.687+0000 7f16d227b700 1 --2- 172.21.15.44:0/3450405994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc0aad20 0x7f16cc138320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f16c4009c90 tx=0x7f16c40091d0).stop 2020-11-20T12:18:44.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.687+0000 7f16d227b700 1 -- 172.21.15.44:0/3450405994 shutdown_connections 2020-11-20T12:18:44.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.687+0000 7f16d227b700 1 --2- 172.21.15.44:0/3450405994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc0aad20 0x7f16cc138320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.687+0000 7f16d227b700 1 -- 172.21.15.44:0/3450405994 >> 172.21.15.44:0/3450405994 conn(0x7f16cc0621c0 msgr2=0x7f16cc060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.687+0000 7f16d227b700 1 -- 172.21.15.44:0/3450405994 shutdown_connections 2020-11-20T12:18:44.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.687+0000 7f16d227b700 1 -- 172.21.15.44:0/3450405994 wait complete. 2020-11-20T12:18:44.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.688+0000 7f16d227b700 1 Processor -- start 2020-11-20T12:18:44.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.688+0000 7f16d227b700 1 -- start start 2020-11-20T12:18:44.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.688+0000 7f16d227b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16cc0a9460 0x7f16cc0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d227b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc0aad20 0x7f16cc131610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d227b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc13b780 0x7f16cc131b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d227b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc131ff0 con 0x7f16cc0aad20 2020-11-20T12:18:44.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d227b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0b0ba0 con 0x7f16cc13b780 2020-11-20T12:18:44.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d227b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0a6a70 con 0x7f16cc0a9460 2020-11-20T12:18:44.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d0818700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc13b780 0x7f16cc131b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d0818700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc13b780 0x7f16cc131b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:34492/0 (socket says 172.21.15.44:34492) 2020-11-20T12:18:44.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16d0818700 1 -- 172.21.15.44:0/3672967405 learned_addr learned my addr 172.21.15.44:0/3672967405 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:44.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c3fff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc0aad20 0x7f16cc131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16cbfff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16cc0a9460 0x7f16cc0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c3fff700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16cc0a9460 msgr2=0x7f16cc0b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c3fff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f16cc0a9460 0x7f16cc0b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c3fff700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc13b780 msgr2=0x7f16cc131b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c3fff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16cc13b780 0x7f16cc131b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c3fff700 1 -- 172.21.15.44:0/3672967405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0a6d10 con 0x7f16cc0aad20 2020-11-20T12:18:44.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c3fff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc0aad20 0x7f16cc131610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f16b400c6e0 tx=0x7f16b4007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.689+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4007020 con 0x7f16cc0aad20 2020-11-20T12:18:44.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.690+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b4007180 con 0x7f16cc0aad20 2020-11-20T12:18:44.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.690+0000 7f16d227b700 1 -- 172.21.15.44:0/3672967405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc0a6fb0 con 0x7f16cc0aad20 2020-11-20T12:18:44.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.690+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b40064f0 con 0x7f16cc0aad20 2020-11-20T12:18:44.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.690+0000 7f16d227b700 1 -- 172.21.15.44:0/3672967405 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc0a67e0 con 0x7f16cc0aad20 2020-11-20T12:18:44.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16c1ffb700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c002a10 0x7f169c004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c007b70 con 0x7f169c002a10 2020-11-20T12:18:44.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16cbfff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c002a10 0x7f169c004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f16b401a060 con 0x7f16cc0aad20 2020-11-20T12:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16cbfff700 1 -- 172.21.15.44:0/3672967405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f169c007d30 con 0x7f169c002a10 2020-11-20T12:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16cbfff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c002a10 0x7f169c004ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f16cc0addc0 tx=0x7f16bc000b80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16c8ff9700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f16a8056d20 0x7f16a80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f16b406f870 con 0x7f16cc0aad20 2020-11-20T12:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16d0818700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f16a8056d20 0x7f16a80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.691+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00daa0 con 0x7f169c002a10 2020-11-20T12:18:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.692+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f16bc00dc00 con 0x7f169c002a10 2020-11-20T12:18:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.692+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c002a10 msgr2=0x7f169c004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.692+0000 7f16c8ff9700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c002a10 0x7f169c004ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f16cc0addc0 tx=0x7f16bc000b80).stop 2020-11-20T12:18:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.692+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c002a10 msgr2=0x7f169c004ed0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.692+0000 7f16c8ff9700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c002a10 0x7f169c004ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.692+0000 7f16d0818700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f16a8056d20 0x7f16a80591e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f16cc0a9230 tx=0x7f16c4016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.701+0000 7f16c1ffb700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c006bb0 0x7f169c007040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.701+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c007650 con 0x7f169c006bb0 2020-11-20T12:18:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.702+0000 7f16cbfff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c006bb0 0x7f169c007040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.702+0000 7f16cbfff700 1 -- 172.21.15.44:0/3672967405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f169c007810 con 0x7f169c006bb0 2020-11-20T12:18:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.702+0000 7f16cbfff700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c006bb0 0x7f169c007040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f16bc015000 tx=0x7f16bc00a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.702+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc014770 con 0x7f169c006bb0 2020-11-20T12:18:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4436 (secure 0 0 0) 0x7f16bc00d890 con 0x7f169c006bb0 2020-11-20T12:18:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c006bb0 msgr2=0x7f169c007040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c8ff9700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c006bb0 0x7f169c007040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f16bc015000 tx=0x7f16bc00a000).stop 2020-11-20T12:18:44.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c8ff9700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c006bb0 msgr2=0x7f169c007040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c8ff9700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f169c006bb0 0x7f169c007040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f16a8056d20 msgr2=0x7f16a80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f16a8056d20 0x7f16a80591e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f16cc0a9230 tx=0x7f16c4016000).stop 2020-11-20T12:18:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc0aad20 msgr2=0x7f16cc131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 --2- 172.21.15.44:0/3672967405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f16cc0aad20 0x7f16cc131610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f16b400c6e0 tx=0x7f16b4007b90).stop 2020-11-20T12:18:44.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16d0818700 1 -- 172.21.15.44:0/3672967405 reap_dead start 2020-11-20T12:18:44.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16d0818700 1 -- 172.21.15.44:0/3672967405 reap_dead start 2020-11-20T12:18:44.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 shutdown_connections 2020-11-20T12:18:44.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 >> 172.21.15.44:0/3672967405 conn(0x7f16cc0621c0 msgr2=0x7f16cc0ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 shutdown_connections 2020-11-20T12:18:44.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.703+0000 7f16c1ffb700 1 -- 172.21.15.44:0/3672967405 wait complete. 2020-11-20T12:18:44.720 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:44.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:44.720 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:44.720 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:18:44.721 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 152, 2020-11-20T12:18:44.721 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:44.721 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:18:44.721 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:44.722 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:44.722 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.722 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:18:44.722 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:44.723 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:44.723 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:44.723 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.723 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.724 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.724 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.724 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.724 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.724 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.725 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.725 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:44.725 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:44.725 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.726 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.726 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.726 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.726 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.727 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.727 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.727 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.727 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:44.728 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:44.728 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:44.728 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:44.728 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:44.729 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:44.729 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:44.729 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:44.729 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:44.730 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:44.730 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:44.730 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:44.730 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:44.731 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:44.731 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:44.731 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.731 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.732 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.732 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.732 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.733 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.733 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.733 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.733 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:44.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:44.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.734 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:44.734 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:44.735 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.735 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.735 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.736 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:44.736 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.736 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.737 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:44.737 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.737 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.738 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.738 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:44.738 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:44.738 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.739 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.739 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.739 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:44.740 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:44.740 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.740 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.741 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:44.741 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.742 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.742 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:44.742 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.742 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.743 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.743 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:44.743 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:44.743 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.744 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.744 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.745 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:44.745 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:44.745 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.745 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.746 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:44.746 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.747 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.747 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:44.747 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.748 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.749 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:44.749 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:44.749 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.750 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.750 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.750 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.750 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.751 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:44.751 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:44.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.752 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.752 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.752 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:44.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.753 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:18:44.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.753 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.753 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.754 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:44.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.754 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.754 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:44.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.755 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.755 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.755 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:44.755 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.756 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.756 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:44.756 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:44.758 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:18:44.899 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.896+0000 7ff60813c700 1 Processor -- start 2020-11-20T12:18:44.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.896+0000 7ff60813c700 1 -- start start 2020-11-20T12:18:44.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff60813c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff6000aa020 0x7ff6000a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff60813c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff60013ac60 0x7ff6000a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff60813c700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000a9770 con 0x7ff6000aa400 2020-11-20T12:18:44.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff60813c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000a56c0 con 0x7ff60013ac60 2020-11-20T12:18:44.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff60813c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000a6da0 con 0x7ff6000aa020 2020-11-20T12:18:44.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff6066d9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff60013ac60 0x7ff6000a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff6056d7700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff6000aa400 0x7ff6000a4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46202/0 (socket says 172.21.15.44:46202) 2020-11-20T12:18:44.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff6056d7700 1 -- 172.21.15.44:0/1928914761 learned_addr learned my addr 172.21.15.44:0/1928914761 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:44.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff605ed8700 1 --2- 172.21.15.44:0/1928914761 >> v2:172.21.15.79:3300/0 conn(0x7ff6000aa020 0x7ff6000a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff6066d9700 1 -- 172.21.15.44:0/1928914761 >> v2:172.21.15.79:3300/0 conn(0x7ff6000aa020 msgr2=0x7ff6000a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:44.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.897+0000 7ff6066d9700 1 --2- 172.21.15.44:0/1928914761 >> v2:172.21.15.79:3300/0 conn(0x7ff6000aa020 0x7ff6000a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff6066d9700 1 -- 172.21.15.44:0/1928914761 >> v1:172.21.15.44:6789/0 conn(0x7ff6000aa400 legacy=0x7ff6000a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff604ed6700 1 -- 172.21.15.44:0/1928914761 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540224894 0 0) 0x7ff5f00010d0 con 0x7ff6000aa400 2020-11-20T12:18:44.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff6066d9700 1 -- 172.21.15.44:0/1928914761 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000a7040 con 0x7ff60013ac60 2020-11-20T12:18:44.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff6066d9700 1 --2- 172.21.15.44:0/1928914761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff6000a3700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff6000adec0 tx=0x7ff5f80091d0).ready entity=mon.0 client_cookie=4f0e1c1a927706d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff604ed6700 1 -- 172.21.15.44:0/1928914761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800ba70 con 0x7ff60013ac60 2020-11-20T12:18:44.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff604ed6700 1 -- 172.21.15.44:0/1928914761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f800bbd0 con 0x7ff60013ac60 2020-11-20T12:18:44.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff604ed6700 1 -- 172.21.15.44:0/1928914761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8011400 con 0x7ff60013ac60 2020-11-20T12:18:44.906 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff60813c700 1 -- 172.21.15.44:0/1928914761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 msgr2=0x7ff6000a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff60813c700 1 --2- 172.21.15.44:0/1928914761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff6000a3700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff6000adec0 tx=0x7ff5f80091d0).stop 2020-11-20T12:18:44.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff60813c700 1 -- 172.21.15.44:0/1928914761 shutdown_connections 2020-11-20T12:18:44.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff60813c700 1 --2- 172.21.15.44:0/1928914761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff6000a3700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff60813c700 1 --2- 172.21.15.44:0/1928914761 >> v2:172.21.15.79:3300/0 conn(0x7ff6000aa020 0x7ff6000a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.898+0000 7ff60813c700 1 -- 172.21.15.44:0/1928914761 >> 172.21.15.44:0/1928914761 conn(0x7ff600062690 msgr2=0x7ff6000a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.899+0000 7ff60813c700 1 -- 172.21.15.44:0/1928914761 shutdown_connections 2020-11-20T12:18:44.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.899+0000 7ff60813c700 1 -- 172.21.15.44:0/1928914761 wait complete. 2020-11-20T12:18:44.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.900+0000 7ff60813c700 1 Processor -- start 2020-11-20T12:18:44.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.900+0000 7ff60813c700 1 -- start start 2020-11-20T12:18:44.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff60813c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6000aa020 0x7ff60012a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff60813c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6000aa400 0x7ff600125bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff60813c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff600129ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff60813c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001291c0 con 0x7ff60013ac60 2020-11-20T12:18:44.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff60813c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60012b210 con 0x7ff6000aa020 2020-11-20T12:18:44.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff60813c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60012b4b0 con 0x7ff6000aa400 2020-11-20T12:18:44.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff600129ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff600129ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42462/0 (socket says 172.21.15.44:42462) 2020-11-20T12:18:44.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 -- 172.21.15.44:0/1967325437 learned_addr learned my addr 172.21.15.44:0/1967325437 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:44.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff605ed8700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6000aa020 0x7ff60012a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6056d7700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6000aa400 0x7ff600125bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6000aa400 msgr2=0x7ff600125bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff6000aa400 0x7ff600125bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6000aa020 msgr2=0x7ff60012a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6000aa020 0x7ff60012a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 -- 172.21.15.44:0/1967325437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60012b750 con 0x7ff60013ac60 2020-11-20T12:18:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff605ed8700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff6000aa020 0x7ff60012a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:44.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.901+0000 7ff6066d9700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff600129ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff5f8009fd0 tx=0x7ff5f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.902+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f801c6c0 con 0x7ff60013ac60 2020-11-20T12:18:44.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.902+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f801c820 con 0x7ff60013ac60 2020-11-20T12:18:44.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.902+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600067c20 con 0x7ff60013ac60 2020-11-20T12:18:44.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.902+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f801cba0 con 0x7ff60013ac60 2020-11-20T12:18:44.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.902+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6001260a0 con 0x7ff60013ac60 2020-11-20T12:18:44.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.903+0000 7ff60813c700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff60012a390 0x7ff600126e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.903+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff600062b30 con 0x7ff60012a390 2020-11-20T12:18:44.925 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.903+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff5f8024030 con 0x7ff60013ac60 2020-11-20T12:18:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.903+0000 7ff605ed8700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff60012a390 0x7ff600126e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.903+0000 7ff5edffb700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5d8056d50 0x7ff5d8059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.903+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff5f80719c0 con 0x7ff60013ac60 2020-11-20T12:18:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.903+0000 7ff605ed8700 1 -- 172.21.15.44:0/1967325437 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6000a8fa0 con 0x7ff60012a390 2020-11-20T12:18:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff6056d7700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5d8056d50 0x7ff5d8059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff605ed8700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff60012a390 0x7ff600126e80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7ff5fc010ad0 tx=0x7ff5fc008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00eac0 con 0x7ff60012a390 2020-11-20T12:18:44.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff6056d7700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5d8056d50 0x7ff5d8059210 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7ff5f0007fd0 tx=0x7ff5f0007860).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7ff5fc00ec20 con 0x7ff60012a390 2020-11-20T12:18:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff60012a390 msgr2=0x7ff600126e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff5edffb700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff60012a390 0x7ff600126e80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7ff5fc010ad0 tx=0x7ff5fc008040).stop 2020-11-20T12:18:44.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff60012a390 msgr2=0x7ff600126e80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:44.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.904+0000 7ff5edffb700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff60012a390 0x7ff600126e80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.914+0000 7ff60813c700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff600127370 0x7ff60014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.914+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff600067c20 con 0x7ff600127370 2020-11-20T12:18:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.914+0000 7ff605ed8700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff600127370 0x7ff60014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff605ed8700 1 -- 172.21.15.44:0/1967325437 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff600061650 con 0x7ff600127370 2020-11-20T12:18:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff605ed8700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff600127370 0x7ff60014b430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff5fc010ad0 tx=0x7ff5fc01c000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0148d0 con 0x7ff600127370 2020-11-20T12:18:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7ff5fc00e8b0 con 0x7ff600127370 2020-11-20T12:18:44.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff600127370 msgr2=0x7ff60014b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff5edffb700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff600127370 0x7ff60014b430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff5fc010ad0 tx=0x7ff5fc01c000).stop 2020-11-20T12:18:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff5edffb700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff600127370 msgr2=0x7ff60014b430 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.915+0000 7ff5edffb700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff600127370 0x7ff60014b430 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5d8056d50 msgr2=0x7ff5d8059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff5d8056d50 0x7ff5d8059210 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7ff5f0007fd0 tx=0x7ff5f0007860).stop 2020-11-20T12:18:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 msgr2=0x7ff600129ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff600129ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff5f8009fd0 tx=0x7ff5f8016000).stop 2020-11-20T12:18:44.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff6066d9700 1 -- 172.21.15.44:0/1967325437 reap_dead start 2020-11-20T12:18:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 shutdown_connections 2020-11-20T12:18:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 --2- 172.21.15.44:0/1967325437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff60013ac60 0x7ff600129ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 >> 172.21.15.44:0/1967325437 conn(0x7ff600062690 msgr2=0x7ff600135a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 shutdown_connections 2020-11-20T12:18:44.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:44.916+0000 7ff60813c700 1 -- 172.21.15.44:0/1967325437 wait complete. 2020-11-20T12:18:44.936 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:44.936 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:44.936 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:44.937 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:18:44.937 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 152, 2020-11-20T12:18:44.937 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:44.938 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:18:44.938 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:44.939 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:44.939 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.939 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:18:44.940 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:44.940 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:44.940 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:44.940 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.941 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.941 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.941 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.941 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.941 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.942 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.942 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.942 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.942 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:44.942 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:44.943 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.943 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.943 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.943 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.943 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.944 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.944 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.944 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.944 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.945 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:44.945 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:44.945 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:44.945 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:44.946 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:44.946 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:44.946 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:44.946 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:44.946 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:44.947 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:44.947 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:44.947 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:44.947 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:44.948 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:44.948 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:44.948 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:44.948 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:44.949 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:44.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:44.949 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:44.949 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:44.949 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:44.950 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:44.950 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.951 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:44.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.952 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:44.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.953 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:44.953 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:44.953 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.954 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.954 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.954 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:44.954 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.955 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:44.955 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.956 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.956 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.956 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:44.956 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:44.957 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.957 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.957 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.958 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:44.958 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:44.958 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.958 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.959 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:44.959 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.960 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.960 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:44.960 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.960 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.961 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.961 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:44.961 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:44.962 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.962 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.962 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.963 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:44.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:44.964 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:44.964 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:44.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:44.965 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:44.966 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:44.967 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:44.967 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:44.968 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.968 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.969 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:44.970 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:44.970 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:44.970 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:44.971 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:44.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.971 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.972 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:44.973 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:44.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.973 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.974 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.974 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:44.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.975 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.975 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:18:44.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.976 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.979 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.979 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.979 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.981 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.981 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.981 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:18:44.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.982 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:44.983 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:18:44.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:44.983 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:44.984 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:44.984 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:44.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:44.985 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:44.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:44.986 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:44.986 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:44.987 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:18:44.987 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:18:45.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.125+0000 7f36be033700 1 Processor -- start 2020-11-20T12:18:45.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.125+0000 7f36be033700 1 -- start start 2020-11-20T12:18:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36be033700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b80a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36be033700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f36b80a86d0 0x7f36b813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36be033700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b81385f0 con 0x7f36b80aad00 2020-11-20T12:18:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36be033700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b813e250 con 0x7f36b80a86d0 2020-11-20T12:18:45.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36be033700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b813e4f0 con 0x7f36b80ab0e0 2020-11-20T12:18:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36b77fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f36b80aad00 0x7f36b80abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46218/0 (socket says 172.21.15.44:46218) 2020-11-20T12:18:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36b77fe700 1 -- 172.21.15.44:0/1111060656 learned_addr learned my addr 172.21.15.44:0/1111060656 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.126+0000 7f36b6ffd700 1 --2- 172.21.15.44:0/1111060656 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b80a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:45.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b7fff700 1 --2- 172.21.15.44:0/1111060656 >> v2:172.21.15.44:3300/0 conn(0x7f36b80a86d0 0x7f36b813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b67fc700 1 -- 172.21.15.44:0/1111060656 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840381584 0 0) 0x7f36b81385f0 con 0x7f36b80aad00 2020-11-20T12:18:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b67fc700 1 -- 172.21.15.44:0/1111060656 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36a4003240 con 0x7f36b80aad00 2020-11-20T12:18:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b6ffd700 1 -- 172.21.15.44:0/1111060656 >> v2:172.21.15.44:3300/0 conn(0x7f36b80a86d0 msgr2=0x7f36b813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b6ffd700 1 --2- 172.21.15.44:0/1111060656 >> v2:172.21.15.44:3300/0 conn(0x7f36b80a86d0 0x7f36b813ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b6ffd700 1 -- 172.21.15.44:0/1111060656 >> v1:172.21.15.44:6789/0 conn(0x7f36b80aad00 legacy=0x7f36b80abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b6ffd700 1 -- 172.21.15.44:0/1111060656 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b80ac340 con 0x7f36b80ab0e0 2020-11-20T12:18:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b6ffd700 1 --2- 172.21.15.44:0/1111060656 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b80a94d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f36ac00b9a0 tx=0x7f36ac006df0).ready entity=mon.2 client_cookie=16a160149ed09c4a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b67fc700 1 -- 172.21.15.44:0/1111060656 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac009840 con 0x7f36b80ab0e0 2020-11-20T12:18:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b67fc700 1 -- 172.21.15.44:0/1111060656 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac0099a0 con 0x7f36b80ab0e0 2020-11-20T12:18:45.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.127+0000 7f36b67fc700 1 -- 172.21.15.44:0/1111060656 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac009d20 con 0x7f36b80ab0e0 2020-11-20T12:18:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 -- 172.21.15.44:0/1111060656 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 msgr2=0x7f36b80a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 --2- 172.21.15.44:0/1111060656 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b80a94d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f36ac00b9a0 tx=0x7f36ac006df0).stop 2020-11-20T12:18:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 -- 172.21.15.44:0/1111060656 shutdown_connections 2020-11-20T12:18:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 --2- 172.21.15.44:0/1111060656 >> v2:172.21.15.44:3300/0 conn(0x7f36b80a86d0 0x7f36b813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 --2- 172.21.15.44:0/1111060656 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b80a94d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 -- 172.21.15.44:0/1111060656 >> 172.21.15.44:0/1111060656 conn(0x7f36b8060170 msgr2=0x7f36b8060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 -- 172.21.15.44:0/1111060656 shutdown_connections 2020-11-20T12:18:45.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.128+0000 7f36be033700 1 -- 172.21.15.44:0/1111060656 wait complete. 2020-11-20T12:18:45.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.129+0000 7f36be033700 1 Processor -- start 2020-11-20T12:18:45.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.129+0000 7f36be033700 1 -- start start 2020-11-20T12:18:45.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36be033700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b80a86d0 0x7f36b813ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:45.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36be033700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36b80aad00 0x7f36b814a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:45.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36be033700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b8139780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:45.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36be033700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8149de0 con 0x7f36b80a86d0 2020-11-20T12:18:45.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36be033700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b813b200 con 0x7f36b80aad00 2020-11-20T12:18:45.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36be033700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b80a61d0 con 0x7f36b80ab0e0 2020-11-20T12:18:45.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b80a86d0 0x7f36b813ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:45.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b80a86d0 0x7f36b813ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42476/0 (socket says 172.21.15.44:42476) 2020-11-20T12:18:45.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 -- 172.21.15.44:0/283974203 learned_addr learned my addr 172.21.15.44:0/283974203 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:45.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b6ffd700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36b80aad00 0x7f36b814a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:45.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b7fff700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b8139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:45.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 -- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 msgr2=0x7f36b8139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b8139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 -- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36b80aad00 msgr2=0x7f36b814a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36b80aad00 0x7f36b814a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 -- 172.21.15.44:0/283974203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b814a040 con 0x7f36b80a86d0 2020-11-20T12:18:45.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b6ffd700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36b80aad00 0x7f36b814a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:45.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.130+0000 7f36b77fe700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b80a86d0 0x7f36b813ad10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f36a800c6f0 tx=0x7f36a8007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:45.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.131+0000 7f36977fe700 1 -- 172.21.15.44:0/283974203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8007320 con 0x7f36b80a86d0 2020-11-20T12:18:45.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.131+0000 7f36977fe700 1 -- 172.21.15.44:0/283974203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a8007480 con 0x7f36b80a86d0 2020-11-20T12:18:45.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.131+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b813a290 con 0x7f36b80a86d0 2020-11-20T12:18:45.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.131+0000 7f36977fe700 1 -- 172.21.15.44:0/283974203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8005800 con 0x7f36b80a86d0 2020-11-20T12:18:45.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.131+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b814ac10 con 0x7f36b80a86d0 2020-11-20T12:18:45.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.132+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b8069680 con 0x7f36b80a86d0 2020-11-20T12:18:45.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.132+0000 7f36977fe700 1 -- 172.21.15.44:0/283974203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f36a8010030 con 0x7f36b80a86d0 2020-11-20T12:18:45.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.132+0000 7f36977fe700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3690056d20 0x7f36900591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:45.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.133+0000 7f36977fe700 1 -- 172.21.15.44:0/283974203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f36a806fef0 con 0x7f36b80a86d0 2020-11-20T12:18:45.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.133+0000 7f36b6ffd700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3690056d20 0x7f36900591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:45.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.133+0000 7f36b6ffd700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3690056d20 0x7f36900591e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f36ac006e30 tx=0x7f36ac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:45.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.137+0000 7f36977fe700 1 -- 172.21.15.44:0/283974203 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f36a8045240 con 0x7f36b80a86d0 2020-11-20T12:18:45.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.425+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f36b80446d0 con 0x7f36b80a86d0 2020-11-20T12:18:45.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.426+0000 7f36977fe700 1 -- 172.21.15.44:0/283974203 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f36a8048860 con 0x7f36b80a86d0 2020-11-20T12:18:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.427+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3690056d20 msgr2=0x7f36900591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.427+0000 7f36be033700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3690056d20 0x7f36900591e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f36ac006e30 tx=0x7f36ac015000).stop 2020-11-20T12:18:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.427+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b80a86d0 msgr2=0x7f36b813ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.427+0000 7f36be033700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b80a86d0 0x7f36b813ad10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f36a800c6f0 tx=0x7f36a8007ad0).stop 2020-11-20T12:18:45.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 shutdown_connections 2020-11-20T12:18:45.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3690056d20 0x7f36900591e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f36b80ab0e0 0x7f36b8139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b80a86d0 0x7f36b813ad10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.432 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 --2- 172.21.15.44:0/283974203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36b80aad00 0x7f36b814a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 >> 172.21.15.44:0/283974203 conn(0x7f36b8060170 msgr2=0x7f36b80af150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:45.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 shutdown_connections 2020-11-20T12:18:45.433 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:45.428+0000 7f36be033700 1 -- 172.21.15.44:0/283974203 wait complete. 2020-11-20T12:18:45.445 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:18:55.447 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:18:55.447 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:55.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.596+0000 7f06a1856700 1 Processor -- start 2020-11-20T12:18:55.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.596+0000 7f06a1856700 1 -- start start 2020-11-20T12:18:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f06a1856700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f069c061340 0x7f069c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f06a1856700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c13c200 0x7f069c13c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f06a1856700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0a7b90 con 0x7f069c0a80e0 2020-11-20T12:18:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f06a1856700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c064f60 con 0x7f069c061340 2020-11-20T12:18:55.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f06a1856700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0687d0 con 0x7f069c13c200 2020-11-20T12:18:55.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f069affd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f069c061340 0x7f069c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f069b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f069c0a80e0 0x7f069c0a8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49112/0 (socket says 172.21.15.44:49112) 2020-11-20T12:18:55.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f069b7fe700 1 -- 172.21.15.44:0/1190438666 learned_addr learned my addr 172.21.15.44:0/1190438666 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:55.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f0693fff700 1 --2- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c13c200 0x7f069c13c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f069affd700 1 -- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c13c200 msgr2=0x7f069c13c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:55.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f069affd700 1 --2- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c13c200 0x7f069c13c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f069affd700 1 -- 172.21.15.44:0/1190438666 >> v1:172.21.15.79:6789/0 conn(0x7f069c0a80e0 legacy=0x7f069c0a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.597+0000 7f069a7fc700 1 -- 172.21.15.44:0/1190438666 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207889615 0 0) 0x7f069c0a7b90 con 0x7f069c0a80e0 2020-11-20T12:18:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f069affd700 1 -- 172.21.15.44:0/1190438666 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c13cb70 con 0x7f069c061340 2020-11-20T12:18:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f0693fff700 1 --2- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c13c200 0x7f069c13c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f069affd700 1 --2- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c061340 0x7f069c0a6e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f068c00b9a0 tx=0x7f068c006df0).ready entity=mon.1 client_cookie=619353c818773c25 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f069a7fc700 1 -- 172.21.15.44:0/1190438666 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c009840 con 0x7f069c061340 2020-11-20T12:18:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f069a7fc700 1 -- 172.21.15.44:0/1190438666 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c0099a0 con 0x7f069c061340 2020-11-20T12:18:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f069a7fc700 1 -- 172.21.15.44:0/1190438666 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c009d20 con 0x7f069c061340 2020-11-20T12:18:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f06a1856700 1 -- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c061340 msgr2=0x7f069c0a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f06a1856700 1 --2- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c061340 0x7f069c0a6e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f068c00b9a0 tx=0x7f068c006df0).stop 2020-11-20T12:18:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.598+0000 7f06a1856700 1 -- 172.21.15.44:0/1190438666 shutdown_connections 2020-11-20T12:18:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.599+0000 7f06a1856700 1 --2- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c061340 0x7f069c0a6e10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.599+0000 7f06a1856700 1 --2- 172.21.15.44:0/1190438666 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c13c200 0x7f069c13c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.599+0000 7f06a1856700 1 -- 172.21.15.44:0/1190438666 >> 172.21.15.44:0/1190438666 conn(0x7f069c0ad6e0 msgr2=0x7f069c0adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:55.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.599+0000 7f06a1856700 1 -- 172.21.15.44:0/1190438666 shutdown_connections 2020-11-20T12:18:55.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.599+0000 7f06a1856700 1 -- 172.21.15.44:0/1190438666 wait complete. 2020-11-20T12:18:55.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 Processor -- start 2020-11-20T12:18:55.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 -- start start 2020-11-20T12:18:55.610 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c061340 0x7f069c149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069c0a80e0 0x7f069c0a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c13c200 0x7f069c0a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0a6180 con 0x7f069c0a80e0 2020-11-20T12:18:55.611 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0a69f0 con 0x7f069c13c200 2020-11-20T12:18:55.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.600+0000 7f06a1856700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0a57f0 con 0x7f069c061340 2020-11-20T12:18:55.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069c0a80e0 0x7f069c0a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069c0a80e0 0x7f069c0a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42492/0 (socket says 172.21.15.44:42492) 2020-11-20T12:18:55.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 -- 172.21.15.44:0/4195080020 learned_addr learned my addr 172.21.15.44:0/4195080020 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:55.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f069affd700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c061340 0x7f069c149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f069b7fe700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c13c200 0x7f069c0a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 -- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c061340 msgr2=0x7f069c149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c061340 0x7f069c149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 -- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c13c200 msgr2=0x7f069c0a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c13c200 0x7f069c0a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 -- 172.21.15.44:0/4195080020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c1366a0 con 0x7f069c0a80e0 2020-11-20T12:18:55.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0693fff700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069c0a80e0 0x7f069c0a63c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f068400ed20 tx=0x7f0684006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0692ffd700 1 -- 172.21.15.44:0/4195080020 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068400caa0 con 0x7f069c0a80e0 2020-11-20T12:18:55.614 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c139390 con 0x7f069c0a80e0 2020-11-20T12:18:55.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c0a2e40 con 0x7f069c0a80e0 2020-11-20T12:18:55.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0692ffd700 1 -- 172.21.15.44:0/4195080020 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068400cc00 con 0x7f069c0a80e0 2020-11-20T12:18:55.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.601+0000 7f0692ffd700 1 -- 172.21.15.44:0/4195080020 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06840123f0 con 0x7f069c0a80e0 2020-11-20T12:18:55.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.602+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c0a6bb0 con 0x7f069c0a80e0 2020-11-20T12:18:55.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.603+0000 7f0692ffd700 1 -- 172.21.15.44:0/4195080020 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f06840127a0 con 0x7f069c0a80e0 2020-11-20T12:18:55.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.603+0000 7f0692ffd700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0678056d20 0x7f06780591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.603+0000 7f0692ffd700 1 -- 172.21.15.44:0/4195080020 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0684070660 con 0x7f069c0a80e0 2020-11-20T12:18:55.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.603+0000 7f069affd700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0678056d20 0x7f06780591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.604+0000 7f069affd700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0678056d20 0x7f06780591e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f068c006e30 tx=0x7f068c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.608+0000 7f0692ffd700 1 -- 172.21.15.44:0/4195080020 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f06840459b0 con 0x7f069c0a80e0 2020-11-20T12:18:55.962 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.959+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f069c067f50 con 0x7f069c0a80e0 2020-11-20T12:18:55.963 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.959+0000 7f0692ffd700 1 -- 172.21.15.44:0/4195080020 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f0684048fd0 con 0x7f069c0a80e0 2020-11-20T12:18:55.963 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:55.964 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0678056d20 msgr2=0x7f06780591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0678056d20 0x7f06780591e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f068c006e30 tx=0x7f068c015000).stop 2020-11-20T12:18:55.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069c0a80e0 msgr2=0x7f069c0a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069c0a80e0 0x7f069c0a63c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f068400ed20 tx=0x7f0684006df0).stop 2020-11-20T12:18:55.965 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 shutdown_connections 2020-11-20T12:18:55.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0678056d20 0x7f06780591e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f069c13c200 0x7f069c0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f069c061340 0x7f069c149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 --2- 172.21.15.44:0/4195080020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f069c0a80e0 0x7f069c0a63c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.966 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 >> 172.21.15.44:0/4195080020 conn(0x7f069c0ad6e0 msgr2=0x7f069c0a96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:55.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 shutdown_connections 2020-11-20T12:18:55.967 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:55.961+0000 7f06a1856700 1 -- 172.21.15.44:0/4195080020 wait complete. 2020-11-20T12:18:55.977 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:18:55.977 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:18:55.978 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:18:56.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.123+0000 7f8cb10a6700 1 Processor -- start 2020-11-20T12:18:56.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.123+0000 7f8cb10a6700 1 -- start start 2020-11-20T12:18:56.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.124+0000 7f8cb10a6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8cac062cc0 0x7f8cac063150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.124+0000 7f8cb10a6700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac13dfe0 con 0x7f8cac0ae8c0 2020-11-20T12:18:56.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.124+0000 7f8cb10a6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac062860 con 0x7f8cac0ae4e0 2020-11-20T12:18:56.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.124+0000 7f8cb10a6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0a91b0 con 0x7f8cac062cc0 2020-11-20T12:18:56.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.124+0000 7f8cab7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8cac0ae8c0 0x7f8cac137c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46248/0 (socket says 172.21.15.44:46248) 2020-11-20T12:18:56.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.124+0000 7f8cab7fe700 1 -- 172.21.15.44:0/3435895436 learned_addr learned my addr 172.21.15.44:0/3435895436 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:56.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.124+0000 7f8cb08a5700 1 --2- 172.21.15.44:0/3435895436 >> v2:172.21.15.79:3300/0 conn(0x7f8cac062cc0 0x7f8cac063150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8caaffd700 1 -- 172.21.15.44:0/3435895436 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039531135 0 0) 0x7f8cac13dfe0 con 0x7f8cac0ae8c0 2020-11-20T12:18:56.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8caaffd700 1 -- 172.21.15.44:0/3435895436 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c98003240 con 0x7f8cac0ae8c0 2020-11-20T12:18:56.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8cb08a5700 1 -- 172.21.15.44:0/3435895436 >> v1:172.21.15.79:6789/0 conn(0x7f8cac0ae4e0 legacy=0x7f8cac0a8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8caaffd700 1 -- 172.21.15.44:0/3435895436 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352665887 0 0) 0x7f8cac062860 con 0x7f8cac0ae4e0 2020-11-20T12:18:56.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8cb08a5700 1 -- 172.21.15.44:0/3435895436 >> v1:172.21.15.44:6789/0 conn(0x7f8cac0ae8c0 legacy=0x7f8cac137c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8cb08a5700 1 -- 172.21.15.44:0/3435895436 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac138350 con 0x7f8cac062cc0 2020-11-20T12:18:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8cb08a5700 1 --2- 172.21.15.44:0/3435895436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac062cc0 0x7f8cac063150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8cac0629f0 tx=0x7f8ca40091d0).ready entity=mon.1 client_cookie=7bcec8339a3a4246 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8caaffd700 1 -- 172.21.15.44:0/3435895436 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400bad0 con 0x7f8cac062cc0 2020-11-20T12:18:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8caaffd700 1 -- 172.21.15.44:0/3435895436 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca400bc30 con 0x7f8cac062cc0 2020-11-20T12:18:56.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.125+0000 7f8caaffd700 1 -- 172.21.15.44:0/3435895436 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4011400 con 0x7f8cac062cc0 2020-11-20T12:18:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.126+0000 7f8cb10a6700 1 -- 172.21.15.44:0/3435895436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac062cc0 msgr2=0x7f8cac063150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.126+0000 7f8cb10a6700 1 --2- 172.21.15.44:0/3435895436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac062cc0 0x7f8cac063150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8cac0629f0 tx=0x7f8ca40091d0).stop 2020-11-20T12:18:56.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.126+0000 7f8cb10a6700 1 -- 172.21.15.44:0/3435895436 shutdown_connections 2020-11-20T12:18:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.126+0000 7f8cb10a6700 1 --2- 172.21.15.44:0/3435895436 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac062cc0 0x7f8cac063150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.126+0000 7f8cb10a6700 1 -- 172.21.15.44:0/3435895436 >> 172.21.15.44:0/3435895436 conn(0x7f8cac0654e0 msgr2=0x7f8cac0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.126+0000 7f8cb10a6700 1 -- 172.21.15.44:0/3435895436 shutdown_connections 2020-11-20T12:18:56.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.126+0000 7f8cb10a6700 1 -- 172.21.15.44:0/3435895436 wait complete. 2020-11-20T12:18:56.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.127+0000 7f8cb10a6700 1 Processor -- start 2020-11-20T12:18:56.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.127+0000 7f8cb10a6700 1 -- start start 2020-11-20T12:18:56.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cb10a6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac062cc0 0x7f8cac14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cb10a6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cac0ae4e0 0x7f8cac141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cb10a6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac0ae8c0 0x7f8cac13af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cb10a6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac13a040 con 0x7f8cac062cc0 2020-11-20T12:18:56.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cb10a6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac141430 con 0x7f8cac0ae8c0 2020-11-20T12:18:56.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cb10a6700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac138680 con 0x7f8cac0ae4e0 2020-11-20T12:18:56.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cabfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac062cc0 0x7f8cac14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cabfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac062cc0 0x7f8cac14cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42504/0 (socket says 172.21.15.44:42504) 2020-11-20T12:18:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cabfff700 1 -- 172.21.15.44:0/2604173488 learned_addr learned my addr 172.21.15.44:0/2604173488 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cab7fe700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cac0ae4e0 0x7f8cac141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.128+0000 7f8cb08a5700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac0ae8c0 0x7f8cac13af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cabfff700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cac0ae4e0 msgr2=0x7f8cac141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cabfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cac0ae4e0 0x7f8cac141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cabfff700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac0ae8c0 msgr2=0x7f8cac13af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cabfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cac0ae8c0 0x7f8cac13af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cabfff700 1 -- 172.21.15.44:0/2604173488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac138c70 con 0x7f8cac062cc0 2020-11-20T12:18:56.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cabfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac062cc0 0x7f8cac14cf10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8c9c007fd0 tx=0x7f8c9c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00e560 con 0x7f8cac062cc0 2020-11-20T12:18:56.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c00e6e0 con 0x7f8cac062cc0 2020-11-20T12:18:56.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac141660 con 0x7f8cac062cc0 2020-11-20T12:18:56.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00eaa0 con 0x7f8cac062cc0 2020-11-20T12:18:56.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.129+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac13b440 con 0x7f8cac062cc0 2020-11-20T12:18:56.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.130+0000 7f8cb10a6700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1403b0 0x7f8cac140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.130+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac061280 con 0x7f8cac1403b0 2020-11-20T12:18:56.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.130+0000 7f8cab7fe700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1403b0 0x7f8cac140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.130+0000 7f8cab7fe700 1 -- 172.21.15.44:0/2604173488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cac138e00 con 0x7f8cac1403b0 2020-11-20T12:18:56.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.130+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8c9c010070 con 0x7f8cac062cc0 2020-11-20T12:18:56.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.130+0000 7f8cab7fe700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1403b0 0x7f8cac140790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8cac13a580 tx=0x7f8ca0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8c84056d20 0x7f8c840591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8c9c06f9d0 con 0x7f8cac062cc0 2020-11-20T12:18:56.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8cb08a5700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8c84056d20 0x7f8c840591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00070d0 con 0x7f8cac1403b0 2020-11-20T12:18:56.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f8ca0007230 con 0x7f8cac1403b0 2020-11-20T12:18:56.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1403b0 msgr2=0x7f8cac140790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1403b0 0x7f8cac140790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8cac13a580 tx=0x7f8ca0009040).stop 2020-11-20T12:18:56.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1403b0 msgr2=0x7f8cac140790 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:56.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.131+0000 7f8c8bfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1403b0 0x7f8cac140790 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0x7f8cac13a580 tx=0x7f8ca0009040).stop 2020-11-20T12:18:56.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.132+0000 7f8cb08a5700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8c84056d20 0x7f8c840591e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f8cac14d490 tx=0x7f8ca4016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.140+0000 7f8cb10a6700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1499a0 0x7f8cac149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.140+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac057d80 con 0x7f8cac1499a0 2020-11-20T12:18:56.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.140+0000 7f8cab7fe700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1499a0 0x7f8cac149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.140+0000 7f8cab7fe700 1 -- 172.21.15.44:0/2604173488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8cac0a9450 con 0x7f8cac1499a0 2020-11-20T12:18:56.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8cab7fe700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1499a0 0x7f8cac149db0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8ca000efd0 tx=0x7f8ca0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca001d040 con 0x7f8cac1499a0 2020-11-20T12:18:56.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4269 (secure 0 0 0) 0x7f8ca00055d0 con 0x7f8cac1499a0 2020-11-20T12:18:56.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1499a0 msgr2=0x7f8cac149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8c8bfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1499a0 0x7f8cac149db0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8ca000efd0 tx=0x7f8ca0017000).stop 2020-11-20T12:18:56.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8c8bfff700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1499a0 msgr2=0x7f8cac149db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:56.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8c8bfff700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac1499a0 0x7f8cac149db0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8c84056d20 msgr2=0x7f8c840591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8cb10a6700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8c84056d20 0x7f8c840591e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f8cac14d490 tx=0x7f8ca4016000).stop 2020-11-20T12:18:56.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac062cc0 msgr2=0x7f8cac14cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8cb10a6700 1 --2- 172.21.15.44:0/2604173488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cac062cc0 0x7f8cac14cf10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8c9c007fd0 tx=0x7f8c9c007ac0).stop 2020-11-20T12:18:56.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.141+0000 7f8cb08a5700 1 -- 172.21.15.44:0/2604173488 reap_dead start 2020-11-20T12:18:56.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.142+0000 7f8cb08a5700 1 -- 172.21.15.44:0/2604173488 reap_dead start 2020-11-20T12:18:56.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.142+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 shutdown_connections 2020-11-20T12:18:56.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.142+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 >> 172.21.15.44:0/2604173488 conn(0x7f8cac0654e0 msgr2=0x7f8cac13c130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.142+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 shutdown_connections 2020-11-20T12:18:56.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.142+0000 7f8cb10a6700 1 -- 172.21.15.44:0/2604173488 wait complete. 2020-11-20T12:18:56.153 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:56.154 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:56.154 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:56.154 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:18:56.154 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 152, 2020-11-20T12:18:56.155 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:56.155 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:18:56.155 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:56.155 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:56.155 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.156 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:18:56.156 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:56.156 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:56.156 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:56.156 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.157 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.158 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.158 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.158 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.158 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.158 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:56.159 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:56.159 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.159 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.159 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.159 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.160 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.160 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.160 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.160 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.160 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.161 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:56.161 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:56.161 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:56.161 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:56.162 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:56.162 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:56.162 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:56.162 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:56.162 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:56.163 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:56.163 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:56.163 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:56.163 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:56.163 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:56.164 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:56.164 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.164 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.164 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.165 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.165 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.165 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.165 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.166 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:56.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:56.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.166 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:56.167 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:56.167 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:56.168 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:56.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:56.170 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:56.170 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.170 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.170 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.171 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:56.171 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:56.171 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:56.172 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:56.173 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.174 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:56.174 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:56.174 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.175 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.175 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.175 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:56.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:56.176 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.176 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:56.177 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:56.178 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.178 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:56.179 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:56.179 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.179 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.180 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.180 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:56.181 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:56.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.181 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.181 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.181 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:56.182 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.182 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:56.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.183 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.183 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.183 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:56.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.184 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:56.184 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:56.185 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:18:56.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.324+0000 7f27cf8c8700 1 Processor -- start 2020-11-20T12:18:56.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.325+0000 7f27cf8c8700 1 -- start start 2020-11-20T12:18:56.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.325+0000 7f27cf8c8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f27c80a86d0 0x7f27c80a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.325+0000 7f27cf8c8700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c81385f0 con 0x7f27c80aad00 2020-11-20T12:18:56.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.325+0000 7f27cf8c8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c80aca50 con 0x7f27c80ab0e0 2020-11-20T12:18:56.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.325+0000 7f27cf8c8700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c813e250 con 0x7f27c80a86d0 2020-11-20T12:18:56.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27cde65700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f27c80a86d0 0x7f27c80a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27cd664700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f27c80aad00 0x7f27c80abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46264/0 (socket says 172.21.15.44:46264) 2020-11-20T12:18:56.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27cd664700 1 -- 172.21.15.44:0/4072564957 learned_addr learned my addr 172.21.15.44:0/4072564957 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:56.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27b7fff700 1 -- 172.21.15.44:0/4072564957 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000543429 0 0) 0x7f27c81385f0 con 0x7f27c80aad00 2020-11-20T12:18:56.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27b7fff700 1 -- 172.21.15.44:0/4072564957 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b0003240 con 0x7f27c80aad00 2020-11-20T12:18:56.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27cde65700 1 -- 172.21.15.44:0/4072564957 >> v1:172.21.15.79:6789/0 conn(0x7f27c80ab0e0 legacy=0x7f27c80ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27cde65700 1 -- 172.21.15.44:0/4072564957 >> v1:172.21.15.44:6789/0 conn(0x7f27c80aad00 legacy=0x7f27c80abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27cde65700 1 -- 172.21.15.44:0/4072564957 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c813e4f0 con 0x7f27c80a86d0 2020-11-20T12:18:56.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27cde65700 1 --2- 172.21.15.44:0/4072564957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a94d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f27c0009c90 tx=0x7f27c00091d0).ready entity=mon.0 client_cookie=67a69afa37786cd4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27b7fff700 1 -- 172.21.15.44:0/4072564957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000ba70 con 0x7f27c80a86d0 2020-11-20T12:18:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27b7fff700 1 -- 172.21.15.44:0/4072564957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c000bbd0 con 0x7f27c80a86d0 2020-11-20T12:18:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.326+0000 7f27b7fff700 1 -- 172.21.15.44:0/4072564957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0011400 con 0x7f27c80a86d0 2020-11-20T12:18:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.327+0000 7f27cf8c8700 1 -- 172.21.15.44:0/4072564957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 msgr2=0x7f27c80a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.327+0000 7f27cf8c8700 1 --2- 172.21.15.44:0/4072564957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a94d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f27c0009c90 tx=0x7f27c00091d0).stop 2020-11-20T12:18:56.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.327+0000 7f27cf8c8700 1 -- 172.21.15.44:0/4072564957 shutdown_connections 2020-11-20T12:18:56.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.327+0000 7f27cf8c8700 1 --2- 172.21.15.44:0/4072564957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a94d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.327+0000 7f27cf8c8700 1 -- 172.21.15.44:0/4072564957 >> 172.21.15.44:0/4072564957 conn(0x7f27c8060170 msgr2=0x7f27c8060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.328+0000 7f27cf8c8700 1 -- 172.21.15.44:0/4072564957 shutdown_connections 2020-11-20T12:18:56.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.328+0000 7f27cf8c8700 1 -- 172.21.15.44:0/4072564957 wait complete. 2020-11-20T12:18:56.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.329+0000 7f27cf8c8700 1 Processor -- start 2020-11-20T12:18:56.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.329+0000 7f27cf8c8700 1 -- start start 2020-11-20T12:18:56.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cf8c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cf8c8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c80aad00 0x7f27c809de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cf8c8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f27c80ab0e0 0x7f27c814d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cf8c8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c814c9a0 con 0x7f27c80a86d0 2020-11-20T12:18:56.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cf8c8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c814cb80 con 0x7f27c80aad00 2020-11-20T12:18:56.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cf8c8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c814ce20 con 0x7f27c80ab0e0 2020-11-20T12:18:56.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cd664700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cd664700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42522/0 (socket says 172.21.15.44:42522) 2020-11-20T12:18:56.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.330+0000 7f27cd664700 1 -- 172.21.15.44:0/2539459756 learned_addr learned my addr 172.21.15.44:0/2539459756 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:56.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cde65700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f27c80ab0e0 0x7f27c814d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cce63700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c80aad00 0x7f27c809de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cd664700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f27c80ab0e0 msgr2=0x7f27c814d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cd664700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f27c80ab0e0 0x7f27c814d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cd664700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c80aad00 msgr2=0x7f27c809de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cd664700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c80aad00 0x7f27c809de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cd664700 1 -- 172.21.15.44:0/2539459756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c814d2c0 con 0x7f27c80a86d0 2020-11-20T12:18:56.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cd664700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a74c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f27c400c6f0 tx=0x7f27c4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40072e0 con 0x7f27c80a86d0 2020-11-20T12:18:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4007440 con 0x7f27c80a86d0 2020-11-20T12:18:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c809e360 con 0x7f27c80a86d0 2020-11-20T12:18:56.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4005800 con 0x7f27c80a86d0 2020-11-20T12:18:56.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.331+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c814dd00 con 0x7f27c80a86d0 2020-11-20T12:18:56.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.332+0000 7f27cf8c8700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c81350a0 0x7f27c8135480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.332+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c805c5f0 con 0x7f27c81350a0 2020-11-20T12:18:56.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.332+0000 7f27cce63700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c81350a0 0x7f27c8135480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.332+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f27c4010070 con 0x7f27c80a86d0 2020-11-20T12:18:56.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27cce63700 1 -- 172.21.15.44:0/2539459756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27c814cfe0 con 0x7f27c81350a0 2020-11-20T12:18:56.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27cce63700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c81350a0 0x7f27c8135480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f27c813b720 tx=0x7f27b8007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27a0056d20 0x7f27a00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f27c406f6d0 con 0x7f27c80a86d0 2020-11-20T12:18:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8011650 con 0x7f27c81350a0 2020-11-20T12:18:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27cde65700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27a0056d20 0x7f27a00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f27b80117b0 con 0x7f27c81350a0 2020-11-20T12:18:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c81350a0 msgr2=0x7f27c8135480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c81350a0 0x7f27c8135480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f27c813b720 tx=0x7f27b8007b90).stop 2020-11-20T12:18:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c81350a0 msgr2=0x7f27c8135480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:56.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.333+0000 7f27b57fa700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c81350a0 0x7f27c8135480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.334+0000 7f27cde65700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27a0056d20 0x7f27a00591e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f27c000b770 tx=0x7f27c0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.346+0000 7f27cf8c8700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c8138b50 0x7f27c8138f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.346+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c814dd00 con 0x7f27c8138b50 2020-11-20T12:18:56.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.346+0000 7f27cce63700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c8138b50 0x7f27c8138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.346+0000 7f27cce63700 1 -- 172.21.15.44:0/2539459756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f27c8069680 con 0x7f27c8138b50 2020-11-20T12:18:56.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.346+0000 7f27cce63700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c8138b50 0x7f27c8138f60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f27b8007570 tx=0x7f27b8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8014df0 con 0x7f27c8138b50 2020-11-20T12:18:56.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4437 (secure 0 0 0) 0x7f27b8011440 con 0x7f27c8138b50 2020-11-20T12:18:56.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c8138b50 msgr2=0x7f27c8138f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27b57fa700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c8138b50 0x7f27c8138f60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f27b8007570 tx=0x7f27b8016000).stop 2020-11-20T12:18:56.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27b57fa700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c8138b50 msgr2=0x7f27c8138f60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:56.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27b57fa700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27c8138b50 0x7f27c8138f60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.353 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27a0056d20 msgr2=0x7f27a00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f27a0056d20 0x7f27a00591e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f27c000b770 tx=0x7f27c0016000).stop 2020-11-20T12:18:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 msgr2=0x7f27c80a74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 --2- 172.21.15.44:0/2539459756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27c80a86d0 0x7f27c80a74c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f27c400c6f0 tx=0x7f27c4007ad0).stop 2020-11-20T12:18:56.354 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cde65700 1 -- 172.21.15.44:0/2539459756 reap_dead start 2020-11-20T12:18:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cde65700 1 -- 172.21.15.44:0/2539459756 reap_dead start 2020-11-20T12:18:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 shutdown_connections 2020-11-20T12:18:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 >> 172.21.15.44:0/2539459756 conn(0x7f27c8060170 msgr2=0x7f27c80ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 shutdown_connections 2020-11-20T12:18:56.355 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.347+0000 7f27cf8c8700 1 -- 172.21.15.44:0/2539459756 wait complete. 2020-11-20T12:18:56.356 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:56.356 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:56.356 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:56.356 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:18:56.357 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 152, 2020-11-20T12:18:56.357 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:56.357 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:18:56.357 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:56.357 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:56.358 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.358 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:18:56.358 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:56.358 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:56.359 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:56.359 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.359 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.359 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.359 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.360 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.360 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.360 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.360 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.361 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.361 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:56.361 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:56.362 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.362 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.362 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.362 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.363 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.363 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.363 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.363 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.363 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.364 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:56.364 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:56.364 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:56.364 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:56.365 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:56.365 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:56.365 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:56.365 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:56.366 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:56.366 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:56.366 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:56.366 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:56.366 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:18:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:56.390 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:56.391 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:18:56.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.538+0000 7fec7b22e700 1 Processor -- start 2020-11-20T12:18:56.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.538+0000 7fec7b22e700 1 -- start start 2020-11-20T12:18:56.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.538+0000 7fec7b22e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fec74066de0 0x7fec740aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.538+0000 7fec7b22e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec74137bb0 0x7fec74138030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.538+0000 7fec7b22e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec74067470 con 0x7fec740a1a40 2020-11-20T12:18:56.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.538+0000 7fec7b22e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec740a9b30 con 0x7fec74066de0 2020-11-20T12:18:56.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.538+0000 7fec7b22e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec740ab240 con 0x7fec74137bb0 2020-11-20T12:18:56.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec73fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fec74066de0 0x7fec740aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec78fca700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fec740a1a40 0x7fec740666d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46282/0 (socket says 172.21.15.44:46282) 2020-11-20T12:18:56.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec78fca700 1 -- 172.21.15.44:0/546618911 learned_addr learned my addr 172.21.15.44:0/546618911 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec797cb700 1 --2- 172.21.15.44:0/546618911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec74137bb0 0x7fec74138030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec73fff700 1 -- 172.21.15.44:0/546618911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec74137bb0 msgr2=0x7fec74138030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec73fff700 1 --2- 172.21.15.44:0/546618911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec74137bb0 0x7fec74138030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec73fff700 1 -- 172.21.15.44:0/546618911 >> v1:172.21.15.44:6789/0 conn(0x7fec740a1a40 legacy=0x7fec740666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec73fff700 1 -- 172.21.15.44:0/546618911 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74067710 con 0x7fec74066de0 2020-11-20T12:18:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec73fff700 1 --2- 172.21.15.44:0/546618911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec740aad50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fec6000b9d0 tx=0x7fec60006df0).ready entity=mon.0 client_cookie=34a18314dcc45c5b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec737fe700 1 -- 172.21.15.44:0/546618911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600096b0 con 0x7fec74066de0 2020-11-20T12:18:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec737fe700 1 -- 172.21.15.44:0/546618911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec60009810 con 0x7fec74066de0 2020-11-20T12:18:56.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.539+0000 7fec737fe700 1 -- 172.21.15.44:0/546618911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60009b90 con 0x7fec74066de0 2020-11-20T12:18:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 -- 172.21.15.44:0/546618911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 msgr2=0x7fec740aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 --2- 172.21.15.44:0/546618911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec740aad50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fec6000b9d0 tx=0x7fec60006df0).stop 2020-11-20T12:18:56.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 -- 172.21.15.44:0/546618911 shutdown_connections 2020-11-20T12:18:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 --2- 172.21.15.44:0/546618911 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec74137bb0 0x7fec74138030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 --2- 172.21.15.44:0/546618911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec740aad50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 -- 172.21.15.44:0/546618911 >> 172.21.15.44:0/546618911 conn(0x7fec740af030 msgr2=0x7fec740af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 -- 172.21.15.44:0/546618911 shutdown_connections 2020-11-20T12:18:56.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.540+0000 7fec7b22e700 1 -- 172.21.15.44:0/546618911 wait complete. 2020-11-20T12:18:56.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.541+0000 7fec7b22e700 1 Processor -- start 2020-11-20T12:18:56.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.541+0000 7fec7b22e700 1 -- start start 2020-11-20T12:18:56.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec7b22e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec74061b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec7b22e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a1a40 0x7fec741361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec7b22e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec74137bb0 0x7fec7413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec7b22e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74063370 con 0x7fec74066de0 2020-11-20T12:18:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec7b22e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74135730 con 0x7fec74137bb0 2020-11-20T12:18:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec7b22e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec74063560 con 0x7fec740a1a40 2020-11-20T12:18:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec78fca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec74061b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec78fca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec74061b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42540/0 (socket says 172.21.15.44:42540) 2020-11-20T12:18:56.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec78fca700 1 -- 172.21.15.44:0/3539809549 learned_addr learned my addr 172.21.15.44:0/3539809549 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:56.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec73fff700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a1a40 0x7fec741361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.542+0000 7fec797cb700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec74137bb0 0x7fec7413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec78fca700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a1a40 msgr2=0x7fec741361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec78fca700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a1a40 0x7fec741361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec78fca700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec74137bb0 msgr2=0x7fec7413e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec78fca700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec74137bb0 0x7fec7413e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec78fca700 1 -- 172.21.15.44:0/3539809549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74063800 con 0x7fec74066de0 2020-11-20T12:18:56.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec73fff700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a1a40 0x7fec741361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:56.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec78fca700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec74061b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fec6800c8b0 tx=0x7fec68007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68007320 con 0x7fec74066de0 2020-11-20T12:18:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68007480 con 0x7fec74066de0 2020-11-20T12:18:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec74136690 con 0x7fec74066de0 2020-11-20T12:18:56.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68005800 con 0x7fec74066de0 2020-11-20T12:18:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.543+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7413f070 con 0x7fec74066de0 2020-11-20T12:18:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.544+0000 7fec7b22e700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a8970 0x7fec740a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.544+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7413f320 con 0x7fec740a8970 2020-11-20T12:18:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.544+0000 7fec73fff700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a8970 0x7fec740a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.544+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fec68010030 con 0x7fec74066de0 2020-11-20T12:18:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.544+0000 7fec73fff700 1 -- 172.21.15.44:0/3539809549 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec7413e6a0 con 0x7fec740a8970 2020-11-20T12:18:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec73fff700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a8970 0x7fec740a8d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fec6000b9d0 tx=0x7fec60015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fec4c056d20 0x7fec4c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fec6806fed0 con 0x7fec74066de0 2020-11-20T12:18:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec797cb700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fec4c056d20 0x7fec4c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6001f650 con 0x7fec740a8970 2020-11-20T12:18:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fec6001e420 con 0x7fec740a8970 2020-11-20T12:18:56.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a8970 msgr2=0x7fec740a8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a8970 0x7fec740a8d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fec6000b9d0 tx=0x7fec60015000).stop 2020-11-20T12:18:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a8970 msgr2=0x7fec740a8d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec70ff9700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec740a8970 0x7fec740a8d50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.545+0000 7fec797cb700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fec4c056d20 0x7fec4c0591e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fec6c009d60 tx=0x7fec6c0091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.554+0000 7fec7b22e700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec7414d9e0 0x7fec7414ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.554+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec74063fd0 con 0x7fec7414d9e0 2020-11-20T12:18:56.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.554+0000 7fec73fff700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec7414d9e0 0x7fec7414ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec73fff700 1 -- 172.21.15.44:0/3539809549 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fec740a2a70 con 0x7fec7414d9e0 2020-11-20T12:18:56.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec73fff700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec7414d9e0 0x7fec7414ddf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fec74136870 tx=0x7fec60021070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60029440 con 0x7fec7414d9e0 2020-11-20T12:18:56.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7fec6001f440 con 0x7fec7414d9e0 2020-11-20T12:18:56.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec7414d9e0 msgr2=0x7fec7414ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec70ff9700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec7414d9e0 0x7fec7414ddf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fec74136870 tx=0x7fec60021070).stop 2020-11-20T12:18:56.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec70ff9700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec7414d9e0 msgr2=0x7fec7414ddf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:18:56.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec70ff9700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fec7414d9e0 0x7fec7414ddf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fec4c056d20 msgr2=0x7fec4c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec7b22e700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fec4c056d20 0x7fec4c0591e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7fec6c009d60 tx=0x7fec6c0091d0).stop 2020-11-20T12:18:56.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 msgr2=0x7fec74061b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec7b22e700 1 --2- 172.21.15.44:0/3539809549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec74066de0 0x7fec74061b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fec6800c8b0 tx=0x7fec68007c00).stop 2020-11-20T12:18:56.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.555+0000 7fec797cb700 1 -- 172.21.15.44:0/3539809549 reap_dead start 2020-11-20T12:18:56.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.556+0000 7fec797cb700 1 -- 172.21.15.44:0/3539809549 reap_dead start 2020-11-20T12:18:56.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.556+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 shutdown_connections 2020-11-20T12:18:56.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.556+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 >> 172.21.15.44:0/3539809549 conn(0x7fec740af030 msgr2=0x7fec740af410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.556+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 shutdown_connections 2020-11-20T12:18:56.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.556+0000 7fec7b22e700 1 -- 172.21.15.44:0/3539809549 wait complete. 2020-11-20T12:18:56.567 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:18:56.567 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:56.567 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:56.568 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:18:56.568 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 152, 2020-11-20T12:18:56.568 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:18:56.568 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:18:56.568 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:18:56.569 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:18:56.569 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.569 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:18:56.569 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:56.569 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:18:56.570 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:18:56.570 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.570 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.570 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.570 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.571 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.571 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.571 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.571 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.572 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.572 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:18:56.572 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:18:56.572 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.573 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.573 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.573 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.573 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.573 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.574 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.574 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.574 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.574 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:18:56.575 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:18:56.575 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:18:56.575 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:18:56.575 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:18:56.576 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:18:56.576 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:56.576 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:18:56.576 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:18:56.576 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:18:56.577 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:18:56.577 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:18:56.577 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:18:56.577 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:18:56.578 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:18:56.578 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:18:56.578 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:18:56.578 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:18:56.578 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:18:56.578 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:18:56.579 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:18:56.579 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:18:56.579 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:18:56.579 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.580 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:18:56.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.580 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:18:56.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.580 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:18:56.581 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:18:56.581 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.581 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.582 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:18:56.582 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.583 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:18:56.583 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.584 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:18:56.584 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:18:56.584 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.584 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.584 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.585 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:18:56.585 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:18:56.585 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.586 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.587 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:18:56.587 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.588 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:18:56.588 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.588 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.588 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.589 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:18:56.589 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:18:56.589 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.589 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.589 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.590 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:18:56.590 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:18:56.590 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:18:56.591 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:18:56.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:18:56.591 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:18:56.591 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:18:56.592 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:18:56.592 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:18:56.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.593 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.593 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:18:56.593 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:18:56.594 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:18:56.594 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:18:56.594 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:18:56.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.595 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.595 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:18:56.596 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:18:56.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.596 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.596 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.596 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:18:56.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.597 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.597 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:18:56.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.598 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.598 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.598 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:56.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.599 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.599 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:18:56.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.599 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.600 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.600 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:18:56.600 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.600 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:18:56.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.601 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.601 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.601 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:18:56.602 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.602 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:18:56.602 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:18:56.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:18:56.602 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:18:56.602 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:18:56.603 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:18:56.603 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:18:56.603 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:18:56.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:18:56.604 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:18:56.604 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:18:56.605 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2020-11-20T12:18:56.606 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:18:56.606 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2020-11-20T12:18:56.606 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2020-11-20T12:18:56.606 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2020-11-20T12:18:56.607 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:18:56.625 INFO:tasks.ceph.mon.c:Stopped 2020-11-20T12:18:56.626 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:18:56.626 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:18:56.768 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.765+0000 7f8e1e150700 1 Processor -- start 2020-11-20T12:18:56.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.765+0000 7f8e1e150700 1 -- start start 2020-11-20T12:18:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.765+0000 7f8e1e150700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8e180b17c0 0x7f8e1813abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e1e150700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e180a9000 con 0x7f8e180b1ba0 2020-11-20T12:18:56.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e1e150700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e180a92a0 con 0x7f8e180a9740 2020-11-20T12:18:56.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e1e150700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e18138600 con 0x7f8e180b17c0 2020-11-20T12:18:56.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e1c94d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8e180b1ba0 0x7f8e180aad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46338/0 (socket says 172.21.15.44:46338) 2020-11-20T12:18:56.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e1c94d700 1 -- 172.21.15.44:0/1505159140 learned_addr learned my addr 172.21.15.44:0/1505159140 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:18:56.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e1d14e700 1 --2- 172.21.15.44:0/1505159140 >> v2:172.21.15.79:3300/0 conn(0x7f8e180b17c0 0x7f8e1813abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633249062 0 0) 0x7f8e180a9000 con 0x7f8e180b1ba0 2020-11-20T12:18:56.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e00003240 con 0x7f8e180b1ba0 2020-11-20T12:18:56.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2391417150 0 0) 0x7f8e00003240 con 0x7f8e180b1ba0 2020-11-20T12:18:56.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 >> v2:172.21.15.79:3300/0 conn(0x7f8e180b17c0 msgr2=0x7f8e1813abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:56.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 --2- 172.21.15.44:0/1505159140 >> v2:172.21.15.79:3300/0 conn(0x7f8e180b17c0 0x7f8e1813abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 >> v1:172.21.15.79:6789/0 conn(0x7f8e180a9740 legacy=0x7f8e18137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e180a4c30 con 0x7f8e180b1ba0 2020-11-20T12:18:56.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f8e080014e0 con 0x7f8e180b1ba0 2020-11-20T12:18:56.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 >> v1:172.21.15.44:6789/0 conn(0x7f8e180b1ba0 legacy=0x7f8e180aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00003730 0x7f8e00003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e000044e0 0x7f8e00004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e00004e50 0x7f8e0000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18138600 con 0x7f8e00004e50 2020-11-20T12:18:56.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00004260 con 0x7f8e00003730 2020-11-20T12:18:56.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.766+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0000bc20 con 0x7f8e000044e0 2020-11-20T12:18:56.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1c94d700 1 -- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e000044e0 msgr2=0x7f8e00004960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:18:56.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1c94d700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e000044e0 0x7f8e00004960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:56.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e00004e50 0x7f8e0000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d14e700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00003730 0x7f8e00003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 -- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e000044e0 msgr2=0x7f8e00004960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:18:56.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e000044e0 0x7f8e00004960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:56.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 -- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00003730 msgr2=0x7f8e00003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:56.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00003730 0x7f8e00003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 -- 172.21.15.44:0/1505159140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00003240 con 0x7f8e00004e50 2020-11-20T12:18:56.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d14e700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00003730 0x7f8e00003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:56.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 -- 172.21.15.44:0/1505159140 reap_dead start 2020-11-20T12:18:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e1d94f700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e00004e50 0x7f8e0000b730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e10001e70 tx=0x7f8e10007f10).ready entity=mon.0 client_cookie=893f5a2c97772eb0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10001540 con 0x7f8e00004e50 2020-11-20T12:18:56.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10003030 con 0x7f8e00004e50 2020-11-20T12:18:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.767+0000 7f8e07fff700 1 -- 172.21.15.44:0/1505159140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1000cdf0 con 0x7f8e00004e50 2020-11-20T12:18:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.768+0000 7f8e1e150700 1 -- 172.21.15.44:0/1505159140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e00004e50 msgr2=0x7f8e0000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.768+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e00004e50 0x7f8e0000b730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e10001e70 tx=0x7f8e10007f10).stop 2020-11-20T12:18:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.768+0000 7f8e1e150700 1 -- 172.21.15.44:0/1505159140 shutdown_connections 2020-11-20T12:18:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.768+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1505159140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e00004e50 0x7f8e0000b730 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.768+0000 7f8e1e150700 1 -- 172.21.15.44:0/1505159140 >> 172.21.15.44:0/1505159140 conn(0x7f8e18060c10 msgr2=0x7f8e180b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.768+0000 7f8e1e150700 1 -- 172.21.15.44:0/1505159140 shutdown_connections 2020-11-20T12:18:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.768+0000 7f8e1e150700 1 -- 172.21.15.44:0/1505159140 wait complete. 2020-11-20T12:18:56.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.769+0000 7f8e1e150700 1 Processor -- start 2020-11-20T12:18:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.769+0000 7f8e1e150700 1 -- start start 2020-11-20T12:18:56.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1e150700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00004e50 0x7f8e1814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1e150700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e18137240 0x7f8e1813dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1e150700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e180a7ae0 0x7f8e180a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1e150700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1809db60 con 0x7f8e18137240 2020-11-20T12:18:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1e150700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1809ee40 con 0x7f8e00004e50 2020-11-20T12:18:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1e150700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1809ebe0 con 0x7f8e180a7ae0 2020-11-20T12:18:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e18137240 0x7f8e1813dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1d94f700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e180a7ae0 msgr2=0x7f8e180a4fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:18:56.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e18137240 0x7f8e1813dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42602/0 (socket says 172.21.15.44:42602) 2020-11-20T12:18:56.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1d94f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e180a7ae0 0x7f8e180a4fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:18:56.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 -- 172.21.15.44:0/1337539974 learned_addr learned my addr 172.21.15.44:0/1337539974 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:18:56.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1d14e700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00004e50 0x7f8e1814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 -- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e180a7ae0 msgr2=0x7f8e180a4fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:18:56.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e180a7ae0 0x7f8e180a4fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 -- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00004e50 msgr2=0x7f8e1814d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00004e50 0x7f8e1814d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 -- 172.21.15.44:0/1337539974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e180a7640 con 0x7f8e18137240 2020-11-20T12:18:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.770+0000 7f8e1c94d700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e18137240 0x7f8e1813dc50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8e0800cb10 tx=0x7f8e08001710).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.771+0000 7f8e057fa700 1 -- 172.21.15.44:0/1337539974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08007170 con 0x7f8e18137240 2020-11-20T12:18:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.771+0000 7f8e057fa700 1 -- 172.21.15.44:0/1337539974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e080072d0 con 0x7f8e18137240 2020-11-20T12:18:56.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.771+0000 7f8e057fa700 1 -- 172.21.15.44:0/1337539974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08004920 con 0x7f8e18137240 2020-11-20T12:18:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.771+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e180a7f60 con 0x7f8e18137240 2020-11-20T12:18:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.771+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1813e770 con 0x7f8e18137240 2020-11-20T12:18:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.772+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e180a77d0 con 0x7f8e18137240 2020-11-20T12:18:56.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.772+0000 7f8e057fa700 1 -- 172.21.15.44:0/1337539974 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8e08010070 con 0x7f8e18137240 2020-11-20T12:18:56.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.773+0000 7f8e057fa700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8df0056cd0 0x7f8df0059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.773+0000 7f8e057fa700 1 -- 172.21.15.44:0/1337539974 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8e0806fa50 con 0x7f8e18137240 2020-11-20T12:18:56.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.773+0000 7f8e1d14e700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8df0056cd0 0x7f8df0059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.773+0000 7f8e1d14e700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8df0056cd0 0x7f8df0059190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f8e14006dc0 tx=0x7f8e14008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:56.777+0000 7f8e057fa700 1 -- 172.21.15.44:0/1337539974 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8e08044d20 con 0x7f8e18137240 2020-11-20T12:18:57.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.086+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8e18057d80 con 0x7f8e18137240 2020-11-20T12:18:57.090 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.087+0000 7f8e057fa700 1 -- 172.21.15.44:0/1337539974 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f8e08048340 con 0x7f8e18137240 2020-11-20T12:18:57.091 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:18:57.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8df0056cd0 msgr2=0x7f8df0059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8df0056cd0 0x7f8df0059190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f8e14006dc0 tx=0x7f8e14008040).stop 2020-11-20T12:18:57.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e18137240 msgr2=0x7f8e1813dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e18137240 0x7f8e1813dc50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f8e0800cb10 tx=0x7f8e08001710).stop 2020-11-20T12:18:57.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 shutdown_connections 2020-11-20T12:18:57.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8df0056cd0 0x7f8df0059190 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8e180a7ae0 0x7f8e180a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:57.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e00004e50 0x7f8e1814d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 --2- 172.21.15.44:0/1337539974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e18137240 0x7f8e1813dc50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.095 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 >> 172.21.15.44:0/1337539974 conn(0x7f8e18060c10 msgr2=0x7f8e180ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 shutdown_connections 2020-11-20T12:18:57.096 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:18:57.089+0000 7f8e1e150700 1 -- 172.21.15.44:0/1337539974 wait complete. 2020-11-20T12:19:00.105 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.248+0000 7faff27c3700 1 Processor -- start 2020-11-20T12:19:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.248+0000 7faff27c3700 1 -- start start 2020-11-20T12:19:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7faff27c3700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fafec0abc70 0x7fafec0671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7faff27c3700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec13c650 con 0x7fafec0a72d0 2020-11-20T12:19:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7faff27c3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec13c8f0 con 0x7fafec067dc0 2020-11-20T12:19:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7faff27c3700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafec0ac550 con 0x7fafec0abc70 2020-11-20T12:19:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7fafebfff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fafec0abc70 0x7fafec0671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7fafeb7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fafec0a72d0 0x7fafec0676b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46366/0 (socket says 172.21.15.44:46366) 2020-11-20T12:19:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7fafeb7fe700 1 -- 172.21.15.44:0/2073185168 learned_addr learned my addr 172.21.15.44:0/2073185168 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7fafebfff700 1 -- 172.21.15.44:0/2073185168 >> v1:172.21.15.79:6789/0 conn(0x7fafec067dc0 legacy=0x7fafec143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7fafebfff700 1 -- 172.21.15.44:0/2073185168 >> v1:172.21.15.44:6789/0 conn(0x7fafec0a72d0 legacy=0x7fafec0676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7fafebfff700 1 -- 172.21.15.44:0/2073185168 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec13cb90 con 0x7fafec0abc70 2020-11-20T12:19:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.249+0000 7fafeaffd700 1 -- 172.21.15.44:0/2073185168 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695541656 0 0) 0x7fafec13c650 con 0x7fafec0a72d0 2020-11-20T12:19:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7fafebfff700 1 --2- 172.21.15.44:0/2073185168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0abc70 0x7fafec0671c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fafdc00b990 tx=0x7fafdc006d90).ready entity=mon.0 client_cookie=51086255b6bf86ee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7fafeaffd700 1 -- 172.21.15.44:0/2073185168 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc006030 con 0x7fafec0abc70 2020-11-20T12:19:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7fafeaffd700 1 -- 172.21.15.44:0/2073185168 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc006190 con 0x7fafec0abc70 2020-11-20T12:19:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7fafeaffd700 1 -- 172.21.15.44:0/2073185168 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc006510 con 0x7fafec0abc70 2020-11-20T12:19:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7faff27c3700 1 -- 172.21.15.44:0/2073185168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0abc70 msgr2=0x7fafec0671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7faff27c3700 1 --2- 172.21.15.44:0/2073185168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0abc70 0x7fafec0671c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fafdc00b990 tx=0x7fafdc006d90).stop 2020-11-20T12:19:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7faff27c3700 1 -- 172.21.15.44:0/2073185168 shutdown_connections 2020-11-20T12:19:00.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7faff27c3700 1 --2- 172.21.15.44:0/2073185168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0abc70 0x7fafec0671c0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0x7fafdc00b990 tx=0x7fafdc006d90).stop 2020-11-20T12:19:00.258 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.250+0000 7faff27c3700 1 -- 172.21.15.44:0/2073185168 >> 172.21.15.44:0/2073185168 conn(0x7fafec060170 msgr2=0x7fafec060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:00.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.252+0000 7faff27c3700 1 -- 172.21.15.44:0/2073185168 shutdown_connections 2020-11-20T12:19:00.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.252+0000 7faff27c3700 1 -- 172.21.15.44:0/2073185168 wait complete. 2020-11-20T12:19:00.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.253+0000 7faff27c3700 1 Processor -- start 2020-11-20T12:19:00.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.253+0000 7faff27c3700 1 -- start start 2020-11-20T12:19:00.261 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7faff27c3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fafec067dc0 0x7fafec0a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7faff27c3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0a72d0 0x7fafec13dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7faff27c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafec0619a0 0x7fafec13e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7faff27c3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec13f040 con 0x7fafec0a72d0 2020-11-20T12:19:00.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7faff27c3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec13e5d0 con 0x7fafec0619a0 2020-11-20T12:19:00.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7faff27c3700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec0633e0 con 0x7fafec067dc0 2020-11-20T12:19:00.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafebfff700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fafec067dc0 msgr2=0x7fafec0a1a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:00.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafebfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fafec067dc0 0x7fafec0a1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:00.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0a72d0 0x7fafec13dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0a72d0 0x7fafec13dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42626/0 (socket says 172.21.15.44:42626) 2020-11-20T12:19:00.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 -- 172.21.15.44:0/602478410 learned_addr learned my addr 172.21.15.44:0/602478410 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7faff0d60700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafec0619a0 0x7fafec13e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 -- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fafec067dc0 msgr2=0x7fafec0a1a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fafec067dc0 0x7fafec0a1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 -- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafec0619a0 msgr2=0x7fafec13e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafec0619a0 0x7fafec13e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.254+0000 7fafeb7fe700 1 -- 172.21.15.44:0/602478410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec063680 con 0x7fafec0a72d0 2020-11-20T12:19:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.255+0000 7fafeb7fe700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0a72d0 0x7fafec13dbf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fafe000c760 tx=0x7fafe0007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.255+0000 7fafcbfff700 1 -- 172.21.15.44:0/602478410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe00072e0 con 0x7fafec0a72d0 2020-11-20T12:19:00.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.255+0000 7fafcbfff700 1 -- 172.21.15.44:0/602478410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe0007440 con 0x7fafec0a72d0 2020-11-20T12:19:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.255+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec063060 con 0x7fafec0a72d0 2020-11-20T12:19:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.255+0000 7fafcbfff700 1 -- 172.21.15.44:0/602478410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0005800 con 0x7fafec0a72d0 2020-11-20T12:19:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.255+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec133720 con 0x7fafec0a72d0 2020-11-20T12:19:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.256+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec13d950 con 0x7fafec0a72d0 2020-11-20T12:19:00.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.256+0000 7fafcbfff700 1 -- 172.21.15.44:0/602478410 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fafe0010070 con 0x7fafec0a72d0 2020-11-20T12:19:00.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.256+0000 7fafcbfff700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fafc4056cd0 0x7fafc4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.257+0000 7fafcbfff700 1 -- 172.21.15.44:0/602478410 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fafe006f750 con 0x7fafec0a72d0 2020-11-20T12:19:00.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.257+0000 7fafebfff700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fafc4056cd0 0x7fafc4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.257+0000 7fafebfff700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fafc4056cd0 0x7fafc4059190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fafec0ac400 tx=0x7fafdc015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.268 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.261+0000 7fafcbfff700 1 -- 172.21.15.44:0/602478410 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fafec13d950 con 0x7fafec0a72d0 2020-11-20T12:19:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.565+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fafec063250 con 0x7fafec0a72d0 2020-11-20T12:19:00.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.565+0000 7fafcbfff700 1 -- 172.21.15.44:0/602478410 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fafe0044a20 con 0x7fafec0a72d0 2020-11-20T12:19:00.569 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:00.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fafc4056cd0 msgr2=0x7fafc4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fafc4056cd0 0x7fafc4059190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fafec0ac400 tx=0x7fafdc015000).stop 2020-11-20T12:19:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0a72d0 msgr2=0x7fafec13dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0a72d0 0x7fafec13dbf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fafe000c760 tx=0x7fafe0007ce0).stop 2020-11-20T12:19:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 shutdown_connections 2020-11-20T12:19:00.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fafc4056cd0 0x7fafc4059190 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafec0619a0 0x7fafec13e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fafec067dc0 0x7fafec0a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 --2- 172.21.15.44:0/602478410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafec0a72d0 0x7fafec13dbf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 >> 172.21.15.44:0/602478410 conn(0x7fafec060170 msgr2=0x7fafec0a77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:00.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 shutdown_connections 2020-11-20T12:19:00.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:00.567+0000 7faff27c3700 1 -- 172.21.15.44:0/602478410 wait complete. 2020-11-20T12:19:03.585 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:03.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.723+0000 7f942449c700 1 Processor -- start 2020-11-20T12:19:03.727 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.723+0000 7f942449c700 1 -- start start 2020-11-20T12:19:03.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f942449c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f941c13a990 0x7f941c13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f942449c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13f800 0x7f941c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f942449c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c0a6260 con 0x7f941c060330 2020-11-20T12:19:03.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f942449c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c0a52e0 con 0x7f941c13a990 2020-11-20T12:19:03.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f942449c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0a6490 con 0x7f941c13f800 2020-11-20T12:19:03.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f9421a37700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f941c13a990 0x7f941c13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f9422a39700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13f800 msgr2=0x7f941c141bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:03.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f9422a39700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13f800 0x7f941c141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:03.738 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f9421a37700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f941c13a990 0x7f941c13ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42636/0 (socket says 172.21.15.44:42636) 2020-11-20T12:19:03.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.724+0000 7f9421a37700 1 -- 172.21.15.44:0/3271365637 learned_addr learned my addr 172.21.15.44:0/3271365637 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:03.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421a37700 1 -- 172.21.15.44:0/3271365637 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13f800 msgr2=0x7f941c141bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:03.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421a37700 1 --2- 172.21.15.44:0/3271365637 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13f800 0x7f941c141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:03.739 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421a37700 1 -- 172.21.15.44:0/3271365637 >> v1:172.21.15.79:6789/0 conn(0x7f941c060330 legacy=0x7f941c0a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421a37700 1 -- 172.21.15.44:0/3271365637 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0a6730 con 0x7f941c13a990 2020-11-20T12:19:03.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421a37700 1 --2- 172.21.15.44:0/3271365637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13a990 0x7f941c13ae20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f940c00ba20 tx=0x7f940c006eb0).ready entity=mon.0 client_cookie=e421c2a9b862bda3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:03.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421236700 1 -- 172.21.15.44:0/3271365637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c009890 con 0x7f941c13a990 2020-11-20T12:19:03.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421236700 1 -- 172.21.15.44:0/3271365637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c0099f0 con 0x7f941c13a990 2020-11-20T12:19:03.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f9421236700 1 -- 172.21.15.44:0/3271365637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c009d70 con 0x7f941c13a990 2020-11-20T12:19:03.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f942449c700 1 -- 172.21.15.44:0/3271365637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13a990 msgr2=0x7f941c13ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.725+0000 7f942449c700 1 --2- 172.21.15.44:0/3271365637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13a990 0x7f941c13ae20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f940c00ba20 tx=0x7f940c006eb0).stop 2020-11-20T12:19:03.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.726+0000 7f942449c700 1 -- 172.21.15.44:0/3271365637 shutdown_connections 2020-11-20T12:19:03.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.726+0000 7f942449c700 1 --2- 172.21.15.44:0/3271365637 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13f800 0x7f941c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:03.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.726+0000 7f942449c700 1 --2- 172.21.15.44:0/3271365637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13a990 0x7f941c13ae20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.726+0000 7f942449c700 1 -- 172.21.15.44:0/3271365637 >> 172.21.15.44:0/3271365637 conn(0x7f941c062690 msgr2=0x7f941c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:03.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.726+0000 7f942449c700 1 -- 172.21.15.44:0/3271365637 shutdown_connections 2020-11-20T12:19:03.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.726+0000 7f942449c700 1 -- 172.21.15.44:0/3271365637 wait complete. 2020-11-20T12:19:03.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.727+0000 7f942449c700 1 Processor -- start 2020-11-20T12:19:03.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.727+0000 7f942449c700 1 -- start start 2020-11-20T12:19:03.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f941c060330 0x7f941c13e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13a990 0x7f941c13e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13f800 0x7f941c148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c13da70 con 0x7f941c13f800 2020-11-20T12:19:03.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c134ca0 con 0x7f941c060330 2020-11-20T12:19:03.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c13ed20 con 0x7f941c13a990 2020-11-20T12:19:03.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13f800 0x7f941c148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9421a37700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13a990 msgr2=0x7f941c13e020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:03.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9421a37700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13a990 0x7f941c13e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13f800 0x7f941c148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42644/0 (socket says 172.21.15.44:42644) 2020-11-20T12:19:03.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 -- 172.21.15.44:0/4072335840 learned_addr learned my addr 172.21.15.44:0/4072335840 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422238700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f941c060330 0x7f941c13e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 -- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13a990 msgr2=0x7f941c13e020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13a990 0x7f941c13e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:03.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 -- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f941c060330 msgr2=0x7f941c13e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f941c060330 0x7f941c13e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 -- 172.21.15.44:0/4072335840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c148030 con 0x7f941c13f800 2020-11-20T12:19:03.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f9422a39700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13f800 0x7f941c148490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f941400bae0 tx=0x7f9414006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:03.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f940a7fc700 1 -- 172.21.15.44:0/4072335840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414003c10 con 0x7f941c13f800 2020-11-20T12:19:03.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f940a7fc700 1 -- 172.21.15.44:0/4072335840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9414003d70 con 0x7f941c13f800 2020-11-20T12:19:03.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c13f200 con 0x7f941c13f800 2020-11-20T12:19:03.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f940a7fc700 1 -- 172.21.15.44:0/4072335840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9414006170 con 0x7f941c13f800 2020-11-20T12:19:03.749 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.728+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c148980 con 0x7f941c13f800 2020-11-20T12:19:03.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.729+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c1481f0 con 0x7f941c13f800 2020-11-20T12:19:03.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.730+0000 7f940a7fc700 1 -- 172.21.15.44:0/4072335840 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f94140065a0 con 0x7f941c13f800 2020-11-20T12:19:03.750 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.730+0000 7f940a7fc700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93f4056cd0 0x7f93f4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.730+0000 7f940a7fc700 1 -- 172.21.15.44:0/4072335840 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f9414011020 con 0x7f941c13f800 2020-11-20T12:19:03.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.730+0000 7f9422238700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93f4056cd0 0x7f93f4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.731+0000 7f9422238700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93f4056cd0 0x7f93f4059190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f94180035f0 tx=0x7f9418007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:03.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:03.734+0000 7f940a7fc700 1 -- 172.21.15.44:0/4072335840 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f941c1481f0 con 0x7f941c13f800 2020-11-20T12:19:04.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.026+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f941c1481f0 con 0x7f941c13f800 2020-11-20T12:19:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.026+0000 7f940a7fc700 1 -- 172.21.15.44:0/4072335840 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f941c1481f0 con 0x7f941c13f800 2020-11-20T12:19:05.227 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93f4056cd0 msgr2=0x7f93f4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93f4056cd0 0x7f93f4059190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f94180035f0 tx=0x7f9418007b90).stop 2020-11-20T12:19:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13f800 msgr2=0x7f941c148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13f800 0x7f941c148490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f941400bae0 tx=0x7f9414006f90).stop 2020-11-20T12:19:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 shutdown_connections 2020-11-20T12:19:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f93f4056cd0 0x7f93f4059190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f941c13f800 0x7f941c148490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f941c060330 0x7f941c13e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 --2- 172.21.15.44:0/4072335840 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f941c13a990 0x7f941c13e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:05.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 >> 172.21.15.44:0/4072335840 conn(0x7f941c062690 msgr2=0x7f941c1343b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:05.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 shutdown_connections 2020-11-20T12:19:05.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:04.029+0000 7f942449c700 1 -- 172.21.15.44:0/4072335840 wait complete. 2020-11-20T12:19:08.231 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:08.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f02f08700 1 Processor -- start 2020-11-20T12:19:08.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f02f08700 1 -- start start 2020-11-20T12:19:08.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f02f08700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3efc0a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f02f08700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f02f08700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0a6260 con 0x7f3efc0a4c30 2020-11-20T12:19:08.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f02f08700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3efc0a52e0 con 0x7f3efc13b0a0 2020-11-20T12:19:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f02f08700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0a6490 con 0x7f3efc060330 2020-11-20T12:19:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f00ca4700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 msgr2=0x7f3efc0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3f00ca4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.373+0000 7f3efbfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f3efc0a4c30 0x7f3efc13a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46396/0 (socket says 172.21.15.44:46396) 2020-11-20T12:19:08.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.374+0000 7f3efbfff700 1 -- 172.21.15.44:0/1739444210 learned_addr learned my addr 172.21.15.44:0/1739444210 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.374+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.374+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:19:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.374+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:08.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.374+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:08.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.374+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:08.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.574+0000 7f3f00ca4700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 msgr2=0x7f3efc0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:08.577 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.574+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:19:08.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.575+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.575+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:19:08.578 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.575+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.575+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:08.579 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.575+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:19:08.693 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:08.699 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.975+0000 7f3f00ca4700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 msgr2=0x7f3efc0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:08.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.975+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:19:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.976+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.976+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:19:08.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.976+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.976+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:08.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:08.976+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:19:09.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:09.776+0000 7f3f00ca4700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 msgr2=0x7f3efc0a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:11.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:09.776+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:19:11.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:09.777+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:09.777+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:19:11.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:09.777+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:11.077 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:09.777+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:11.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:09.777+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:19:11.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.304+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429295173 0 0) 0x7f3efc0a6260 con 0x7f3efc0a4c30 2020-11-20T12:19:11.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.304+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee0003240 con 0x7f3efc0a4c30 2020-11-20T12:19:11.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3920813153 0 0) 0x7f3ee0003240 con 0x7f3efc0a4c30 2020-11-20T12:19:11.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 msgr2=0x7f3efc0a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:11.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3efc0a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 msgr2=0x7f3efc141810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 --2- 172.21.15.44:0/1739444210 >> v2:172.21.15.44:3300/0 conn(0x7f3efc13b0a0 0x7f3efc141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc0a6730 con 0x7f3efc0a4c30 2020-11-20T12:19:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f3ee80014c0 con 0x7f3efc0a4c30 2020-11-20T12:19:11.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 >> v1:172.21.15.44:6789/0 conn(0x7f3efc0a4c30 legacy=0x7f3efc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ee0003730 0x7f3ee0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ee0004060 0x7f3ee0004540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3ee0004bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0a52e0 con 0x7f3ee0004060 2020-11-20T12:19:11.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0a6490 con 0x7f3ee0003730 2020-11-20T12:19:11.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0005180 con 0x7f3efc060330 2020-11-20T12:19:11.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 msgr2=0x7f3ee0004bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:11.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3f014a5700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3ee0004bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:11.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3efbfff700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ee0004060 0x7f3ee0004540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.307+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ee0003730 0x7f3ee0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.308+0000 7f3efbfff700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 msgr2=0x7f3ee0004bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:11.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.308+0000 7f3efbfff700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc060330 0x7f3ee0004bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.308+0000 7f3efbfff700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ee0003730 msgr2=0x7f3ee0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:11.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.308+0000 7f3efbfff700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ee0003730 0x7f3ee0003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.308+0000 7f3efbfff700 1 -- 172.21.15.44:0/1739444210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0003240 con 0x7f3ee0004060 2020-11-20T12:19:11.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.308+0000 7f3efbfff700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ee0004060 0x7f3ee0004540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3ee800c410 tx=0x7f3efc0a5880).ready entity=mon.0 client_cookie=6799687172cbf7d2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.309+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80070d0 con 0x7f3ee0004060 2020-11-20T12:19:11.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8007230 con 0x7f3ee0004060 2020-11-20T12:19:11.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3efb7fe700 1 -- 172.21.15.44:0/1739444210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80055f0 con 0x7f3ee0004060 2020-11-20T12:19:11.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3f02f08700 1 -- 172.21.15.44:0/1739444210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ee0004060 msgr2=0x7f3ee0004540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3f02f08700 1 --2- 172.21.15.44:0/1739444210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ee0004060 0x7f3ee0004540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3ee800c410 tx=0x7f3efc0a5880).stop 2020-11-20T12:19:11.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3f014a5700 1 -- 172.21.15.44:0/1739444210 reap_dead start 2020-11-20T12:19:11.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3f02f08700 1 -- 172.21.15.44:0/1739444210 shutdown_connections 2020-11-20T12:19:11.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3f02f08700 1 -- 172.21.15.44:0/1739444210 >> 172.21.15.44:0/1739444210 conn(0x7f3efc062690 msgr2=0x7f3efc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3f02f08700 1 -- 172.21.15.44:0/1739444210 shutdown_connections 2020-11-20T12:19:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.310+0000 7f3f02f08700 1 -- 172.21.15.44:0/1739444210 wait complete. 2020-11-20T12:19:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 Processor -- start 2020-11-20T12:19:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 -- start start 2020-11-20T12:19:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3efc066c20 0x7f3efc0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc09d680 0x7f3efc0ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3efc09f9f0 0x7f3efc0aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0ad960 con 0x7f3efc09f9f0 2020-11-20T12:19:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc09db90 con 0x7f3efc066c20 2020-11-20T12:19:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.312+0000 7f3f02f08700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc09de30 con 0x7f3efc09d680 2020-11-20T12:19:11.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3efbfff700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc09d680 msgr2=0x7f3efc0ad470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3efbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc09d680 0x7f3efc0ad470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3efc09f9f0 0x7f3efc0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3efc09f9f0 0x7f3efc0aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42678/0 (socket says 172.21.15.44:42678) 2020-11-20T12:19:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 -- 172.21.15.44:0/437449100 learned_addr learned my addr 172.21.15.44:0/437449100 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:11.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3efc066c20 0x7f3efc0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 -- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc09d680 msgr2=0x7f3efc0ad470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:11.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc09d680 0x7f3efc0ad470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 -- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3efc066c20 msgr2=0x7f3efc0acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3efc066c20 0x7f3efc0acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 -- 172.21.15.44:0/437449100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc067000 con 0x7f3efc09f9f0 2020-11-20T12:19:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3efc066c20 0x7f3efc0acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f014a5700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3efc09f9f0 0x7f3efc0aac60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3ef4006fd0 tx=0x7f3ef4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3ef8ff9700 1 -- 172.21.15.44:0/437449100 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4010920 con 0x7f3efc09f9f0 2020-11-20T12:19:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3ef8ff9700 1 -- 172.21.15.44:0/437449100 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef400c040 con 0x7f3efc09f9f0 2020-11-20T12:19:11.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc09e0d0 con 0x7f3efc09f9f0 2020-11-20T12:19:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3ef8ff9700 1 -- 172.21.15.44:0/437449100 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4010cb0 con 0x7f3efc09f9f0 2020-11-20T12:19:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.313+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc0ab150 con 0x7f3efc09f9f0 2020-11-20T12:19:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.314+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc061650 con 0x7f3efc09f9f0 2020-11-20T12:19:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.315+0000 7f3ef8ff9700 1 -- 172.21.15.44:0/437449100 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3ef4027070 con 0x7f3efc09f9f0 2020-11-20T12:19:11.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.315+0000 7f3ef8ff9700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3ed4056cd0 0x7f3ed4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.315+0000 7f3ef8ff9700 1 -- 172.21.15.44:0/437449100 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3ef4070670 con 0x7f3efc09f9f0 2020-11-20T12:19:11.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.315+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3ed4056cd0 0x7f3ed4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.316+0000 7f3f00ca4700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3ed4056cd0 0x7f3ed4059190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f3ef0003650 tx=0x7f3ef0009220).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.319+0000 7f3ef8ff9700 1 -- 172.21.15.44:0/437449100 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3ef4045940 con 0x7f3efc09f9f0 2020-11-20T12:19:11.612 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.608+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3efc0446d0 con 0x7f3efc09f9f0 2020-11-20T12:19:11.613 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.609+0000 7f3ef8ff9700 1 -- 172.21.15.44:0/437449100 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f3ef4048f60 con 0x7f3efc09f9f0 2020-11-20T12:19:11.613 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":156,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:11.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3ed4056cd0 msgr2=0x7f3ed4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3ed4056cd0 0x7f3ed4059190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f3ef0003650 tx=0x7f3ef0009220).stop 2020-11-20T12:19:11.615 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3efc09f9f0 msgr2=0x7f3efc0aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3efc09f9f0 0x7f3efc0aac60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3ef4006fd0 tx=0x7f3ef4008040).stop 2020-11-20T12:19:11.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 shutdown_connections 2020-11-20T12:19:11.616 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3ed4056cd0 0x7f3ed4059190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3efc09f9f0 0x7f3efc0aac60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.617 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3efc066c20 0x7f3efc0acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.610+0000 7f3f02f08700 1 --2- 172.21.15.44:0/437449100 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3efc09d680 0x7f3efc0ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.611+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 >> 172.21.15.44:0/437449100 conn(0x7f3efc062690 msgr2=0x7f3efc138ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.611+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 shutdown_connections 2020-11-20T12:19:11.618 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.611+0000 7f3f02f08700 1 -- 172.21.15.44:0/437449100 wait complete. 2020-11-20T12:19:11.626 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:19:11.627 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:19:11.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.768+0000 7f83c39af700 1 Processor -- start 2020-11-20T12:19:11.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.769+0000 7f83c39af700 1 -- start start 2020-11-20T12:19:11.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.769+0000 7f83c39af700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 0x7f83bc0a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c39af700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83bc0a9770 con 0x7f83bc13ac60 2020-11-20T12:19:11.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c39af700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83bc0a6da0 con 0x7f83bc0aa400 2020-11-20T12:19:11.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c39af700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0a56c0 con 0x7f83bc0aa020 2020-11-20T12:19:11.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c1f4c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f83bc13ac60 0x7f83bc0a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46432/0 (socket says 172.21.15.44:46432) 2020-11-20T12:19:11.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c174b700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 msgr2=0x7f83bc0a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:11.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c1f4c700 1 -- 172.21.15.44:0/2449602531 learned_addr learned my addr 172.21.15.44:0/2449602531 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:11.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c174b700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 0x7f83bc0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:11.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511934697 0 0) 0x7f83b4001450 con 0x7f83bc13ac60 2020-11-20T12:19:11.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a4003240 con 0x7f83bc13ac60 2020-11-20T12:19:11.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1424445315 0 0) 0x7f83a4003240 con 0x7f83bc13ac60 2020-11-20T12:19:11.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 msgr2=0x7f83bc0a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:11.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 0x7f83bc0a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 >> v1:172.21.15.79:6789/0 conn(0x7f83bc0aa400 legacy=0x7f83bc0a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc0a7040 con 0x7f83bc13ac60 2020-11-20T12:19:11.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f83b40019f0 con 0x7f83bc13ac60 2020-11-20T12:19:11.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 >> v1:172.21.15.44:6789/0 conn(0x7f83bc13ac60 legacy=0x7f83bc0a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a4003760 0x7f83a4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 0x7f83a40044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a40049d0 0x7f83a4004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0a56c0 con 0x7f83a40049d0 2020-11-20T12:19:11.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4004140 con 0x7f83a4003760 2020-11-20T12:19:11.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4004300 con 0x7f83bc0aa020 2020-11-20T12:19:11.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.770+0000 7f83c0f4a700 1 -- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 msgr2=0x7f83a40044e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:11.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c0f4a700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 0x7f83a40044e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:11.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c174b700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a4003760 0x7f83a4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c1f4c700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a40049d0 0x7f83a4004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c174b700 1 -- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 msgr2=0x7f83a40044e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:11.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c174b700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0aa020 0x7f83a40044e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c174b700 1 -- 172.21.15.44:0/2449602531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a40049d0 msgr2=0x7f83a4004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:11.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c174b700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a40049d0 0x7f83a4004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c174b700 1 -- 172.21.15.44:0/2449602531 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4003240 con 0x7f83a4003760 2020-11-20T12:19:11.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c1f4c700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a40049d0 0x7f83a4004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:11.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83c174b700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a4003760 0x7f83a4003b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f83b8003650 tx=0x7f83b8006e90).ready entity=mon.1 client_cookie=7d7f4ea66c0db89a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80050d0 con 0x7f83a4003760 2020-11-20T12:19:11.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8005250 con 0x7f83a4003760 2020-11-20T12:19:11.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.771+0000 7f83abfff700 1 -- 172.21.15.44:0/2449602531 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80038e0 con 0x7f83a4003760 2020-11-20T12:19:11.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.772+0000 7f83c39af700 1 -- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a4003760 msgr2=0x7f83a4003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.772+0000 7f83c39af700 1 --2- 172.21.15.44:0/2449602531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83a4003760 0x7f83a4003b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f83b8003650 tx=0x7f83b8006e90).stop 2020-11-20T12:19:11.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.772+0000 7f83c1f4c700 1 -- 172.21.15.44:0/2449602531 reap_dead start 2020-11-20T12:19:11.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.772+0000 7f83c39af700 1 -- 172.21.15.44:0/2449602531 shutdown_connections 2020-11-20T12:19:11.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.772+0000 7f83c39af700 1 -- 172.21.15.44:0/2449602531 >> 172.21.15.44:0/2449602531 conn(0x7f83bc062690 msgr2=0x7f83bc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.772+0000 7f83c39af700 1 -- 172.21.15.44:0/2449602531 shutdown_connections 2020-11-20T12:19:11.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.772+0000 7f83c39af700 1 -- 172.21.15.44:0/2449602531 wait complete. 2020-11-20T12:19:11.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.773+0000 7f83c39af700 1 Processor -- start 2020-11-20T12:19:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.773+0000 7f83c39af700 1 -- start start 2020-11-20T12:19:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c39af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a4003760 0x7f83bc13e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c39af700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83bc0ac1d0 0x7f83bc0650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c39af700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0674f0 0x7f83bc067970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c39af700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc066080 con 0x7f83a4003760 2020-11-20T12:19:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c39af700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc147230 con 0x7f83bc0ac1d0 2020-11-20T12:19:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c39af700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1474d0 con 0x7f83bc0674f0 2020-11-20T12:19:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c174b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a4003760 0x7f83bc13e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c174b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a4003760 0x7f83bc13e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42694/0 (socket says 172.21.15.44:42694) 2020-11-20T12:19:11.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c174b700 1 -- 172.21.15.44:0/2522892198 learned_addr learned my addr 172.21.15.44:0/2522892198 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c1f4c700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0674f0 msgr2=0x7f83bc067970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c1f4c700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0674f0 0x7f83bc067970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.774+0000 7f83c0f4a700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83bc0ac1d0 0x7f83bc0650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c174b700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0674f0 msgr2=0x7f83bc067970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:11.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c174b700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f83bc0674f0 0x7f83bc067970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c174b700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83bc0ac1d0 msgr2=0x7f83bc0650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c174b700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83bc0ac1d0 0x7f83bc0650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c174b700 1 -- 172.21.15.44:0/2522892198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc147770 con 0x7f83a4003760 2020-11-20T12:19:11.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c174b700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a4003760 0x7f83bc13e9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f83b8006820 tx=0x7f83b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8021030 con 0x7f83a4003760 2020-11-20T12:19:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8018070 con 0x7f83a4003760 2020-11-20T12:19:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc065590 con 0x7f83a4003760 2020-11-20T12:19:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8006860 con 0x7f83a4003760 2020-11-20T12:19:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.775+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc0adae0 con 0x7f83a4003760 2020-11-20T12:19:11.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.776+0000 7f83c39af700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc146e40 0x7f83bc147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.776+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc068020 con 0x7f83bc146e40 2020-11-20T12:19:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.776+0000 7f83c0f4a700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc146e40 0x7f83bc147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.776+0000 7f83c0f4a700 1 -- 172.21.15.44:0/2522892198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83bc147df0 con 0x7f83bc146e40 2020-11-20T12:19:11.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.776+0000 7f83c0f4a700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc146e40 0x7f83bc147900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f83bc062aa0 tx=0x7f83ac009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f83b800f070 con 0x7f83a4003760 2020-11-20T12:19:11.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8394056cd0 0x7f8394059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0070d0 con 0x7f83bc146e40 2020-11-20T12:19:11.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f83b8022070 con 0x7f83a4003760 2020-11-20T12:19:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f83ac007230 con 0x7f83bc146e40 2020-11-20T12:19:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc146e40 msgr2=0x7f83bc147900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc146e40 0x7f83bc147900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f83bc062aa0 tx=0x7f83ac009040).stop 2020-11-20T12:19:11.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc146e40 msgr2=0x7f83bc147900 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83a97fa700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc146e40 0x7f83bc147900 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0x7f83bc062aa0 tx=0x7f83ac009040).stop 2020-11-20T12:19:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.777+0000 7f83c1f4c700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8394056cd0 0x7f8394059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.778+0000 7f83c1f4c700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8394056cd0 0x7f8394059190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f83bc0aa880 tx=0x7f83bc0a5f30).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.789+0000 7f83c39af700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc14a410 0x7f83bc14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.789+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc061650 con 0x7f83bc14a410 2020-11-20T12:19:11.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.789+0000 7f83c0f4a700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc14a410 0x7f83bc14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83c0f4a700 1 -- 172.21.15.44:0/2522892198 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f83bc0446d0 con 0x7f83bc14a410 2020-11-20T12:19:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83c0f4a700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc14a410 0x7f83bc14a7f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f83bc05ef10 tx=0x7f83ac018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac005cb0 con 0x7f83bc14a410 2020-11-20T12:19:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4767 (secure 0 0 0) 0x7f83ac005e10 con 0x7f83bc14a410 2020-11-20T12:19:11.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc14a410 msgr2=0x7f83bc14a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83a97fa700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc14a410 0x7f83bc14a7f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f83bc05ef10 tx=0x7f83ac018000).stop 2020-11-20T12:19:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83a97fa700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc14a410 msgr2=0x7f83bc14a7f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.790+0000 7f83a97fa700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83bc14a410 0x7f83bc14a7f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8394056cd0 msgr2=0x7f8394059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8394056cd0 0x7f8394059190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f83bc0aa880 tx=0x7f83bc0a5f30).stop 2020-11-20T12:19:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a4003760 msgr2=0x7f83bc13e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 --2- 172.21.15.44:0/2522892198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83a4003760 0x7f83bc13e9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f83b8006820 tx=0x7f83b8015000).stop 2020-11-20T12:19:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c1f4c700 1 -- 172.21.15.44:0/2522892198 reap_dead start 2020-11-20T12:19:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c1f4c700 1 -- 172.21.15.44:0/2522892198 reap_dead start 2020-11-20T12:19:11.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 shutdown_connections 2020-11-20T12:19:11.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 >> 172.21.15.44:0/2522892198 conn(0x7f83bc062690 msgr2=0x7f83bc135600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 shutdown_connections 2020-11-20T12:19:11.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.791+0000 7f83c39af700 1 -- 172.21.15.44:0/2522892198 wait complete. 2020-11-20T12:19:11.802 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:19:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 156, 2020-11-20T12:19:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: 1 2020-11-20T12:19:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:19:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:11.814 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:11.814 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:11.814 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:11.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.815 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:11.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.815 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:11.815 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:11.815 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:11.815 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:11.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.816 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:11.816 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:11.816 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:11.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.817 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:11.817 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:11.817 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:11.817 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.818 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:11.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.818 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:11.818 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:11.818 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:11.819 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:11.819 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:11.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.819 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:11.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:11.820 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:11.820 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:11.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:11.821 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:11.821 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:11.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:11.822 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:11.822 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:11.822 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.822 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:11.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.823 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:11.823 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:11.823 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:11.823 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:11.823 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:11.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.824 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:11.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:11.824 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:11.825 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:11.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:11.825 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:11.826 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:11.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.826 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:11.826 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:11.827 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:11.827 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.827 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:11.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.827 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:11.828 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:11.828 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:11.828 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:11.828 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:11.828 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.829 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:11.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.829 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:11.829 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:11.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.830 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:11.830 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:11.830 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:11.830 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.830 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:11.831 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:19:11.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.831 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:11.831 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:11.831 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:19:11.832 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.832 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:11.832 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:19:11.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.832 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:11.833 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:11.833 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:11.833 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.833 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:11.833 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:11.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.834 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:11.834 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:11.834 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:11.834 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.835 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:11.835 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:19:11.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:11.835 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:11.835 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:11.836 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:11.836 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:11.836 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:11.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:11.836 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:11.837 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:11.838 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:19:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.977+0000 7fd6e759e700 1 Processor -- start 2020-11-20T12:19:11.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.977+0000 7fd6e759e700 1 -- start start 2020-11-20T12:19:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e759e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6e813a2c0 0x7fd6e8067a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e759e700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e80a6dc0 con 0x7fd6e80a4c70 2020-11-20T12:19:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e759e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e80a6880 con 0x7fd6e80a9810 2020-11-20T12:19:11.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e759e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e80a6b20 con 0x7fd6e813a2c0 2020-11-20T12:19:11.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e5d9b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd6e813a2c0 0x7fd6e8067a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e6d9d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd6e80a4c70 0x7fd6e8141660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46456/0 (socket says 172.21.15.44:46456) 2020-11-20T12:19:11.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e6d9d700 1 -- 172.21.15.44:0/701481712 learned_addr learned my addr 172.21.15.44:0/701481712 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:11.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e5d9b700 1 -- 172.21.15.44:0/701481712 >> v1:172.21.15.79:6789/0 conn(0x7fd6e80a9810 legacy=0x7fd6e8067380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e5d9b700 1 -- 172.21.15.44:0/701481712 >> v1:172.21.15.44:6789/0 conn(0x7fd6e80a4c70 legacy=0x7fd6e8141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e5d9b700 1 -- 172.21.15.44:0/701481712 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80a5100 con 0x7fd6e813a2c0 2020-11-20T12:19:11.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.978+0000 7fd6e5d9b700 1 --2- 172.21.15.44:0/701481712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e813a2c0 0x7fd6e8067a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd6dc00b9a0 tx=0x7fd6dc006df0).ready entity=mon.0 client_cookie=56ef5e6c9e8430a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e559a700 1 -- 172.21.15.44:0/701481712 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc009840 con 0x7fd6e813a2c0 2020-11-20T12:19:11.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e559a700 1 -- 172.21.15.44:0/701481712 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc0099a0 con 0x7fd6e813a2c0 2020-11-20T12:19:11.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e559a700 1 -- 172.21.15.44:0/701481712 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc009d20 con 0x7fd6e813a2c0 2020-11-20T12:19:11.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e759e700 1 -- 172.21.15.44:0/701481712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e813a2c0 msgr2=0x7fd6e8067a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e759e700 1 --2- 172.21.15.44:0/701481712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e813a2c0 0x7fd6e8067a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd6dc00b9a0 tx=0x7fd6dc006df0).stop 2020-11-20T12:19:11.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e759e700 1 -- 172.21.15.44:0/701481712 shutdown_connections 2020-11-20T12:19:11.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e759e700 1 --2- 172.21.15.44:0/701481712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e813a2c0 0x7fd6e8067a90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e759e700 1 -- 172.21.15.44:0/701481712 >> 172.21.15.44:0/701481712 conn(0x7fd6e8061b60 msgr2=0x7fd6e8061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e759e700 1 -- 172.21.15.44:0/701481712 shutdown_connections 2020-11-20T12:19:11.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.979+0000 7fd6e759e700 1 -- 172.21.15.44:0/701481712 wait complete. 2020-11-20T12:19:11.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.980+0000 7fd6e759e700 1 Processor -- start 2020-11-20T12:19:11.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.980+0000 7fd6e759e700 1 -- start start 2020-11-20T12:19:11.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e759e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a4c70 0x7fd6e809d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e759e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e80a9810 0x7fd6e8136800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e759e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e813a2c0 0x7fd6e8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e759e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8136cf0 con 0x7fd6e80a9810 2020-11-20T12:19:11.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e759e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8134c00 con 0x7fd6e80a4c70 2020-11-20T12:19:11.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e759e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8135120 con 0x7fd6e813a2c0 2020-11-20T12:19:11.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e6d9d700 1 -- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e813a2c0 msgr2=0x7fd6e8148f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3301/0 2020-11-20T12:19:11.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e80a9810 0x7fd6e8136800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e6d9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e813a2c0 0x7fd6e8148f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e80a9810 0x7fd6e8136800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42714/0 (socket says 172.21.15.44:42714) 2020-11-20T12:19:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e5d9b700 1 -- 172.21.15.44:0/2596813701 learned_addr learned my addr 172.21.15.44:0/2596813701 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.981+0000 7fd6e659c700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a4c70 0x7fd6e809d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e5d9b700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e813a2c0 msgr2=0x7fd6e8148f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e5d9b700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd6e813a2c0 0x7fd6e8148f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e5d9b700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a4c70 msgr2=0x7fd6e809d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e5d9b700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a4c70 0x7fd6e809d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e5d9b700 1 -- 172.21.15.44:0/2596813701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e81353c0 con 0x7fd6e80a9810 2020-11-20T12:19:11.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e5d9b700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e80a9810 0x7fd6e8136800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd6dc000dd0 tx=0x7fd6dc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc024400 con 0x7fd6e80a9810 2020-11-20T12:19:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc024560 con 0x7fd6e80a9810 2020-11-20T12:19:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e8135660 con 0x7fd6e80a9810 2020-11-20T12:19:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0248e0 con 0x7fd6e80a9810 2020-11-20T12:19:11.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.982+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e8066b50 con 0x7fd6e80a9810 2020-11-20T12:19:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.983+0000 7fd6e759e700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e8135a20 0x7fd6e80a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.983+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8149ee0 con 0x7fd6e8135a20 2020-11-20T12:19:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.983+0000 7fd6e659c700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e8135a20 0x7fd6e80a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.984+0000 7fd6e659c700 1 -- 172.21.15.44:0/2596813701 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6e81357f0 con 0x7fd6e8135a20 2020-11-20T12:19:11.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.984+0000 7fd6e659c700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e8135a20 0x7fd6e80a31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd6e8137920 tx=0x7fd6d8009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.984+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800d6c0 con 0x7fd6e8135a20 2020-11-20T12:19:11.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.984+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fd6dc021020 con 0x7fd6e80a9810 2020-11-20T12:19:11.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.984+0000 7fd6d67fc700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6bc056dc0 0x7fd6bc059280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.984+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fd6dc073620 con 0x7fd6e80a9810 2020-11-20T12:19:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.985+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fd6d800d820 con 0x7fd6e8135a20 2020-11-20T12:19:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.985+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e8135a20 msgr2=0x7fd6e80a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.985+0000 7fd6d67fc700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e8135a20 0x7fd6e80a31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd6e8137920 tx=0x7fd6d8009040).stop 2020-11-20T12:19:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.985+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e8135a20 msgr2=0x7fd6e80a31c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:11.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.985+0000 7fd6d67fc700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e8135a20 0x7fd6e80a31c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.985+0000 7fd6e6d9d700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6bc056dc0 0x7fd6bc059280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.985+0000 7fd6e6d9d700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6bc056dc0 0x7fd6bc059280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fd6e8137bc0 tx=0x7fd6e0007bc0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:12.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.994+0000 7fd6e759e700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a36b0 0x7fd6e81459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.994+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8061180 con 0x7fd6e80a36b0 2020-11-20T12:19:12.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.994+0000 7fd6e659c700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a36b0 0x7fd6e81459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.994+0000 7fd6e659c700 1 -- 172.21.15.44:0/2596813701 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd6e8057d80 con 0x7fd6e80a36b0 2020-11-20T12:19:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.994+0000 7fd6e659c700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a36b0 0x7fd6e81459f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd6d8010010 tx=0x7fd6d8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.994+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8014c30 con 0x7fd6e80a36b0 2020-11-20T12:19:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4595 (secure 0 0 0) 0x7fd6d800d4b0 con 0x7fd6e80a36b0 2020-11-20T12:19:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a36b0 msgr2=0x7fd6e81459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6d67fc700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a36b0 0x7fd6e81459f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd6d8010010 tx=0x7fd6d8016000).stop 2020-11-20T12:19:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6d67fc700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a36b0 msgr2=0x7fd6e81459f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6d67fc700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd6e80a36b0 0x7fd6e81459f0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0x7fd6d8010010 tx=0x7fd6d8016000).stop 2020-11-20T12:19:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6bc056dc0 msgr2=0x7fd6bc059280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fd6bc056dc0 0x7fd6bc059280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fd6e8137bc0 tx=0x7fd6e0007bc0).stop 2020-11-20T12:19:12.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e80a9810 msgr2=0x7fd6e8136800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 --2- 172.21.15.44:0/2596813701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6e80a9810 0x7fd6e8136800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd6dc000dd0 tx=0x7fd6dc017000).stop 2020-11-20T12:19:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e6d9d700 1 -- 172.21.15.44:0/2596813701 reap_dead start 2020-11-20T12:19:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e6d9d700 1 -- 172.21.15.44:0/2596813701 reap_dead start 2020-11-20T12:19:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 shutdown_connections 2020-11-20T12:19:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 >> 172.21.15.44:0/2596813701 conn(0x7fd6e8061b60 msgr2=0x7fd6e80a7560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 shutdown_connections 2020-11-20T12:19:12.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:11.995+0000 7fd6e759e700 1 -- 172.21.15.44:0/2596813701 wait complete. 2020-11-20T12:19:12.004 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:19:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 156, 2020-11-20T12:19:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: 1 2020-11-20T12:19:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:19:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:19:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:19:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:19:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:12.039 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:12.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:19:32.041 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2020-11-20T12:19:32.041 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2020-11-20T12:19:32.042 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-20T12:19:32.042 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-20T12:19:32.051 INFO:tasks.ceph.mon.c:Started 2020-11-20T12:19:32.057 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:19:32.060 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.220+0000 7f6215eec700 1 Processor -- start 2020-11-20T12:19:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.220+0000 7f6215eec700 1 -- start start 2020-11-20T12:19:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f6215eec700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f621013a990 0x7f621013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f6215eec700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f6210141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f6215eec700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62100a6260 con 0x7f6210060330 2020-11-20T12:19:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f6215eec700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100a52e0 con 0x7f621013a990 2020-11-20T12:19:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f6215eec700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100a6490 con 0x7f621013f800 2020-11-20T12:19:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620effd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f621013a990 0x7f621013ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620f7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6210060330 0x7f62100a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46474/0 (socket says 172.21.15.44:46474) 2020-11-20T12:19:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620f7fe700 1 -- 172.21.15.44:0/3851382523 learned_addr learned my addr 172.21.15.44:0/3851382523 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620ffff700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f6210141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620ffff700 1 -- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 msgr2=0x7f6210141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:19:32.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620ffff700 1 -- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 msgr2=0x7f6210141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:32.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620ffff700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f6210141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:32.243 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620effd700 1 -- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 msgr2=0x7f6210141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:32.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.221+0000 7f620ffff700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f6210141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:32.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620effd700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f6210141bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620effd700 1 -- 172.21.15.44:0/3851382523 >> v1:172.21.15.44:6789/0 conn(0x7f6210060330 legacy=0x7f62100a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.244 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620e7fc700 1 -- 172.21.15.44:0/3851382523 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823278485 0 0) 0x7f62100a6260 con 0x7f6210060330 2020-11-20T12:19:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620effd700 1 -- 172.21.15.44:0/3851382523 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100a6730 con 0x7f621013a990 2020-11-20T12:19:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620effd700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621013a990 0x7f621013ae20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f620400dc30 tx=0x7f6204006d80).ready entity=mon.0 client_cookie=fd8306ee546fe29e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620e7fc700 1 -- 172.21.15.44:0/3851382523 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400bd60 con 0x7f621013a990 2020-11-20T12:19:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620e7fc700 1 -- 172.21.15.44:0/3851382523 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62040153f0 con 0x7f621013a990 2020-11-20T12:19:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f620e7fc700 1 -- 172.21.15.44:0/3851382523 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204015770 con 0x7f621013a990 2020-11-20T12:19:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f6215eec700 1 -- 172.21.15.44:0/3851382523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621013a990 msgr2=0x7f621013ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f6215eec700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621013a990 0x7f621013ae20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f620400dc30 tx=0x7f6204006d80).stop 2020-11-20T12:19:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f6215eec700 1 -- 172.21.15.44:0/3851382523 shutdown_connections 2020-11-20T12:19:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f6215eec700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f6210141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f6215eec700 1 --2- 172.21.15.44:0/3851382523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f621013a990 0x7f621013ae20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.222+0000 7f6215eec700 1 -- 172.21.15.44:0/3851382523 >> 172.21.15.44:0/3851382523 conn(0x7f6210062690 msgr2=0x7f62100a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.223+0000 7f6215eec700 1 -- 172.21.15.44:0/3851382523 shutdown_connections 2020-11-20T12:19:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.223+0000 7f6215eec700 1 -- 172.21.15.44:0/3851382523 wait complete. 2020-11-20T12:19:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 Processor -- start 2020-11-20T12:19:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 -- start start 2020-11-20T12:19:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6210060330 0x7f6210134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621013a990 0x7f6210136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f621014d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210135810 con 0x7f6210060330 2020-11-20T12:19:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100a19d0 con 0x7f621013a990 2020-11-20T12:19:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.224+0000 7f6215eec700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62101352d0 con 0x7f621013f800 2020-11-20T12:19:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6210060330 0x7f6210134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6210060330 0x7f6210134ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42732/0 (socket says 172.21.15.44:42732) 2020-11-20T12:19:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 -- 172.21.15.44:0/4125142152 learned_addr learned my addr 172.21.15.44:0/4125142152 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620effd700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621013a990 0x7f6210136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620ffff700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f621014d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620ffff700 1 -- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 msgr2=0x7f621014d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:19:32.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620ffff700 1 -- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 msgr2=0x7f621014d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:32.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620ffff700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f621014d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:32.251 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 -- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 msgr2=0x7f621014d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620ffff700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f621014d960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f621014d960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 -- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621013a990 msgr2=0x7f6210136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621013a990 0x7f6210136d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 -- 172.21.15.44:0/4125142152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210135570 con 0x7f6210060330 2020-11-20T12:19:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f620f7fe700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6210060330 0x7f6210134ca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f620000c750 tx=0x7f6200007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f61ef7fe700 1 -- 172.21.15.44:0/4125142152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62000072e0 con 0x7f6210060330 2020-11-20T12:19:32.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f61ef7fe700 1 -- 172.21.15.44:0/4125142152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6200007440 con 0x7f6210060330 2020-11-20T12:19:32.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6210137520 con 0x7f6210060330 2020-11-20T12:19:32.254 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f61ef7fe700 1 -- 172.21.15.44:0/4125142152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200005800 con 0x7f6210060330 2020-11-20T12:19:32.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.225+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621014e010 con 0x7f6210060330 2020-11-20T12:19:32.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.226+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6210061650 con 0x7f6210060330 2020-11-20T12:19:32.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.227+0000 7f61ef7fe700 1 -- 172.21.15.44:0/4125142152 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f620001a060 con 0x7f6210060330 2020-11-20T12:19:32.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.227+0000 7f61ef7fe700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61e8056d20 0x7f61e80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.255 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.227+0000 7f61ef7fe700 1 -- 172.21.15.44:0/4125142152 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f620006fa00 con 0x7f6210060330 2020-11-20T12:19:32.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.227+0000 7f620effd700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61e8056d20 0x7f61e80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.256 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.227+0000 7f620effd700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61e8056d20 0x7f61e80591e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f620400f010 tx=0x7f6204019000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:32.257 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.231+0000 7f61ef7fe700 1 -- 172.21.15.44:0/4125142152 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6200044bc0 con 0x7f6210060330 2020-11-20T12:19:32.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.533+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f62100446d0 con 0x7f6210060330 2020-11-20T12:19:32.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.533+0000 7f61ef7fe700 1 -- 172.21.15.44:0/4125142152 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f6200018070 con 0x7f6210060330 2020-11-20T12:19:32.538 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":156,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:32.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61e8056d20 msgr2=0x7f61e80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61e8056d20 0x7f61e80591e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f620400f010 tx=0x7f6204019000).stop 2020-11-20T12:19:32.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6210060330 msgr2=0x7f6210134ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6210060330 0x7f6210134ca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f620000c750 tx=0x7f6200007ac0).stop 2020-11-20T12:19:32.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 shutdown_connections 2020-11-20T12:19:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f61e8056d20 0x7f61e80591e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f621013f800 0x7f621014d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6210060330 0x7f6210134ca0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 --2- 172.21.15.44:0/4125142152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621013a990 0x7f6210136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 >> 172.21.15.44:0/4125142152 conn(0x7f6210062690 msgr2=0x7f6210130490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:32.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 shutdown_connections 2020-11-20T12:19:32.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:32.535+0000 7f6215eec700 1 -- 172.21.15.44:0/4125142152 wait complete. 2020-11-20T12:19:35.555 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:35.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.691+0000 7fbe88eb8700 1 Processor -- start 2020-11-20T12:19:35.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.691+0000 7fbe88eb8700 1 -- start start 2020-11-20T12:19:35.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.691+0000 7fbe88eb8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe84066a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:35.700 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe88eb8700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbe84137370 0x7fbe84137820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:35.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe88eb8700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe840a9350 con 0x7fbe84066f80 2020-11-20T12:19:35.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe88eb8700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840a95f0 con 0x7fbe84137370 2020-11-20T12:19:35.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe88eb8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840a9890 con 0x7fbe84061340 2020-11-20T12:19:35.701 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe83fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbe84137370 0x7fbe84137820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe82ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbe84066f80 0x7fbe8413a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46490/0 (socket says 172.21.15.44:46490) 2020-11-20T12:19:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe82ffd700 1 -- 172.21.15.44:0/1349501682 learned_addr learned my addr 172.21.15.44:0/1349501682 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:35.702 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe837fe700 1 --2- 172.21.15.44:0/1349501682 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe84066a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:35.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe83fff700 1 -- 172.21.15.44:0/1349501682 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 msgr2=0x7fbe84066a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:35.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe83fff700 1 --2- 172.21.15.44:0/1349501682 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe84066a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe83fff700 1 -- 172.21.15.44:0/1349501682 >> v1:172.21.15.44:6789/0 conn(0x7fbe84066f80 legacy=0x7fbe8413a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:35.703 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe827fc700 1 -- 172.21.15.44:0/1349501682 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304575067 0 0) 0x7fbe840a9350 con 0x7fbe84066f80 2020-11-20T12:19:35.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe83fff700 1 -- 172.21.15.44:0/1349501682 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84061720 con 0x7fbe84137370 2020-11-20T12:19:35.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.692+0000 7fbe83fff700 1 --2- 172.21.15.44:0/1349501682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe84137820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fbe7c009c90 tx=0x7fbe7c0091d0).ready entity=mon.0 client_cookie=9a86fdf53efc7384 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:35.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe827fc700 1 -- 172.21.15.44:0/1349501682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00bbc0 con 0x7fbe84137370 2020-11-20T12:19:35.704 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe827fc700 1 -- 172.21.15.44:0/1349501682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c00bd20 con 0x7fbe84137370 2020-11-20T12:19:35.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe827fc700 1 -- 172.21.15.44:0/1349501682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c011610 con 0x7fbe84137370 2020-11-20T12:19:35.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 -- 172.21.15.44:0/1349501682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 msgr2=0x7fbe84137820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:35.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/1349501682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe84137820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fbe7c009c90 tx=0x7fbe7c0091d0).stop 2020-11-20T12:19:35.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 -- 172.21.15.44:0/1349501682 shutdown_connections 2020-11-20T12:19:35.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/1349501682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe84137820 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/1349501682 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe84066a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 -- 172.21.15.44:0/1349501682 >> 172.21.15.44:0/1349501682 conn(0x7fbe840a9fa0 msgr2=0x7fbe840aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:35.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 -- 172.21.15.44:0/1349501682 shutdown_connections 2020-11-20T12:19:35.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.693+0000 7fbe88eb8700 1 -- 172.21.15.44:0/1349501682 wait complete. 2020-11-20T12:19:35.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.694+0000 7fbe88eb8700 1 Processor -- start 2020-11-20T12:19:35.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe88eb8700 1 -- start start 2020-11-20T12:19:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe88eb8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe840a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:35.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe88eb8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe84066f80 0x7fbe8413e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:35.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe88eb8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe8413cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:35.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe88eb8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe841334f0 con 0x7fbe84137370 2020-11-20T12:19:35.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe88eb8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8414dd70 con 0x7fbe84066f80 2020-11-20T12:19:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe88eb8700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840a1da0 con 0x7fbe84061340 2020-11-20T12:19:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.695+0000 7fbe83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe8413cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe8413cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42750/0 (socket says 172.21.15.44:42750) 2020-11-20T12:19:35.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 -- 172.21.15.44:0/3643672745 learned_addr learned my addr 172.21.15.44:0/3643672745 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe837fe700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe840a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe82ffd700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe84066f80 0x7fbe8413e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe837fe700 1 -- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 msgr2=0x7fbe840a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:19:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe837fe700 1 -- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 msgr2=0x7fbe840a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:35.713 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe837fe700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe840a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe837fe700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe840a3e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 -- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 msgr2=0x7fbe840a3e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe840a3e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.714 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 -- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe84066f80 msgr2=0x7fbe8413e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe84066f80 0x7fbe8413e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 -- 172.21.15.44:0/3643672745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840a2040 con 0x7fbe84137370 2020-11-20T12:19:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe82ffd700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe84066f80 0x7fbe8413e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:35.715 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe83fff700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe8413cbd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbe7c009210 tx=0x7fbe7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:35.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe637fe700 1 -- 172.21.15.44:0/3643672745 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c011e20 con 0x7fbe84137370 2020-11-20T12:19:35.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe637fe700 1 -- 172.21.15.44:0/3643672745 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c010c40 con 0x7fbe84137370 2020-11-20T12:19:35.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe637fe700 1 -- 172.21.15.44:0/3643672745 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c01d480 con 0x7fbe84137370 2020-11-20T12:19:35.716 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe840a22e0 con 0x7fbe84137370 2020-11-20T12:19:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.696+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8413eac0 con 0x7fbe84137370 2020-11-20T12:19:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.697+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe840a2470 con 0x7fbe84137370 2020-11-20T12:19:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.698+0000 7fbe637fe700 1 -- 172.21.15.44:0/3643672745 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbe7c017070 con 0x7fbe84137370 2020-11-20T12:19:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.698+0000 7fbe637fe700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe5c056d50 0x7fbe5c059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:35.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.698+0000 7fbe637fe700 1 -- 172.21.15.44:0/3643672745 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbe7c072700 con 0x7fbe84137370 2020-11-20T12:19:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.698+0000 7fbe837fe700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe5c056d50 0x7fbe5c059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.699+0000 7fbe837fe700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe5c056d50 0x7fbe5c059210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fbe74006fd0 tx=0x7fbe74008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:35.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.703+0000 7fbe637fe700 1 -- 172.21.15.44:0/3643672745 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbe7c0479d0 con 0x7fbe84137370 2020-11-20T12:19:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.986+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbe84057d80 con 0x7fbe84137370 2020-11-20T12:19:35.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.987+0000 7fbe637fe700 1 -- 172.21.15.44:0/3643672745 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fbe7c015070 con 0x7fbe84137370 2020-11-20T12:19:35.990 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":156,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe5c056d50 msgr2=0x7fbe5c059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe5c056d50 0x7fbe5c059210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fbe74006fd0 tx=0x7fbe74008040).stop 2020-11-20T12:19:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 msgr2=0x7fbe8413cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe8413cbd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbe7c009210 tx=0x7fbe7c016000).stop 2020-11-20T12:19:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 shutdown_connections 2020-11-20T12:19:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbe5c056d50 0x7fbe5c059210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe84137370 0x7fbe8413cbd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbe84061340 0x7fbe840a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 --2- 172.21.15.44:0/3643672745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe84066f80 0x7fbe8413e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.988+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 >> 172.21.15.44:0/3643672745 conn(0x7fbe840a9fa0 msgr2=0x7fbe840aa380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.989+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 shutdown_connections 2020-11-20T12:19:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:35.989+0000 7fbe88eb8700 1 -- 172.21.15.44:0/3643672745 wait complete. 2020-11-20T12:19:38.784 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:38.949 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:39.005 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:39.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.147+0000 7f268eab2700 1 Processor -- start 2020-11-20T12:19:39.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.148+0000 7f268eab2700 1 -- start start 2020-11-20T12:19:39.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.148+0000 7f268eab2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26880aa400 0x7f26880a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.148+0000 7f268eab2700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f268813ac60 0x7f26880a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.148+0000 7f268eab2700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26880a9770 con 0x7f26880aa020 2020-11-20T12:19:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.148+0000 7f268eab2700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26880a56c0 con 0x7f268813ac60 2020-11-20T12:19:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.148+0000 7f268eab2700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26880a6da0 con 0x7f26880aa400 2020-11-20T12:19:39.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f268813ac60 0x7f26880a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f268813ac60 0x7f26880a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42760/0 (socket says 172.21.15.44:42760) 2020-11-20T12:19:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 -- 172.21.15.44:0/2273301489 learned_addr learned my addr 172.21.15.44:0/2273301489 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f2687fff700 1 --2- 172.21.15.44:0/2273301489 >> v2:172.21.15.79:3300/0 conn(0x7f26880aa400 0x7f26880a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 -- 172.21.15.44:0/2273301489 >> v2:172.21.15.79:3300/0 conn(0x7f26880aa400 msgr2=0x7f26880a9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:39.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 --2- 172.21.15.44:0/2273301489 >> v2:172.21.15.79:3300/0 conn(0x7f26880aa400 0x7f26880a9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 -- 172.21.15.44:0/2273301489 >> v1:172.21.15.79:6789/0 conn(0x7f26880aa020 legacy=0x7f26880a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 -- 172.21.15.44:0/2273301489 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880a7040 con 0x7f268813ac60 2020-11-20T12:19:39.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f268d04f700 1 --2- 172.21.15.44:0/2273301489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f26880a3700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f26880a4990 tx=0x7f2680009290).ready entity=mon.0 client_cookie=bb6a01af6b8418a9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f26877fe700 1 -- 172.21.15.44:0/2273301489 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268000bbc0 con 0x7f268813ac60 2020-11-20T12:19:39.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f26877fe700 1 -- 172.21.15.44:0/2273301489 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268000bd20 con 0x7f268813ac60 2020-11-20T12:19:39.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.149+0000 7f26877fe700 1 -- 172.21.15.44:0/2273301489 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680011610 con 0x7f268813ac60 2020-11-20T12:19:39.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 -- 172.21.15.44:0/2273301489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 msgr2=0x7f26880a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.157 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 --2- 172.21.15.44:0/2273301489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f26880a3700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f26880a4990 tx=0x7f2680009290).stop 2020-11-20T12:19:39.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 -- 172.21.15.44:0/2273301489 shutdown_connections 2020-11-20T12:19:39.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 --2- 172.21.15.44:0/2273301489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f26880a3700 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 --2- 172.21.15.44:0/2273301489 >> v2:172.21.15.79:3300/0 conn(0x7f26880aa400 0x7f26880a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 -- 172.21.15.44:0/2273301489 >> 172.21.15.44:0/2273301489 conn(0x7f2688062690 msgr2=0x7f26880a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:39.158 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 -- 172.21.15.44:0/2273301489 shutdown_connections 2020-11-20T12:19:39.159 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.150+0000 7f268eab2700 1 -- 172.21.15.44:0/2273301489 wait complete. 2020-11-20T12:19:39.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.151+0000 7f268eab2700 1 Processor -- start 2020-11-20T12:19:39.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.151+0000 7f268eab2700 1 -- start start 2020-11-20T12:19:39.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268eab2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 0x7f2688139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268eab2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26880aa400 0x7f268813e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268eab2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f268813d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268eab2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268813e690 con 0x7f268813ac60 2020-11-20T12:19:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268eab2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880661b0 con 0x7f26880aa400 2020-11-20T12:19:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268eab2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2688066450 con 0x7f26880aa020 2020-11-20T12:19:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268d04f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f268813d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268d04f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f268813d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42766/0 (socket says 172.21.15.44:42766) 2020-11-20T12:19:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f2687fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26880aa400 0x7f268813e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268d04f700 1 -- 172.21.15.44:0/3191677312 learned_addr learned my addr 172.21.15.44:0/3191677312 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268c84e700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 0x7f2688139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268c84e700 1 -- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 msgr2=0x7f2688139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:19:39.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268c84e700 1 -- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 msgr2=0x7f2688139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:39.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268c84e700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 0x7f2688139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:39.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f268c84e700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 0x7f2688139cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:39.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f2687fff700 1 -- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 msgr2=0x7f2688139cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:39.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f2687fff700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 0x7f2688139cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f2687fff700 1 -- 172.21.15.44:0/3191677312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 msgr2=0x7f268813d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f2687fff700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f268813d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.152+0000 7f2687fff700 1 -- 172.21.15.44:0/3191677312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688132970 con 0x7f26880aa400 2020-11-20T12:19:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.153+0000 7f2687fff700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26880aa400 0x7f268813e1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f267400eba0 tx=0x7f2674006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.153+0000 7f2684ff9700 1 -- 172.21.15.44:0/3191677312 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400c8e0 con 0x7f26880aa400 2020-11-20T12:19:39.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.153+0000 7f2684ff9700 1 -- 172.21.15.44:0/3191677312 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267400ca40 con 0x7f26880aa400 2020-11-20T12:19:39.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.153+0000 7f2684ff9700 1 -- 172.21.15.44:0/3191677312 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400cdc0 con 0x7f26880aa400 2020-11-20T12:19:39.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.153+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2688065dc0 con 0x7f26880aa400 2020-11-20T12:19:39.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.153+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268813d730 con 0x7f26880aa400 2020-11-20T12:19:39.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.154+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2688066610 con 0x7f26880aa400 2020-11-20T12:19:39.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.154+0000 7f2684ff9700 1 -- 172.21.15.44:0/3191677312 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2674012600 con 0x7f26880aa400 2020-11-20T12:19:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.154+0000 7f2684ff9700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2660056d20 0x7f26600591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.155+0000 7f2684ff9700 1 -- 172.21.15.44:0/3191677312 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f26740703e0 con 0x7f26880aa400 2020-11-20T12:19:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.155+0000 7f268c84e700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2660056d20 0x7f26600591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.155+0000 7f268c84e700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2660056d20 0x7f26600591e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f268813eda0 tx=0x7f267c009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.159+0000 7f2684ff9700 1 -- 172.21.15.44:0/3191677312 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2674045650 con 0x7f26880aa400 2020-11-20T12:19:39.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.453+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2688065fb0 con 0x7f26880aa400 2020-11-20T12:19:39.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.454+0000 7f2684ff9700 1 -- 172.21.15.44:0/3191677312 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f2674048c70 con 0x7f26880aa400 2020-11-20T12:19:39.458 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:39.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2660056d20 msgr2=0x7f26600591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2660056d20 0x7f26600591e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f268813eda0 tx=0x7f267c009040).stop 2020-11-20T12:19:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26880aa400 msgr2=0x7f268813e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26880aa400 0x7f268813e1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f267400eba0 tx=0x7f2674006df0).stop 2020-11-20T12:19:39.460 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 shutdown_connections 2020-11-20T12:19:39.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2660056d20 0x7f26600591e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f268813ac60 0x7f268813d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.462 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f26880aa020 0x7f2688139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 --2- 172.21.15.44:0/3191677312 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26880aa400 0x7f268813e1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 >> 172.21.15.44:0/3191677312 conn(0x7f2688062690 msgr2=0x7f26880a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 shutdown_connections 2020-11-20T12:19:39.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.456+0000 7f268eab2700 1 -- 172.21.15.44:0/3191677312 wait complete. 2020-11-20T12:19:39.471 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:19:39.472 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:19:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.627+0000 7f37800ca700 1 Processor -- start 2020-11-20T12:19:39.631 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.628+0000 7f37800ca700 1 -- start start 2020-11-20T12:19:39.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.628+0000 7f37800ca700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f37780a1a40 0x7f37780aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.628+0000 7f37800ca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f3778066b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.628+0000 7f37800ca700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3778137bb0 0x7f3778138040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.628+0000 7f37800ca700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37780a9b30 con 0x7f37780a1a40 2020-11-20T12:19:39.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.628+0000 7f37800ca700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37780ab240 con 0x7f3778137bb0 2020-11-20T12:19:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.628+0000 7f37800ca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3778067470 con 0x7f37780666d0 2020-11-20T12:19:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f37780a1a40 0x7f37780aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f37780a1a40 0x7f37780aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42770/0 (socket says 172.21.15.44:42770) 2020-11-20T12:19:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 -- 172.21.15.44:0/2233614699 learned_addr learned my addr 172.21.15.44:0/2233614699 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:39.636 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377d665700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f3778066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377d665700 1 -- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 msgr2=0x7f3778066b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:19:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377d665700 1 -- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 msgr2=0x7f3778066b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 -- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 msgr2=0x7f3778066b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:39.637 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377d665700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f3778066b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:39.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377d665700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f3778066b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f3778066b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 -- 172.21.15.44:0/2233614699 >> v2:172.21.15.79:3300/0 conn(0x7f3778137bb0 msgr2=0x7f3778138040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 --2- 172.21.15.44:0/2233614699 >> v2:172.21.15.79:3300/0 conn(0x7f3778137bb0 0x7f3778138040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 -- 172.21.15.44:0/2233614699 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3778067710 con 0x7f37780a1a40 2020-11-20T12:19:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377de66700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37780a1a40 0x7f37780aad50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f377813ab70 tx=0x7f3774006eb0).ready entity=mon.0 client_cookie=f2cea0a96bf4f8b1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377ce64700 1 -- 172.21.15.44:0/2233614699 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774009890 con 0x7f37780a1a40 2020-11-20T12:19:39.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377ce64700 1 -- 172.21.15.44:0/2233614699 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37740099f0 con 0x7f37780a1a40 2020-11-20T12:19:39.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.629+0000 7f377ce64700 1 -- 172.21.15.44:0/2233614699 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774009d70 con 0x7f37780a1a40 2020-11-20T12:19:39.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 -- 172.21.15.44:0/2233614699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37780a1a40 msgr2=0x7f37780aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37780a1a40 0x7f37780aad50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f377813ab70 tx=0x7f3774006eb0).stop 2020-11-20T12:19:39.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 -- 172.21.15.44:0/2233614699 shutdown_connections 2020-11-20T12:19:39.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37780a1a40 0x7f37780aad50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 --2- 172.21.15.44:0/2233614699 >> v2:172.21.15.79:3300/0 conn(0x7f3778137bb0 0x7f3778138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:39.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 --2- 172.21.15.44:0/2233614699 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f3778066b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 -- 172.21.15.44:0/2233614699 >> 172.21.15.44:0/2233614699 conn(0x7f37780af030 msgr2=0x7f37780af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 -- 172.21.15.44:0/2233614699 shutdown_connections 2020-11-20T12:19:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.630+0000 7f37800ca700 1 -- 172.21.15.44:0/2233614699 wait complete. 2020-11-20T12:19:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.631+0000 7f37800ca700 1 Processor -- start 2020-11-20T12:19:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.631+0000 7f37800ca700 1 -- start start 2020-11-20T12:19:39.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f37800ca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f37780ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f37800ca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37780a1a40 0x7f377813eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f37800ca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778137bb0 0x7f3778130250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f37800ca700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377813e5b0 con 0x7f3778137bb0 2020-11-20T12:19:39.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f37800ca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3778136140 con 0x7f37780a1a40 2020-11-20T12:19:39.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f37800ca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377813e850 con 0x7f37780666d0 2020-11-20T12:19:39.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f377e667700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778137bb0 0x7f3778130250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f377e667700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778137bb0 0x7f3778130250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42774/0 (socket says 172.21.15.44:42774) 2020-11-20T12:19:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f377e667700 1 -- 172.21.15.44:0/1688451540 learned_addr learned my addr 172.21.15.44:0/1688451540 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:39.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f377de66700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f37780ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f377e667700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 msgr2=0x7f37780ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.632+0000 7f377e667700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f37780666d0 0x7f37780ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f377e667700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37780a1a40 msgr2=0x7f377813eb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f377e667700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37780a1a40 0x7f377813eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:39.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f377e667700 1 -- 172.21.15.44:0/1688451540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377813e230 con 0x7f3778137bb0 2020-11-20T12:19:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f377e667700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778137bb0 0x7f3778130250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f377000bab0 tx=0x7f37700091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000c920 con 0x7f3778137bb0 2020-11-20T12:19:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377000ca80 con 0x7f3778137bb0 2020-11-20T12:19:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3778130740 con 0x7f3778137bb0 2020-11-20T12:19:39.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000cbf0 con 0x7f3778137bb0 2020-11-20T12:19:39.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.633+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37781365e0 con 0x7f3778137bb0 2020-11-20T12:19:39.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.634+0000 7f37800ca700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778136860 0x7f3778136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.634+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3778130cd0 con 0x7f3778136860 2020-11-20T12:19:39.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.634+0000 7f377de66700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778136860 0x7f3778136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.634+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f377001c070 con 0x7f3778137bb0 2020-11-20T12:19:39.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.634+0000 7f377de66700 1 -- 172.21.15.44:0/1688451540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3778137130 con 0x7f3778136860 2020-11-20T12:19:39.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f377de66700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778136860 0x7f3778136c40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f3774006ef0 tx=0x7f3774015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3750056d20 0x7f37500591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f377006fa50 con 0x7f3778137bb0 2020-11-20T12:19:39.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774011e60 con 0x7f3778136860 2020-11-20T12:19:39.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f377401f440 con 0x7f3778136860 2020-11-20T12:19:39.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778136860 msgr2=0x7f3778136c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778136860 0x7f3778136c40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f3774006ef0 tx=0x7f3774015000).stop 2020-11-20T12:19:39.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778136860 msgr2=0x7f3778136c40 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:39.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f3765ffb700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778136860 0x7f3778136c40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.635+0000 7f377d665700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3750056d20 0x7f37500591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.636+0000 7f377d665700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3750056d20 0x7f37500591e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f3778140290 tx=0x7f3768008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.644+0000 7f37800ca700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377814db40 0x7f377814df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.644+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37780ae740 con 0x7f377814db40 2020-11-20T12:19:39.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f377de66700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377814db40 0x7f377814df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f377de66700 1 -- 172.21.15.44:0/1688451540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3778136300 con 0x7f377814db40 2020-11-20T12:19:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f377de66700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377814db40 0x7f377814df50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f377805ef10 tx=0x7f3774020000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377402eb70 con 0x7f377814db40 2020-11-20T12:19:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7f3774011c50 con 0x7f377814db40 2020-11-20T12:19:39.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377814db40 msgr2=0x7f377814df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f3765ffb700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377814db40 0x7f377814df50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f377805ef10 tx=0x7f3774020000).stop 2020-11-20T12:19:39.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f3765ffb700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377814db40 msgr2=0x7f377814df50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:39.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.645+0000 7f3765ffb700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377814db40 0x7f377814df50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3750056d20 msgr2=0x7f37500591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3750056d20 0x7f37500591e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f3778140290 tx=0x7f3768008040).stop 2020-11-20T12:19:39.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778137bb0 msgr2=0x7f3778130250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 --2- 172.21.15.44:0/1688451540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3778137bb0 0x7f3778130250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f377000bab0 tx=0x7f37700091a0).stop 2020-11-20T12:19:39.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f377e667700 1 -- 172.21.15.44:0/1688451540 reap_dead start 2020-11-20T12:19:39.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f377e667700 1 -- 172.21.15.44:0/1688451540 reap_dead start 2020-11-20T12:19:39.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 shutdown_connections 2020-11-20T12:19:39.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 >> 172.21.15.44:0/1688451540 conn(0x7f37780af030 msgr2=0x7f377805ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:39.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 shutdown_connections 2020-11-20T12:19:39.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.646+0000 7f37800ca700 1 -- 172.21.15.44:0/1688451540 wait complete. 2020-11-20T12:19:39.666 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:39.666 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:39.666 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:39.667 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:19:39.667 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 158, 2020-11-20T12:19:39.667 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:39.667 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:39.668 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:19:39.668 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:19:39.668 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.668 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:19:39.669 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:39.669 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:39.669 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:39.669 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:39.669 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:39.670 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:39.670 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:39.670 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:39.670 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:39.671 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:39.671 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:39.671 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.671 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:39.672 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:39.672 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:39.672 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:39.673 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:39.673 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:39.673 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:39.674 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:39.674 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:39.675 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:39.675 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.675 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:39.675 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:39.676 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:39.676 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:39.676 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:39.676 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:39.677 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:39.677 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:39.677 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:39.677 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:39.678 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:39.678 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:39.678 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:39.678 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:39.678 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:39.679 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:39.679 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:39.679 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:39.679 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:39.680 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:39.680 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:39.680 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:39.681 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:39.681 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.681 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:39.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.682 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:39.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.682 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:39.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:39.682 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:39.683 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:39.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.683 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:39.683 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:39.684 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.684 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:39.684 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:39.685 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.685 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.686 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:39.686 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:39.686 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:39.687 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:39.687 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:39.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.688 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:39.688 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:39.688 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:39.688 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:39.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:39.689 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:39.689 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:39.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:39.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.692 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:39.692 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:39.693 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:39.693 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:39.693 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:39.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.694 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:39.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:39.694 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:39.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:39.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:39.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:39.696 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:39.697 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:39.697 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.697 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.698 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:39.698 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:39.698 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:39.699 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:39.699 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:39.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.699 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.700 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:39.700 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:39.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.700 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.701 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.701 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:39.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.702 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.702 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:19:39.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.702 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.703 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.703 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:19:39.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.704 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.704 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:19:39.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.704 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.705 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.705 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:39.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.706 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.706 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:39.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.706 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.706 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:19:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.708 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.709 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:39.710 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:39.711 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:19:39.870 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.867+0000 7f7e95a30700 1 Processor -- start 2020-11-20T12:19:39.871 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.867+0000 7f7e95a30700 1 -- start start 2020-11-20T12:19:39.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e95a30700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e900b17c0 0x7f7e9013abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e95a30700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 0x7f7e900aad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.873 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e95a30700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900a9000 con 0x7f7e900a9740 2020-11-20T12:19:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e95a30700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e900a92a0 con 0x7f7e900b17c0 2020-11-20T12:19:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e95a30700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90138600 con 0x7f7e900b1ba0 2020-11-20T12:19:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 0x7f7e900aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e94a2e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e900b17c0 0x7f7e9013abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 0x7f7e900aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54224/0 (socket says 172.21.15.44:54224) 2020-11-20T12:19:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e8ffff700 1 -- 172.21.15.44:0/563129664 learned_addr learned my addr 172.21.15.44:0/563129664 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e8ffff700 1 -- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 msgr2=0x7f7e900aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:19:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e8ffff700 1 -- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 msgr2=0x7f7e900aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 0x7f7e900aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.868+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 0x7f7e900aad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e94a2e700 1 -- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 msgr2=0x7f7e900aad80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e94a2e700 1 --2- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 0x7f7e900aad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e94a2e700 1 -- 172.21.15.44:0/563129664 >> v1:172.21.15.79:6789/0 conn(0x7f7e900a9740 legacy=0x7f7e90137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e94a2e700 1 -- 172.21.15.44:0/563129664 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900a4c30 con 0x7f7e900b17c0 2020-11-20T12:19:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e94a2e700 1 --2- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900b17c0 0x7f7e9013abc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f7e8400b9a0 tx=0x7f7e84006df0).ready entity=mon.1 client_cookie=9671f0c84b289952 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e8f7fe700 1 -- 172.21.15.44:0/563129664 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e840096b0 con 0x7f7e900b17c0 2020-11-20T12:19:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e8f7fe700 1 -- 172.21.15.44:0/563129664 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e84009810 con 0x7f7e900b17c0 2020-11-20T12:19:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.869+0000 7f7e8f7fe700 1 -- 172.21.15.44:0/563129664 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84009b90 con 0x7f7e900b17c0 2020-11-20T12:19:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 -- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900b17c0 msgr2=0x7f7e9013abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 --2- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900b17c0 0x7f7e9013abc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f7e8400b9a0 tx=0x7f7e84006df0).stop 2020-11-20T12:19:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 -- 172.21.15.44:0/563129664 shutdown_connections 2020-11-20T12:19:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 --2- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900b17c0 0x7f7e9013abc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 --2- 172.21.15.44:0/563129664 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b1ba0 0x7f7e900aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 -- 172.21.15.44:0/563129664 >> 172.21.15.44:0/563129664 conn(0x7f7e90060c10 msgr2=0x7f7e900b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 -- 172.21.15.44:0/563129664 shutdown_connections 2020-11-20T12:19:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.870+0000 7f7e95a30700 1 -- 172.21.15.44:0/563129664 wait complete. 2020-11-20T12:19:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.871+0000 7f7e95a30700 1 Processor -- start 2020-11-20T12:19:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.871+0000 7f7e95a30700 1 -- start start 2020-11-20T12:19:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e95a30700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900a9740 0x7f7e90141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e95a30700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 0x7f7e900a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e95a30700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e900b1ba0 0x7f7e900a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e95a30700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900a64e0 con 0x7f7e900b1ba0 2020-11-20T12:19:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e95a30700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900a6730 con 0x7f7e900a9740 2020-11-20T12:19:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e95a30700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900a69d0 con 0x7f7e900b17c0 2020-11-20T12:19:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e9522f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e900b1ba0 0x7f7e900a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e9522f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e900b1ba0 0x7f7e900a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42792/0 (socket says 172.21.15.44:42792) 2020-11-20T12:19:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e9522f700 1 -- 172.21.15.44:0/867267513 learned_addr learned my addr 172.21.15.44:0/867267513 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 0x7f7e900a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e94a2e700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900a9740 0x7f7e90141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e8ffff700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 msgr2=0x7f7e900a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:19:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e8ffff700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 msgr2=0x7f7e900a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 0x7f7e900a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.872+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 0x7f7e900a5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e94a2e700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 msgr2=0x7f7e900a5120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e94a2e700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7e900b17c0 0x7f7e900a5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e94a2e700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e900b1ba0 msgr2=0x7f7e900a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e94a2e700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e900b1ba0 0x7f7e900a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e94a2e700 1 -- 172.21.15.44:0/867267513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9013e620 con 0x7f7e900a9740 2020-11-20T12:19:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e9522f700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e900b1ba0 0x7f7e900a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:39.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e94a2e700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900a9740 0x7f7e90141eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f7e84012000 tx=0x7f7e84015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84011c40 con 0x7f7e900a9740 2020-11-20T12:19:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e84011da0 con 0x7f7e900a9740 2020-11-20T12:19:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8401e670 con 0x7f7e900a9740 2020-11-20T12:19:39.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e95a30700 1 -- 172.21.15.44:0/867267513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e900a7de0 con 0x7f7e900a9740 2020-11-20T12:19:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.873+0000 7f7e95a30700 1 -- 172.21.15.44:0/867267513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e900a5ec0 con 0x7f7e900a9740 2020-11-20T12:19:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.874+0000 7f7e727fc700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e90136ab0 0x7f7e9013d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.874+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9005c430 con 0x7f7e90136ab0 2020-11-20T12:19:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.874+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e90136ab0 0x7f7e9013d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.874+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7e8401a040 con 0x7f7e900a9740 2020-11-20T12:19:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7e68056d20 0x7f7e680591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8ffff700 1 -- 172.21.15.44:0/867267513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e900a4e80 con 0x7f7e90136ab0 2020-11-20T12:19:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f7e840715e0 con 0x7f7e900a9740 2020-11-20T12:19:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e90136ab0 0x7f7e9013d040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7e9009d980 tx=0x7f7e7c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e9522f700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7e68056d20 0x7f7e680591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c011660 con 0x7f7e90136ab0 2020-11-20T12:19:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e9522f700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7e68056d20 0x7f7e680591e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f7e88000f40 tx=0x7f7e88009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f7e7c0117c0 con 0x7f7e90136ab0 2020-11-20T12:19:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e90136ab0 msgr2=0x7f7e9013d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e90136ab0 0x7f7e9013d040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7e9009d980 tx=0x7f7e7c008040).stop 2020-11-20T12:19:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e90136ab0 msgr2=0x7f7e9013d040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.875+0000 7f7e8cff9700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e90136ab0 0x7f7e9013d040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.886+0000 7f7e727fc700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900611c0 0x7f7e9013d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.886+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900a6870 con 0x7f7e900611c0 2020-11-20T12:19:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.886+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900611c0 0x7f7e9013d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8ffff700 1 -- 172.21.15.44:0/867267513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7e90057d80 con 0x7f7e900611c0 2020-11-20T12:19:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8ffff700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900611c0 0x7f7e9013d530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f7e7c018000 tx=0x7f7e7c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c005b30 con 0x7f7e900611c0 2020-11-20T12:19:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f7e7c011450 con 0x7f7e900611c0 2020-11-20T12:19:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900611c0 msgr2=0x7f7e9013d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8cff9700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900611c0 0x7f7e9013d530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f7e7c018000 tx=0x7f7e7c008040).stop 2020-11-20T12:19:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8cff9700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900611c0 msgr2=0x7f7e9013d530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.887+0000 7f7e8cff9700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900611c0 0x7f7e9013d530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7e68056d20 msgr2=0x7f7e680591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7e68056d20 0x7f7e680591e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f7e88000f40 tx=0x7f7e88009040).stop 2020-11-20T12:19:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900a9740 msgr2=0x7f7e90141eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900a9740 0x7f7e90141eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f7e84012000 tx=0x7f7e84015000).stop 2020-11-20T12:19:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e9522f700 1 -- 172.21.15.44:0/867267513 reap_dead start 2020-11-20T12:19:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 shutdown_connections 2020-11-20T12:19:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 --2- 172.21.15.44:0/867267513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e900a9740 0x7f7e90141eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 >> 172.21.15.44:0/867267513 conn(0x7f7e90060c10 msgr2=0x7f7e900a8420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 shutdown_connections 2020-11-20T12:19:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:39.888+0000 7f7e727fc700 1 -- 172.21.15.44:0/867267513 wait complete. 2020-11-20T12:19:39.897 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:19:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 158, 2020-11-20T12:19:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:19:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:19:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:19:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:39.909 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:39.909 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:39.909 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:39.909 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.909 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:39.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.910 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:39.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.910 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:39.910 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:39.911 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:39.911 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:39.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.911 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:39.911 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:39.912 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:39.912 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:39.913 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.913 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.913 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.913 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:39.914 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:39.914 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:39.914 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:39.914 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:39.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.915 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:39.915 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:39.915 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:39.915 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:39.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:39.916 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:39.916 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:39.917 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:39.917 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.917 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.918 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.918 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:39.918 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:39.918 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:39.919 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:39.919 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:39.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.919 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:39.920 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:39.920 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:39.920 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:39.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.920 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:39.922 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:39.922 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:39.922 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.922 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.923 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:39.925 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:39.925 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:39.925 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:39.925 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:39.926 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.926 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.926 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:39.926 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:39.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.927 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.927 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.927 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:39.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.928 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.928 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:19:39.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.928 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.929 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.929 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:39.930 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.930 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.930 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:39.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.931 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.931 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.931 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:39.931 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.932 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:39.932 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:19:39.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:39.933 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:39.933 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:39.933 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:39.934 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:39.934 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:39.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:39.934 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:39.935 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:39.936 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:19:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.094+0000 7f3d36d5f700 1 Processor -- start 2020-11-20T12:19:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.094+0000 7f3d36d5f700 1 -- start start 2020-11-20T12:19:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d36d5f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3d30060330 0x7f3d300a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d36d5f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d3013a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d36d5f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300a6260 con 0x7f3d3013ae80 2020-11-20T12:19:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d36d5f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d300a52e0 con 0x7f3d30060330 2020-11-20T12:19:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d36d5f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300a6490 con 0x7f3d300a4c30 2020-11-20T12:19:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d34afb700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3d30060330 0x7f3d300a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d352fc700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f3d3013ae80 0x7f3d30136f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46552/0 (socket says 172.21.15.44:46552) 2020-11-20T12:19:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d352fc700 1 -- 172.21.15.44:0/1084852189 learned_addr learned my addr 172.21.15.44:0/1084852189 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2ffff700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d3013a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2ffff700 1 -- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 msgr2=0x7f3d3013a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:19:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2ffff700 1 -- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 msgr2=0x7f3d3013a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:19:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2ffff700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d3013a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2f7fe700 1 -- 172.21.15.44:0/1084852189 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819514484 0 0) 0x7f3d300a6260 con 0x7f3d3013ae80 2020-11-20T12:19:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2ffff700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d3013a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2f7fe700 1 -- 172.21.15.44:0/1084852189 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d14003240 con 0x7f3d3013ae80 2020-11-20T12:19:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d34afb700 1 -- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 msgr2=0x7f3d3013a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d34afb700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d3013a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d34afb700 1 -- 172.21.15.44:0/1084852189 >> v1:172.21.15.44:6789/0 conn(0x7f3d3013ae80 legacy=0x7f3d30136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d34afb700 1 -- 172.21.15.44:0/1084852189 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300a6730 con 0x7f3d30060330 2020-11-20T12:19:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d34afb700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d30060330 0x7f3d300a4740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3d2400b9a0 tx=0x7f3d24006df0).ready entity=mon.0 client_cookie=943457399fc606af server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.095+0000 7f3d2f7fe700 1 -- 172.21.15.44:0/1084852189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d240097b0 con 0x7f3d30060330 2020-11-20T12:19:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d2f7fe700 1 -- 172.21.15.44:0/1084852189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d24009910 con 0x7f3d30060330 2020-11-20T12:19:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d2f7fe700 1 -- 172.21.15.44:0/1084852189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24009c90 con 0x7f3d30060330 2020-11-20T12:19:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 -- 172.21.15.44:0/1084852189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d30060330 msgr2=0x7f3d300a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d30060330 0x7f3d300a4740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3d2400b9a0 tx=0x7f3d24006df0).stop 2020-11-20T12:19:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 -- 172.21.15.44:0/1084852189 shutdown_connections 2020-11-20T12:19:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d30060330 0x7f3d300a4740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 --2- 172.21.15.44:0/1084852189 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d3013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 -- 172.21.15.44:0/1084852189 >> 172.21.15.44:0/1084852189 conn(0x7f3d30062690 msgr2=0x7f3d300a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 -- 172.21.15.44:0/1084852189 shutdown_connections 2020-11-20T12:19:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.096+0000 7f3d36d5f700 1 -- 172.21.15.44:0/1084852189 wait complete. 2020-11-20T12:19:40.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.097+0000 7f3d36d5f700 1 Processor -- start 2020-11-20T12:19:40.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d36d5f700 1 -- start start 2020-11-20T12:19:40.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d36d5f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d301367e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d36d5f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3013ae80 0x7f3d30130070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d36d5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3013d7f0 0x7f3d30130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d36d5f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d30135020 con 0x7f3d3013d7f0 2020-11-20T12:19:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d36d5f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3013cad0 con 0x7f3d3013ae80 2020-11-20T12:19:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d36d5f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d30137240 con 0x7f3d300a4c30 2020-11-20T12:19:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d352fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3013d7f0 0x7f3d30130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d352fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3013d7f0 0x7f3d30130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42810/0 (socket says 172.21.15.44:42810) 2020-11-20T12:19:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d352fc700 1 -- 172.21.15.44:0/2678510961 learned_addr learned my addr 172.21.15.44:0/2678510961 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d301367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.098+0000 7f3d2ffff700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3013ae80 0x7f3d30130070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 msgr2=0x7f3d301367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:19:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 msgr2=0x7f3d301367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d301367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d301367e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d352fc700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 msgr2=0x7f3d301367e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d352fc700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d300a4c30 0x7f3d301367e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d352fc700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3013ae80 msgr2=0x7f3d30130070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d352fc700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3013ae80 0x7f3d30130070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d352fc700 1 -- 172.21.15.44:0/2678510961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d30134ca0 con 0x7f3d3013d7f0 2020-11-20T12:19:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d352fc700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3013d7f0 0x7f3d30130560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f3d28001420 tx=0x7f3d28007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28005b80 con 0x7f3d3013d7f0 2020-11-20T12:19:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d28005ce0 con 0x7f3d3013d7f0 2020-11-20T12:19:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28007320 con 0x7f3d3013d7f0 2020-11-20T12:19:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3013bd90 con 0x7f3d3013d7f0 2020-11-20T12:19:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.099+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d30130c10 con 0x7f3d3013d7f0 2020-11-20T12:19:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.100+0000 7f3d36d5f700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.100+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3013bf80 con 0x7f3d30066260 2020-11-20T12:19:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.100+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.100+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2020-11-20T12:19:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.100+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.100+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.100+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.101+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3d28010030 con 0x7f3d3013d7f0 2020-11-20T12:19:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.101+0000 7f3d2cff9700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d08056d20 0x7f3d080591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.101+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3d2806ff30 con 0x7f3d3013d7f0 2020-11-20T12:19:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.101+0000 7f3d2ffff700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d08056d20 0x7f3d080591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.101+0000 7f3d2ffff700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d08056d20 0x7f3d080591e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f3d300a3250 tx=0x7f3d1c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.301+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.301+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:19:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.301+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.301+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.301+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:19:40.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.702+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.702+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:19:40.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.702+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:40.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.702+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:40.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:40.702+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:19:41.506 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:41.503+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:41.503+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2020-11-20T12:19:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:41.503+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:19:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:41.503+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:19:41.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:41.503+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:19:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.105+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.105+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d3013c950 con 0x7f3d30066260 2020-11-20T12:19:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.105+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3d2400b9a0 tx=0x7f3d24015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.106+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24027070 con 0x7f3d30066260 2020-11-20T12:19:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.106+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f3d24011c80 con 0x7f3d30066260 2020-11-20T12:19:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.106+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.106+0000 7f3d2cff9700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3d2400b9a0 tx=0x7f3d24015000).stop 2020-11-20T12:19:43.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.106+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 msgr2=0x7f3d30066640 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:43.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.106+0000 7f3d2cff9700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066260 0x7f3d30066640 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.115+0000 7f3d36d5f700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066b30 0x7f3d30025a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.115+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3013ce90 con 0x7f3d30066b30 2020-11-20T12:19:43.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.116+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066b30 0x7f3d30025a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.116+0000 7f3d34afb700 1 -- 172.21.15.44:0/2678510961 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3d300a47a0 con 0x7f3d30066b30 2020-11-20T12:19:43.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.116+0000 7f3d34afb700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066b30 0x7f3d30025a60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3d30135b70 tx=0x7f3d24027000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.116+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401f7d0 con 0x7f3d30066b30 2020-11-20T12:19:43.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f3d24011a70 con 0x7f3d30066b30 2020-11-20T12:19:43.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066b30 msgr2=0x7f3d30025a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d2cff9700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066b30 0x7f3d30025a60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3d30135b70 tx=0x7f3d24027000).stop 2020-11-20T12:19:43.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d2cff9700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066b30 msgr2=0x7f3d30025a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:43.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d2cff9700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d30066b30 0x7f3d30025a60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d08056d20 msgr2=0x7f3d080591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d08056d20 0x7f3d080591e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f3d300a3250 tx=0x7f3d1c008040).stop 2020-11-20T12:19:43.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3013d7f0 msgr2=0x7f3d30130560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 --2- 172.21.15.44:0/2678510961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3013d7f0 0x7f3d30130560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f3d28001420 tx=0x7f3d28007e20).stop 2020-11-20T12:19:43.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d352fc700 1 -- 172.21.15.44:0/2678510961 reap_dead start 2020-11-20T12:19:43.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d352fc700 1 -- 172.21.15.44:0/2678510961 reap_dead start 2020-11-20T12:19:43.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 shutdown_connections 2020-11-20T12:19:43.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 >> 172.21.15.44:0/2678510961 conn(0x7f3d30062690 msgr2=0x7f3d3013a760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 shutdown_connections 2020-11-20T12:19:43.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.117+0000 7f3d36d5f700 1 -- 172.21.15.44:0/2678510961 wait complete. 2020-11-20T12:19:43.125 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:19:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 158, 2020-11-20T12:19:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:19:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:19:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 5, 2020-11-20T12:19:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:43.156 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:43.157 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:19:43.158 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:19:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc0df73700 1 Processor -- start 2020-11-20T12:19:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc0df73700 1 -- start start 2020-11-20T12:19:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc0df73700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc080a4ef0 0x7efc080a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc0df73700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc080a8450 con 0x7efc080a6600 2020-11-20T12:19:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc0df73700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc080a6b80 con 0x7efc080a9880 2020-11-20T12:19:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc0df73700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc080a71b0 con 0x7efc080a4ef0 2020-11-20T12:19:43.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc07fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7efc080a6600 0x7efc080a7d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46574/0 (socket says 172.21.15.44:46574) 2020-11-20T12:19:43.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.296+0000 7efc07fff700 1 -- 172.21.15.44:0/3620852674 learned_addr learned my addr 172.21.15.44:0/3620852674 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:43.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc0d772700 1 --2- 172.21.15.44:0/3620852674 >> v2:172.21.15.79:3300/0 conn(0x7efc080a4ef0 0x7efc080a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc077fe700 1 -- 172.21.15.44:0/3620852674 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445097659 0 0) 0x7efc080a8450 con 0x7efc080a6600 2020-11-20T12:19:43.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc077fe700 1 -- 172.21.15.44:0/3620852674 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbec003240 con 0x7efc080a6600 2020-11-20T12:19:43.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc0d772700 1 -- 172.21.15.44:0/3620852674 >> v1:172.21.15.79:6789/0 conn(0x7efc080a9880 legacy=0x7efc080a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc077fe700 1 -- 172.21.15.44:0/3620852674 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066836265 0 0) 0x7efc080a6b80 con 0x7efc080a9880 2020-11-20T12:19:43.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc0d772700 1 -- 172.21.15.44:0/3620852674 >> v1:172.21.15.44:6789/0 conn(0x7efc080a6600 legacy=0x7efc080a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc0d772700 1 -- 172.21.15.44:0/3620852674 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc080a6e20 con 0x7efc080a4ef0 2020-11-20T12:19:43.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc0d772700 1 --2- 172.21.15.44:0/3620852674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a4ef0 0x7efc080a8af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efc00009c90 tx=0x7efc000091d0).ready entity=mon.1 client_cookie=2017b4f5687558b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc077fe700 1 -- 172.21.15.44:0/3620852674 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0000bb00 con 0x7efc080a4ef0 2020-11-20T12:19:43.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc077fe700 1 -- 172.21.15.44:0/3620852674 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0000bc60 con 0x7efc080a4ef0 2020-11-20T12:19:43.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.297+0000 7efc077fe700 1 -- 172.21.15.44:0/3620852674 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00011400 con 0x7efc080a4ef0 2020-11-20T12:19:43.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.298+0000 7efc0df73700 1 -- 172.21.15.44:0/3620852674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a4ef0 msgr2=0x7efc080a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.298+0000 7efc0df73700 1 --2- 172.21.15.44:0/3620852674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a4ef0 0x7efc080a8af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7efc00009c90 tx=0x7efc000091d0).stop 2020-11-20T12:19:43.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.298+0000 7efc0df73700 1 -- 172.21.15.44:0/3620852674 shutdown_connections 2020-11-20T12:19:43.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.298+0000 7efc0df73700 1 --2- 172.21.15.44:0/3620852674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a4ef0 0x7efc080a8af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.298+0000 7efc0df73700 1 -- 172.21.15.44:0/3620852674 >> 172.21.15.44:0/3620852674 conn(0x7efc08061b60 msgr2=0x7efc08061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.298+0000 7efc0df73700 1 -- 172.21.15.44:0/3620852674 shutdown_connections 2020-11-20T12:19:43.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.298+0000 7efc0df73700 1 -- 172.21.15.44:0/3620852674 wait complete. 2020-11-20T12:19:43.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.299+0000 7efc0df73700 1 Processor -- start 2020-11-20T12:19:43.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.299+0000 7efc0df73700 1 -- start start 2020-11-20T12:19:43.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0df73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc080a4ef0 0x7efc08065d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0df73700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a6600 0x7efc0813abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0df73700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc080a9880 0x7efc081497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0df73700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08139ca0 con 0x7efc080a4ef0 2020-11-20T12:19:43.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0df73700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08139f00 con 0x7efc080a6600 2020-11-20T12:19:43.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0df73700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0813a1a0 con 0x7efc080a9880 2020-11-20T12:19:43.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc080a4ef0 0x7efc08065d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc080a4ef0 0x7efc08065d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42832/0 (socket says 172.21.15.44:42832) 2020-11-20T12:19:43.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 -- 172.21.15.44:0/148778435 learned_addr learned my addr 172.21.15.44:0/148778435 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:43.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc07fff700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a6600 0x7efc0813abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0d772700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc080a9880 0x7efc081497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 -- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc080a9880 msgr2=0x7efc081497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc080a9880 0x7efc081497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 -- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a6600 msgr2=0x7efc0813abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a6600 0x7efc0813abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 -- 172.21.15.44:0/148778435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0813b0a0 con 0x7efc080a4ef0 2020-11-20T12:19:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc07fff700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a6600 0x7efc0813abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.300+0000 7efc0cf71700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc080a4ef0 0x7efc08065d20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7efbfc00c730 tx=0x7efbfc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.317 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.301+0000 7efc04ff9700 1 -- 172.21.15.44:0/148778435 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc007020 con 0x7efc080a4ef0 2020-11-20T12:19:43.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.301+0000 7efc04ff9700 1 -- 172.21.15.44:0/148778435 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc007180 con 0x7efc080a4ef0 2020-11-20T12:19:43.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.301+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0813a640 con 0x7efc080a4ef0 2020-11-20T12:19:43.318 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.301+0000 7efc04ff9700 1 -- 172.21.15.44:0/148778435 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00e560 con 0x7efc080a4ef0 2020-11-20T12:19:43.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.301+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc08136a80 con 0x7efc080a4ef0 2020-11-20T12:19:43.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.302+0000 7efc04ff9700 1 -- 172.21.15.44:0/148778435 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7efbfc010070 con 0x7efc080a4ef0 2020-11-20T12:19:43.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.302+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc08139b00 con 0x7efc080a4ef0 2020-11-20T12:19:43.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.302+0000 7efc04ff9700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efbe0056d20 0x7efbe00591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.302+0000 7efc04ff9700 1 -- 172.21.15.44:0/148778435 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7efbfc06f700 con 0x7efc080a4ef0 2020-11-20T12:19:43.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.303+0000 7efc07fff700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efbe0056d20 0x7efbe00591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.303+0000 7efc07fff700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efbe0056d20 0x7efbe00591e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7efc080a3130 tx=0x7efbf4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.307+0000 7efc04ff9700 1 -- 172.21.15.44:0/148778435 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efc08139b00 con 0x7efc080a4ef0 2020-11-20T12:19:43.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.589+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7efc08139b00 con 0x7efc080a4ef0 2020-11-20T12:19:43.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.590+0000 7efc04ff9700 1 -- 172.21.15.44:0/148778435 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7efc08139b00 con 0x7efc080a4ef0 2020-11-20T12:19:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efbe0056d20 msgr2=0x7efbe00591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efbe0056d20 0x7efbe00591e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7efc080a3130 tx=0x7efbf4009040).stop 2020-11-20T12:19:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc080a4ef0 msgr2=0x7efc08065d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc080a4ef0 0x7efc08065d20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7efbfc00c730 tx=0x7efbfc007c50).stop 2020-11-20T12:19:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 shutdown_connections 2020-11-20T12:19:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7efbe0056d20 0x7efbe00591e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7efc080a9880 0x7efc081497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc080a4ef0 0x7efc08065d20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 --2- 172.21.15.44:0/148778435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc080a6600 0x7efc0813abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.592+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 >> 172.21.15.44:0/148778435 conn(0x7efc08061b60 msgr2=0x7efc081340d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.593+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 shutdown_connections 2020-11-20T12:19:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:43.593+0000 7efc0df73700 1 -- 172.21.15.44:0/148778435 wait complete. 2020-11-20T12:19:43.608 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:19:53.609 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:19:53.610 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:53.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.777+0000 7fe297564700 1 Processor -- start 2020-11-20T12:19:53.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.778+0000 7fe297564700 1 -- start start 2020-11-20T12:19:53.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.778+0000 7fe297564700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe290060330 0x7fe2900a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.778+0000 7fe297564700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe29013a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.778+0000 7fe297564700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900a6260 con 0x7fe29013ae80 2020-11-20T12:19:53.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.778+0000 7fe297564700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900a52e0 con 0x7fe290060330 2020-11-20T12:19:53.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.778+0000 7fe297564700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900a6490 con 0x7fe2900a4c30 2020-11-20T12:19:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe290060330 0x7fe2900a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe290060330 0x7fe2900a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42840/0 (socket says 172.21.15.44:42840) 2020-11-20T12:19:53.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 -- 172.21.15.44:0/1156418162 learned_addr learned my addr 172.21.15.44:0/1156418162 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe294aff700 1 --2- 172.21.15.44:0/1156418162 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe29013a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 -- 172.21.15.44:0/1156418162 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 msgr2=0x7fe29013a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 --2- 172.21.15.44:0/1156418162 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe29013a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 -- 172.21.15.44:0/1156418162 >> v1:172.21.15.79:6789/0 conn(0x7fe29013ae80 legacy=0x7fe290136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 -- 172.21.15.44:0/1156418162 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900a6730 con 0x7fe290060330 2020-11-20T12:19:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe294aff700 1 --2- 172.21.15.44:0/1156418162 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe29013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:53.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe295300700 1 --2- 172.21.15.44:0/1156418162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe290060330 0x7fe2900a4740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe28c006fd0 tx=0x7fe28c006eb0).ready entity=mon.0 client_cookie=c0ac34a8d0bcc854 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe27ffff700 1 -- 172.21.15.44:0/1156418162 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0098d0 con 0x7fe290060330 2020-11-20T12:19:53.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe27ffff700 1 -- 172.21.15.44:0/1156418162 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c009a30 con 0x7fe290060330 2020-11-20T12:19:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.779+0000 7fe27ffff700 1 -- 172.21.15.44:0/1156418162 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c009db0 con 0x7fe290060330 2020-11-20T12:19:53.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 -- 172.21.15.44:0/1156418162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe290060330 msgr2=0x7fe2900a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 --2- 172.21.15.44:0/1156418162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe290060330 0x7fe2900a4740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe28c006fd0 tx=0x7fe28c006eb0).stop 2020-11-20T12:19:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 -- 172.21.15.44:0/1156418162 shutdown_connections 2020-11-20T12:19:53.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 --2- 172.21.15.44:0/1156418162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe290060330 0x7fe2900a4740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 --2- 172.21.15.44:0/1156418162 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe29013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 -- 172.21.15.44:0/1156418162 >> 172.21.15.44:0/1156418162 conn(0x7fe290062690 msgr2=0x7fe2900a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:53.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 -- 172.21.15.44:0/1156418162 shutdown_connections 2020-11-20T12:19:53.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.780+0000 7fe297564700 1 -- 172.21.15.44:0/1156418162 wait complete. 2020-11-20T12:19:53.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.781+0000 7fe297564700 1 Processor -- start 2020-11-20T12:19:53.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.781+0000 7fe297564700 1 -- start start 2020-11-20T12:19:53.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.782+0000 7fe297564700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe290060330 0x7fe290131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.782+0000 7fe297564700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe2900ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.782+0000 7fe297564700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe29013ae80 0x7fe29014d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.782+0000 7fe297564700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290131d10 con 0x7fe29013ae80 2020-11-20T12:19:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.782+0000 7fe297564700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900ab090 con 0x7fe290060330 2020-11-20T12:19:53.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.782+0000 7fe297564700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900ab330 con 0x7fe2900a4c30 2020-11-20T12:19:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe29013ae80 0x7fe29014d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295300700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe290060330 0x7fe290131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe29013ae80 0x7fe29014d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42850/0 (socket says 172.21.15.44:42850) 2020-11-20T12:19:53.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 -- 172.21.15.44:0/2816905025 learned_addr learned my addr 172.21.15.44:0/2816905025 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:53.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe294aff700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe2900ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 -- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 msgr2=0x7fe2900ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe2900ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 -- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe290060330 msgr2=0x7fe290131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe290060330 0x7fe290131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 -- 172.21.15.44:0/2816905025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900ab5d0 con 0x7fe29013ae80 2020-11-20T12:19:53.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295300700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe290060330 0x7fe290131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:53.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe295b01700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe29013ae80 0x7fe29014d6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe28800ca90 tx=0x7fe288001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe27d7fa700 1 -- 172.21.15.44:0/2816905025 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800ce60 con 0x7fe29013ae80 2020-11-20T12:19:53.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe27d7fa700 1 -- 172.21.15.44:0/2816905025 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800a730 con 0x7fe29013ae80 2020-11-20T12:19:53.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2900ad5f0 con 0x7fe29013ae80 2020-11-20T12:19:53.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe27d7fa700 1 -- 172.21.15.44:0/2816905025 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800aad0 con 0x7fe29013ae80 2020-11-20T12:19:53.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.783+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe290130950 con 0x7fe29013ae80 2020-11-20T12:19:53.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.785+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2900608f0 con 0x7fe29013ae80 2020-11-20T12:19:53.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.785+0000 7fe27d7fa700 1 -- 172.21.15.44:0/2816905025 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe288019030 con 0x7fe29013ae80 2020-11-20T12:19:53.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.785+0000 7fe27d7fa700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe268056d20 0x7fe2680591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.785+0000 7fe27d7fa700 1 -- 172.21.15.44:0/2816905025 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe28806f7b0 con 0x7fe29013ae80 2020-11-20T12:19:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.785+0000 7fe295300700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe268056d20 0x7fe2680591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.786+0000 7fe295300700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe268056d20 0x7fe2680591e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fe28c00b1b0 tx=0x7fe28c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:53.789+0000 7fe27d7fa700 1 -- 172.21.15.44:0/2816905025 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe288043fd0 con 0x7fe29013ae80 2020-11-20T12:19:54.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.113+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe290061650 con 0x7fe29013ae80 2020-11-20T12:19:54.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.113+0000 7fe27d7fa700 1 -- 172.21.15.44:0/2816905025 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fe288044b30 con 0x7fe29013ae80 2020-11-20T12:19:54.117 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:54.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe268056d20 msgr2=0x7fe2680591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe268056d20 0x7fe2680591e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fe28c00b1b0 tx=0x7fe28c015000).stop 2020-11-20T12:19:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe29013ae80 msgr2=0x7fe29014d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe29013ae80 0x7fe29014d6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe28800ca90 tx=0x7fe288001360).stop 2020-11-20T12:19:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 shutdown_connections 2020-11-20T12:19:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe268056d20 0x7fe2680591e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe29013ae80 0x7fe29014d6f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe290060330 0x7fe290131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 --2- 172.21.15.44:0/2816905025 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2900a4c30 0x7fe2900ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 >> 172.21.15.44:0/2816905025 conn(0x7fe290062690 msgr2=0x7fe2900a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 shutdown_connections 2020-11-20T12:19:54.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.115+0000 7fe297564700 1 -- 172.21.15.44:0/2816905025 wait complete. 2020-11-20T12:19:54.130 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:19:54.131 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:19:54.131 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:19:54.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.276+0000 7fe108704700 1 Processor -- start 2020-11-20T12:19:54.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.276+0000 7fe108704700 1 -- start start 2020-11-20T12:19:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe108704700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe1000a8d60 0x7fe1000adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe108704700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe100137df0 0x7fe100138250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe108704700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000678c0 con 0x7fe1000a9140 2020-11-20T12:19:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe108704700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10013e440 con 0x7fe100137df0 2020-11-20T12:19:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe108704700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10013e6e0 con 0x7fe1000a8d60 2020-11-20T12:19:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe106ca1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe100137df0 0x7fe100138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe105c9f700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe1000a9140 0x7fe10013ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46604/0 (socket says 172.21.15.44:46604) 2020-11-20T12:19:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe105c9f700 1 -- 172.21.15.44:0/745239115 learned_addr learned my addr 172.21.15.44:0/745239115 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.277+0000 7fe1064a0700 1 --2- 172.21.15.44:0/745239115 >> v2:172.21.15.79:3300/0 conn(0x7fe1000a8d60 0x7fe1000adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe10549e700 1 -- 172.21.15.44:0/745239115 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788989550 0 0) 0x7fe1000678c0 con 0x7fe1000a9140 2020-11-20T12:19:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe10549e700 1 -- 172.21.15.44:0/745239115 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e8003240 con 0x7fe1000a9140 2020-11-20T12:19:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe1064a0700 1 -- 172.21.15.44:0/745239115 >> v2:172.21.15.44:3300/0 conn(0x7fe100137df0 msgr2=0x7fe100138250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe1064a0700 1 --2- 172.21.15.44:0/745239115 >> v2:172.21.15.44:3300/0 conn(0x7fe100137df0 0x7fe100138250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe1064a0700 1 -- 172.21.15.44:0/745239115 >> v1:172.21.15.44:6789/0 conn(0x7fe1000a9140 legacy=0x7fe10013ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe1064a0700 1 -- 172.21.15.44:0/745239115 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10013e980 con 0x7fe1000a8d60 2020-11-20T12:19:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe1064a0700 1 --2- 172.21.15.44:0/745239115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 0x7fe1000adef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe1000adec0 tx=0x7fe0f0006df0).ready entity=mon.1 client_cookie=e5e15ff247424fe3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe10549e700 1 -- 172.21.15.44:0/745239115 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0009640 con 0x7fe1000a8d60 2020-11-20T12:19:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe10549e700 1 -- 172.21.15.44:0/745239115 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f00097a0 con 0x7fe1000a8d60 2020-11-20T12:19:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.278+0000 7fe10549e700 1 -- 172.21.15.44:0/745239115 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0009b20 con 0x7fe1000a8d60 2020-11-20T12:19:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 -- 172.21.15.44:0/745239115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 msgr2=0x7fe1000adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 --2- 172.21.15.44:0/745239115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 0x7fe1000adef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe1000adec0 tx=0x7fe0f0006df0).stop 2020-11-20T12:19:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 -- 172.21.15.44:0/745239115 shutdown_connections 2020-11-20T12:19:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 --2- 172.21.15.44:0/745239115 >> v2:172.21.15.44:3300/0 conn(0x7fe100137df0 0x7fe100138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 --2- 172.21.15.44:0/745239115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 0x7fe1000adef0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 -- 172.21.15.44:0/745239115 >> 172.21.15.44:0/745239115 conn(0x7fe1000658a0 msgr2=0x7fe100062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 -- 172.21.15.44:0/745239115 shutdown_connections 2020-11-20T12:19:54.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.279+0000 7fe108704700 1 -- 172.21.15.44:0/745239115 wait complete. 2020-11-20T12:19:54.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 Processor -- start 2020-11-20T12:19:54.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 -- start start 2020-11-20T12:19:54.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 0x7fe1000ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000a9140 0x7fe1000a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe100137df0 0x7fe1000a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.292 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10014d3d0 con 0x7fe1000a9140 2020-11-20T12:19:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000a25c0 con 0x7fe1000a8d60 2020-11-20T12:19:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.280+0000 7fe108704700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000a2280 con 0x7fe100137df0 2020-11-20T12:19:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000a9140 0x7fe1000a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000a9140 0x7fe1000a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42864/0 (socket says 172.21.15.44:42864) 2020-11-20T12:19:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe1064a0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 0x7fe1000ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 -- 172.21.15.44:0/1769289893 learned_addr learned my addr 172.21.15.44:0/1769289893 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe106ca1700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe100137df0 0x7fe1000a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe100137df0 msgr2=0x7fe1000a2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe100137df0 0x7fe1000a2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 msgr2=0x7fe1000ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 0x7fe1000ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 -- 172.21.15.44:0/1769289893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000a4b60 con 0x7fe1000a9140 2020-11-20T12:19:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe1064a0700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1000a8d60 0x7fe1000ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe105c9f700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000a9140 0x7fe1000a1d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fe0fc00c6a0 tx=0x7fe0fc007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0072e0 con 0x7fe1000a9140 2020-11-20T12:19:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc007440 con 0x7fe1000a9140 2020-11-20T12:19:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc005820 con 0x7fe1000a9140 2020-11-20T12:19:54.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1000a4db0 con 0x7fe1000a9140 2020-11-20T12:19:54.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.281+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1000a3650 con 0x7fe1000a9140 2020-11-20T12:19:54.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.282+0000 7fe108704700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10014d8c0 0x7fe10014dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.282+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000a38d0 con 0x7fe10014d8c0 2020-11-20T12:19:54.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.282+0000 7fe1064a0700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10014d8c0 0x7fe10014dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe0fc010070 con 0x7fe1000a9140 2020-11-20T12:19:54.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe1064a0700 1 -- 172.21.15.44:0/1769289893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1000a23f0 con 0x7fe10014d8c0 2020-11-20T12:19:54.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe1064a0700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10014d8c0 0x7fe10014dcd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fe1000adec0 tx=0x7fe0f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.302 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe0dc056d20 0x7fe0dc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe0fc06f620 con 0x7fe1000a9140 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f001e640 con 0x7fe10014d8c0 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fe0f0022070 con 0x7fe10014d8c0 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10014d8c0 msgr2=0x7fe10014dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10014d8c0 0x7fe10014dcd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fe1000adec0 tx=0x7fe0f0015000).stop 2020-11-20T12:19:54.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10014d8c0 msgr2=0x7fe10014dcd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:54.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.283+0000 7fe0ee7fc700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10014d8c0 0x7fe10014dcd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.284+0000 7fe106ca1700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe0dc056d20 0x7fe0dc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.304 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.284+0000 7fe106ca1700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe0dc056d20 0x7fe0dc0591e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fe1000a4a80 tx=0x7fe0f80091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.294+0000 7fe108704700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe100149e80 0x7fe10014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.294+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100061330 con 0x7fe100149e80 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.294+0000 7fe1064a0700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe100149e80 0x7fe10014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe1064a0700 1 -- 172.21.15.44:0/1769289893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe1000a2760 con 0x7fe100149e80 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe1064a0700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe100149e80 0x7fe10014a260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fe0f0006b80 tx=0x7fe0f0020070).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0027b80 con 0x7fe100149e80 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4779 (secure 0 0 0) 0x7fe0f001e430 con 0x7fe100149e80 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe100149e80 msgr2=0x7fe10014a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe0ee7fc700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe100149e80 0x7fe10014a260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fe0f0006b80 tx=0x7fe0f0020070).stop 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe0ee7fc700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe100149e80 msgr2=0x7fe10014a260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.295+0000 7fe0ee7fc700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe100149e80 0x7fe10014a260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe0dc056d20 msgr2=0x7fe0dc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe0dc056d20 0x7fe0dc0591e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7fe1000a4a80 tx=0x7fe0f80091d0).stop 2020-11-20T12:19:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000a9140 msgr2=0x7fe1000a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 --2- 172.21.15.44:0/1769289893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1000a9140 0x7fe1000a1d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fe0fc00c6a0 tx=0x7fe0fc007ac0).stop 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe106ca1700 1 -- 172.21.15.44:0/1769289893 reap_dead start 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe106ca1700 1 -- 172.21.15.44:0/1769289893 reap_dead start 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 shutdown_connections 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 >> 172.21.15.44:0/1769289893 conn(0x7fe1000658a0 msgr2=0x7fe100060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 shutdown_connections 2020-11-20T12:19:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.296+0000 7fe108704700 1 -- 172.21.15.44:0/1769289893 wait complete. 2020-11-20T12:19:54.311 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:54.311 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:54.311 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:54.312 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:19:54.312 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 158, 2020-11-20T12:19:54.312 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:54.312 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:54.312 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:19:54.313 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:19:54.313 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.313 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 16, 2020-11-20T12:19:54.313 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:54.314 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:54.314 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:54.314 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.314 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.314 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.317 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.317 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.317 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.317 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:54.321 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:54.321 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:54.321 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:54.321 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.322 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.322 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.322 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.322 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.323 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.323 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.323 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.323 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:54.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.324 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:54.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.324 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:54.324 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:54.324 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.325 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.325 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:54.325 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:54.328 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.328 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.328 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.329 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:54.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:54.329 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.329 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.330 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:54.330 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.331 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:54.331 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.331 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.332 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:54.332 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:54.332 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.333 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.333 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.334 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:54.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:54.334 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.334 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.335 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:54.335 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.336 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:54.336 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.336 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.337 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:54.337 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:54.337 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.337 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.338 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.338 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.338 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:19:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:19:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:19:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:19:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:54.346 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:54.347 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:19:54.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.525+0000 7fad56a4b700 1 Processor -- start 2020-11-20T12:19:54.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.525+0000 7fad56a4b700 1 -- start start 2020-11-20T12:19:54.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad56a4b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad50064570 0x7fad50138df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad56a4b700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fad500a91e0 0x7fad501392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad56a4b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad50136420 con 0x7fad50064190 2020-11-20T12:19:54.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad56a4b700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad50138a00 con 0x7fad500a91e0 2020-11-20T12:19:54.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad56a4b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad501397d0 con 0x7fad50064570 2020-11-20T12:19:54.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad50064570 0x7fad50138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad4ffff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fad50064190 0x7fad500a8ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49496/0 (socket says 172.21.15.44:49496) 2020-11-20T12:19:54.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.526+0000 7fad4ffff700 1 -- 172.21.15.44:0/453703487 learned_addr learned my addr 172.21.15.44:0/453703487 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad54fe8700 1 --2- 172.21.15.44:0/453703487 >> v2:172.21.15.44:3300/0 conn(0x7fad500a91e0 0x7fad501392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4f7fe700 1 -- 172.21.15.44:0/453703487 >> v2:172.21.15.44:3300/0 conn(0x7fad500a91e0 msgr2=0x7fad501392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4f7fe700 1 --2- 172.21.15.44:0/453703487 >> v2:172.21.15.44:3300/0 conn(0x7fad500a91e0 0x7fad501392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4f7fe700 1 -- 172.21.15.44:0/453703487 >> v1:172.21.15.79:6789/0 conn(0x7fad50064190 legacy=0x7fad500a8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4effd700 1 -- 172.21.15.44:0/453703487 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369248174 0 0) 0x7fad50136420 con 0x7fad50064190 2020-11-20T12:19:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4f7fe700 1 -- 172.21.15.44:0/453703487 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad500ab660 con 0x7fad50064570 2020-11-20T12:19:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad54fe8700 1 --2- 172.21.15.44:0/453703487 >> v2:172.21.15.44:3300/0 conn(0x7fad500a91e0 0x7fad501392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4f7fe700 1 --2- 172.21.15.44:0/453703487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad50064570 0x7fad50138df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fad4400b9a0 tx=0x7fad44006df0).ready entity=mon.2 client_cookie=a4594ba433877f2d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4effd700 1 -- 172.21.15.44:0/453703487 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440097b0 con 0x7fad50064570 2020-11-20T12:19:54.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4effd700 1 -- 172.21.15.44:0/453703487 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44009910 con 0x7fad50064570 2020-11-20T12:19:54.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.527+0000 7fad4effd700 1 -- 172.21.15.44:0/453703487 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44009c90 con 0x7fad50064570 2020-11-20T12:19:54.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 -- 172.21.15.44:0/453703487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad50064570 msgr2=0x7fad50138df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 --2- 172.21.15.44:0/453703487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad50064570 0x7fad50138df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fad4400b9a0 tx=0x7fad44006df0).stop 2020-11-20T12:19:54.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 -- 172.21.15.44:0/453703487 shutdown_connections 2020-11-20T12:19:54.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 --2- 172.21.15.44:0/453703487 >> v2:172.21.15.44:3300/0 conn(0x7fad500a91e0 0x7fad501392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 --2- 172.21.15.44:0/453703487 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad50064570 0x7fad50138df0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 -- 172.21.15.44:0/453703487 >> 172.21.15.44:0/453703487 conn(0x7fad500af030 msgr2=0x7fad500af440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 -- 172.21.15.44:0/453703487 shutdown_connections 2020-11-20T12:19:54.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.528+0000 7fad56a4b700 1 -- 172.21.15.44:0/453703487 wait complete. 2020-11-20T12:19:54.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.529+0000 7fad56a4b700 1 Processor -- start 2020-11-20T12:19:54.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.529+0000 7fad56a4b700 1 -- start start 2020-11-20T12:19:54.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad56a4b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad50064190 0x7fad5013e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad56a4b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad50064570 0x7fad50149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad56a4b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad500a91e0 0x7fad50135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad56a4b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5014a500 con 0x7fad50064570 2020-11-20T12:19:54.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad56a4b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5014a7a0 con 0x7fad50064190 2020-11-20T12:19:54.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad56a4b700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5014aa40 con 0x7fad500a91e0 2020-11-20T12:19:54.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad50064570 0x7fad50149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad50064570 0x7fad50149c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42880/0 (socket says 172.21.15.44:42880) 2020-11-20T12:19:54.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad4f7fe700 1 -- 172.21.15.44:0/4122292108 learned_addr learned my addr 172.21.15.44:0/4122292108 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:54.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad54fe8700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad500a91e0 0x7fad50135390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.530+0000 7fad4ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad50064190 0x7fad5013e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad4f7fe700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad500a91e0 msgr2=0x7fad50135390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad4f7fe700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fad500a91e0 0x7fad50135390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad4f7fe700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad50064190 msgr2=0x7fad5013e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad4f7fe700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad50064190 0x7fad5013e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad4f7fe700 1 -- 172.21.15.44:0/4122292108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5014a150 con 0x7fad50064570 2020-11-20T12:19:54.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad4f7fe700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad50064570 0x7fad50149c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fad44010000 tx=0x7fad44015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44011dd0 con 0x7fad50064570 2020-11-20T12:19:54.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5013ea60 con 0x7fad50064570 2020-11-20T12:19:54.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4401e460 con 0x7fad50064570 2020-11-20T12:19:54.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4401e7e0 con 0x7fad50064570 2020-11-20T12:19:54.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.531+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad50135a40 con 0x7fad50064570 2020-11-20T12:19:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.532+0000 7fad56a4b700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5005c2a0 0x7fad500a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.532+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5013e910 con 0x7fad5005c2a0 2020-11-20T12:19:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.532+0000 7fad4ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5005c2a0 0x7fad500a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.532+0000 7fad4ffff700 1 -- 172.21.15.44:0/4122292108 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad500446d0 con 0x7fad5005c2a0 2020-11-20T12:19:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fad44025030 con 0x7fad50064570 2020-11-20T12:19:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad4ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5005c2a0 0x7fad500a4fc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad40001520 tx=0x7fad40007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fad28056d20 0x7fad280591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fad44071920 con 0x7fad50064570 2020-11-20T12:19:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad54fe8700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fad28056d20 0x7fad280591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40011650 con 0x7fad5005c2a0 2020-11-20T12:19:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fad400117b0 con 0x7fad5005c2a0 2020-11-20T12:19:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5005c2a0 msgr2=0x7fad500a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5005c2a0 0x7fad500a4fc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad40001520 tx=0x7fad40007a00).stop 2020-11-20T12:19:54.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5005c2a0 msgr2=0x7fad500a4fc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:54.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad2ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad5005c2a0 0x7fad500a4fc0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0x7fad40001520 tx=0x7fad40007a00).stop 2020-11-20T12:19:54.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.533+0000 7fad54fe8700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fad28056d20 0x7fad280591e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fad500a95e0 tx=0x7fad48008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.542+0000 7fad56a4b700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad500a54b0 0x7fad5014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.542+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5014ac00 con 0x7fad500a54b0 2020-11-20T12:19:54.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad4ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad500a54b0 0x7fad5014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad4ffff700 1 -- 172.21.15.44:0/4122292108 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fad50135c60 con 0x7fad500a54b0 2020-11-20T12:19:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad4ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad500a54b0 0x7fad5014d230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fad40007a60 tx=0x7fad40016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40013d20 con 0x7fad500a54b0 2020-11-20T12:19:54.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7fad40011440 con 0x7fad500a54b0 2020-11-20T12:19:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad500a54b0 msgr2=0x7fad5014d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad2ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad500a54b0 0x7fad5014d230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fad40007a60 tx=0x7fad40016000).stop 2020-11-20T12:19:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad2ffff700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad500a54b0 msgr2=0x7fad5014d230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:54.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.543+0000 7fad2ffff700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad500a54b0 0x7fad5014d230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fad28056d20 msgr2=0x7fad280591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fad28056d20 0x7fad280591e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fad500a95e0 tx=0x7fad48008040).stop 2020-11-20T12:19:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad50064570 msgr2=0x7fad50149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 --2- 172.21.15.44:0/4122292108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad50064570 0x7fad50149c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fad44010000 tx=0x7fad44015000).stop 2020-11-20T12:19:54.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad54fe8700 1 -- 172.21.15.44:0/4122292108 reap_dead start 2020-11-20T12:19:54.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad54fe8700 1 -- 172.21.15.44:0/4122292108 reap_dead start 2020-11-20T12:19:54.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 shutdown_connections 2020-11-20T12:19:54.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 >> 172.21.15.44:0/4122292108 conn(0x7fad500af030 msgr2=0x7fad500acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 shutdown_connections 2020-11-20T12:19:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.544+0000 7fad56a4b700 1 -- 172.21.15.44:0/4122292108 wait complete. 2020-11-20T12:19:54.555 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:19:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 158, 2020-11-20T12:19:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:19:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:19:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 17, 2020-11-20T12:19:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:19:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:19:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:19:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:19:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:54.590 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:54.591 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:19:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.763+0000 7ff59dcc4700 1 Processor -- start 2020-11-20T12:19:54.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.763+0000 7ff59dcc4700 1 -- start start 2020-11-20T12:19:54.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff59dcc4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff5980adeb0 0x7ff598135290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff59dcc4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff598135780 0x7ff598135c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff59dcc4700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff598136100 con 0x7ff5980a9510 2020-11-20T12:19:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff59dcc4700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5980a5760 con 0x7ff5980adeb0 2020-11-20T12:19:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff59dcc4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980a5950 con 0x7ff598135780 2020-11-20T12:19:54.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff596ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff5980adeb0 0x7ff598135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff596ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff5980adeb0 0x7ff598135290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42892/0 (socket says 172.21.15.44:42892) 2020-11-20T12:19:54.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff596ffd700 1 -- 172.21.15.44:0/1506016075 learned_addr learned my addr 172.21.15.44:0/1506016075 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.764+0000 7ff597fff700 1 --2- 172.21.15.44:0/1506016075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff598135780 0x7ff598135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff596ffd700 1 -- 172.21.15.44:0/1506016075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff598135780 msgr2=0x7ff598135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff596ffd700 1 --2- 172.21.15.44:0/1506016075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff598135780 0x7ff598135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff596ffd700 1 -- 172.21.15.44:0/1506016075 >> v1:172.21.15.79:6789/0 conn(0x7ff5980a9510 legacy=0x7ff598060690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff596ffd700 1 -- 172.21.15.44:0/1506016075 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980a5bf0 con 0x7ff5980adeb0 2020-11-20T12:19:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff597fff700 1 --2- 172.21.15.44:0/1506016075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff598135780 0x7ff598135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff596ffd700 1 --2- 172.21.15.44:0/1506016075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5980adeb0 0x7ff598135290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7ff58c006fd0 tx=0x7ff58c006eb0).ready entity=mon.0 client_cookie=2669a29b28befc81 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff5967fc700 1 -- 172.21.15.44:0/1506016075 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c009960 con 0x7ff5980adeb0 2020-11-20T12:19:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff5967fc700 1 -- 172.21.15.44:0/1506016075 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c009ac0 con 0x7ff5980adeb0 2020-11-20T12:19:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff5967fc700 1 -- 172.21.15.44:0/1506016075 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c009e40 con 0x7ff5980adeb0 2020-11-20T12:19:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1506016075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5980adeb0 msgr2=0x7ff598135290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.765+0000 7ff59dcc4700 1 --2- 172.21.15.44:0/1506016075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5980adeb0 0x7ff598135290 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7ff58c006fd0 tx=0x7ff58c006eb0).stop 2020-11-20T12:19:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.766+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1506016075 shutdown_connections 2020-11-20T12:19:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.766+0000 7ff59dcc4700 1 --2- 172.21.15.44:0/1506016075 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff598135780 0x7ff598135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.766+0000 7ff59dcc4700 1 --2- 172.21.15.44:0/1506016075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5980adeb0 0x7ff598135290 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.766+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1506016075 >> 172.21.15.44:0/1506016075 conn(0x7ff598062bc0 msgr2=0x7ff598062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.766+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1506016075 shutdown_connections 2020-11-20T12:19:54.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.766+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1506016075 wait complete. 2020-11-20T12:19:54.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.767+0000 7ff59dcc4700 1 Processor -- start 2020-11-20T12:19:54.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.767+0000 7ff59dcc4700 1 -- start start 2020-11-20T12:19:54.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff59dcc4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5980a9510 0x7ff598134250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff59dcc4700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980adeb0 0x7ff5980a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff59dcc4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff598135780 0x7ff5980a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff59dcc4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598067890 con 0x7ff598135780 2020-11-20T12:19:54.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff59dcc4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598067af0 con 0x7ff5980a9510 2020-11-20T12:19:54.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff59dcc4700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff598067d90 con 0x7ff5980adeb0 2020-11-20T12:19:54.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff598135780 0x7ff5980a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff598135780 0x7ff5980a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42900/0 (socket says 172.21.15.44:42900) 2020-11-20T12:19:54.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 -- 172.21.15.44:0/1773110696 learned_addr learned my addr 172.21.15.44:0/1773110696 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:54.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff596ffd700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980adeb0 0x7ff5980a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff5977fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5980a9510 0x7ff598134250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980adeb0 msgr2=0x7ff5980a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980adeb0 0x7ff5980a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5980a9510 msgr2=0x7ff598134250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5980a9510 0x7ff598134250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 -- 172.21.15.44:0/1773110696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff598134e10 con 0x7ff598135780 2020-11-20T12:19:54.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.768+0000 7ff597fff700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff598135780 0x7ff5980a1fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff59000db60 tx=0x7ff590008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.769+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59000b900 con 0x7ff598135780 2020-11-20T12:19:54.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.769+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59000ba60 con 0x7ff598135780 2020-11-20T12:19:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.769+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5980a61a0 con 0x7ff598135780 2020-11-20T12:19:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.769+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59000be00 con 0x7ff598135780 2020-11-20T12:19:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.769+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5980a6660 con 0x7ff598135780 2020-11-20T12:19:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff590012630 con 0x7ff598135780 2020-11-20T12:19:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff59dcc4700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980a6880 0x7ff5981322e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff598134740 con 0x7ff5980a6880 2020-11-20T12:19:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff5977fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980a6880 0x7ff5981322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff5777fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff570056d20 0x7ff5700591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff590070060 con 0x7ff598135780 2020-11-20T12:19:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff596ffd700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff570056d20 0x7ff5700591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.770+0000 7ff5977fe700 1 -- 172.21.15.44:0/1773110696 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff598067f50 con 0x7ff5980a6880 2020-11-20T12:19:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff5977fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980a6880 0x7ff5981322e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff58800f820 tx=0x7ff588007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588011650 con 0x7ff5980a6880 2020-11-20T12:19:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff596ffd700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff570056d20 0x7ff5700591e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7ff5980a4980 tx=0x7ff58c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7ff5880117b0 con 0x7ff5980a6880 2020-11-20T12:19:54.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980a6880 msgr2=0x7ff5981322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff5777fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980a6880 0x7ff5981322e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff58800f820 tx=0x7ff588007b90).stop 2020-11-20T12:19:54.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980a6880 msgr2=0x7ff5981322e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:54.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.771+0000 7ff5777fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5980a6880 0x7ff5981322e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.780+0000 7ff59dcc4700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5981327d0 0x7ff598066090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.780+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980a50e0 con 0x7ff5981327d0 2020-11-20T12:19:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.780+0000 7ff5977fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5981327d0 0x7ff598066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5977fe700 1 -- 172.21.15.44:0/1773110696 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff5980ae370 con 0x7ff5981327d0 2020-11-20T12:19:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5977fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5981327d0 0x7ff598066090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff58800f820 tx=0x7ff588018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588014e30 con 0x7ff5981327d0 2020-11-20T12:19:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7ff588011440 con 0x7ff5981327d0 2020-11-20T12:19:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5981327d0 msgr2=0x7ff598066090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5777fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5981327d0 0x7ff598066090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7ff58800f820 tx=0x7ff588018000).stop 2020-11-20T12:19:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5777fe700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5981327d0 msgr2=0x7ff598066090 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:19:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.781+0000 7ff5777fe700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff5981327d0 0x7ff598066090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff570056d20 msgr2=0x7ff5700591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff570056d20 0x7ff5700591e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7ff5980a4980 tx=0x7ff58c015000).stop 2020-11-20T12:19:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff598135780 msgr2=0x7ff5980a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 --2- 172.21.15.44:0/1773110696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff598135780 0x7ff5980a1fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff59000db60 tx=0x7ff590008040).stop 2020-11-20T12:19:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff597fff700 1 -- 172.21.15.44:0/1773110696 reap_dead start 2020-11-20T12:19:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff597fff700 1 -- 172.21.15.44:0/1773110696 reap_dead start 2020-11-20T12:19:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 shutdown_connections 2020-11-20T12:19:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 >> 172.21.15.44:0/1773110696 conn(0x7ff598062bc0 msgr2=0x7ff598138890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 shutdown_connections 2020-11-20T12:19:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:54.782+0000 7ff59dcc4700 1 -- 172.21.15.44:0/1773110696 wait complete. 2020-11-20T12:19:54.798 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:19:54.798 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:54.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:54.799 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:19:54.799 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 158, 2020-11-20T12:19:54.799 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:19:54.799 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:19:54.799 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:19:54.800 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:19:54.800 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.800 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 17, 2020-11-20T12:19:54.800 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:54.800 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:19:54.801 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:19:54.801 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.801 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.801 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.802 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.802 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.802 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.802 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.803 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.803 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:19:54.803 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:19:54.803 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.803 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.804 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.804 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.804 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.804 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.804 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.805 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.805 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.805 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:19:54.805 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:19:54.806 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:19:54.806 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:19:54.806 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:19:54.806 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:19:54.806 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:54.807 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:19:54.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:19:54.807 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:19:54.807 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:19:54.807 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:19:54.808 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:19:54.808 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:19:54.808 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:19:54.808 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:19:54.808 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:19:54.809 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:19:54.809 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:19:54.809 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:19:54.809 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:19:54.809 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:19:54.810 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:19:54.810 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.810 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:19:54.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.810 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:19:54.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:19:54.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:19:54.811 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.811 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.812 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.812 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:19:54.812 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.813 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:19:54.813 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.814 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.814 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.814 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:19:54.814 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:19:54.815 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.815 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.815 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.816 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:19:54.816 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:19:54.816 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.816 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.817 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.817 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:19:54.817 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.818 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.818 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:19:54.818 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.818 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.818 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.819 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:19:54.819 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:19:54.819 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.819 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.820 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.820 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:19:54.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:19:54.821 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:19:54.821 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:19:54.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:19:54.821 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:19:54.822 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.822 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:19:54.822 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:19:54.823 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:19:54.823 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.823 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.823 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:19:54.824 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:19:54.824 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:19:54.824 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:19:54.824 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:19:54.824 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.825 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.825 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:19:54.825 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:19:54.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.826 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.826 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.826 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:54.826 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.827 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:19:54.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:19:54.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:19:54.827 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:19:54.827 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:19:54.827 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:19:54.828 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:19:54.828 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:19:54.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:19:54.828 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:19:54.829 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:19:54.830 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2020-11-20T12:19:54.830 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:19:54.831 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2020-11-20T12:19:54.831 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:19:54.831 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-11-20T12:19:54.831 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:54.843 INFO:tasks.ceph.mon.a:Stopped 2020-11-20T12:19:54.843 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:19:54.843 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:55.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.032+0000 7fbffa0db700 1 Processor -- start 2020-11-20T12:19:55.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.032+0000 7fbffa0db700 1 -- start start 2020-11-20T12:19:55.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.032+0000 7fbffa0db700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff40b1bb0 0x7fbff40a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:55.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbffa0db700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbff40a7d90 0x7fbff4145460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:55.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbffa0db700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40aa810 con 0x7fbff40b17d0 2020-11-20T12:19:55.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbffa0db700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff413e120 con 0x7fbff40a7d90 2020-11-20T12:19:55.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbffa0db700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff413e310 con 0x7fbff40b1bb0 2020-11-20T12:19:55.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff40b1bb0 0x7fbff40a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:55.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff98da700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbff40a7d90 0x7fbff4145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:55.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff40b1bb0 0x7fbff40a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54378/0 (socket says 172.21.15.44:54378) 2020-11-20T12:19:55.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 -- 172.21.15.44:0/3509972458 learned_addr learned my addr 172.21.15.44:0/3509972458 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:55.039 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff98da700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbff40a7d90 0x7fbff4145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:34992/0 (socket says 172.21.15.44:34992) 2020-11-20T12:19:55.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 -- 172.21.15.44:0/3509972458 >> v2:172.21.15.79:3300/0 conn(0x7fbff40a7d90 msgr2=0x7fbff4145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:55.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 --2- 172.21.15.44:0/3509972458 >> v2:172.21.15.79:3300/0 conn(0x7fbff40a7d90 0x7fbff4145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 -- 172.21.15.44:0/3509972458 >> v1:172.21.15.79:6789/0 conn(0x7fbff40b17d0 legacy=0x7fbff413ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:55.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 -- 172.21.15.44:0/3509972458 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff413e5b0 con 0x7fbff40b1bb0 2020-11-20T12:19:55.040 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff98da700 1 --2- 172.21.15.44:0/3509972458 >> v2:172.21.15.79:3300/0 conn(0x7fbff40a7d90 0x7fbff4145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:55.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.033+0000 7fbff88d8700 1 --2- 172.21.15.44:0/3509972458 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff40b1bb0 0x7fbff40a78a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fbfe4006fd0 tx=0x7fbfe4006eb0).ready entity=mon.2 client_cookie=f245ecc7d065dcc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:55.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbfe3fff700 1 -- 172.21.15.44:0/3509972458 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4009810 con 0x7fbff40b1bb0 2020-11-20T12:19:55.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbfe3fff700 1 -- 172.21.15.44:0/3509972458 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4009970 con 0x7fbff40b1bb0 2020-11-20T12:19:55.041 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbfe3fff700 1 -- 172.21.15.44:0/3509972458 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4009cf0 con 0x7fbff40b1bb0 2020-11-20T12:19:55.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 -- 172.21.15.44:0/3509972458 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff40b1bb0 msgr2=0x7fbff40a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:55.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 --2- 172.21.15.44:0/3509972458 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff40b1bb0 0x7fbff40a78a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fbfe4006fd0 tx=0x7fbfe4006eb0).stop 2020-11-20T12:19:55.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 -- 172.21.15.44:0/3509972458 shutdown_connections 2020-11-20T12:19:55.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 --2- 172.21.15.44:0/3509972458 >> v2:172.21.15.79:3300/0 conn(0x7fbff40a7d90 0x7fbff4145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 --2- 172.21.15.44:0/3509972458 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff40b1bb0 0x7fbff40a78a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 -- 172.21.15.44:0/3509972458 >> 172.21.15.44:0/3509972458 conn(0x7fbff4060c40 msgr2=0x7fbff40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:55.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 -- 172.21.15.44:0/3509972458 shutdown_connections 2020-11-20T12:19:55.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.034+0000 7fbffa0db700 1 -- 172.21.15.44:0/3509972458 wait complete. 2020-11-20T12:19:55.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.035+0000 7fbffa0db700 1 Processor -- start 2020-11-20T12:19:55.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbffa0db700 1 -- start start 2020-11-20T12:19:55.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbffa0db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff40b17d0 0x7fbff40a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:55.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbffa0db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff40b1bb0 0x7fbff40afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:55.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbffa0db700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff414dd10 0x7fbff414c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:55.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbffa0db700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff409fe90 con 0x7fbff40b1bb0 2020-11-20T12:19:55.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbffa0db700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b0740 con 0x7fbff40b17d0 2020-11-20T12:19:55.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbffa0db700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff414d5d0 con 0x7fbff414dd10 2020-11-20T12:19:55.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbff88d8700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff40b1bb0 msgr2=0x7fbff40afea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:19:55.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.036+0000 7fbff88d8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff40b1bb0 0x7fbff40afea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:55.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff414dd10 0x7fbff414c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:55.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff414dd10 0x7fbff414c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54384/0 (socket says 172.21.15.44:54384) 2020-11-20T12:19:55.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 -- 172.21.15.44:0/1002293992 learned_addr learned my addr 172.21.15.44:0/1002293992 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:55.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff90d9700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff40b17d0 0x7fbff40a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:55.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 -- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff40b17d0 msgr2=0x7fbff40a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:55.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff40b17d0 0x7fbff40a7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 -- 172.21.15.44:0/1002293992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff40b1bb0 msgr2=0x7fbff40afea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:55.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff40b1bb0 0x7fbff40afea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:55.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 -- 172.21.15.44:0/1002293992 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff414d870 con 0x7fbff414dd10 2020-11-20T12:19:55.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbff98da700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff414dd10 0x7fbff414c910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fbfec00ef30 tx=0x7fbfec0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:55.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbfe17fa700 1 -- 172.21.15.44:0/1002293992 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00cd40 con 0x7fbff414dd10 2020-11-20T12:19:55.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbfe17fa700 1 -- 172.21.15.44:0/1002293992 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00cea0 con 0x7fbff414dd10 2020-11-20T12:19:55.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.037+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff414b430 con 0x7fbff414dd10 2020-11-20T12:19:55.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.038+0000 7fbfe17fa700 1 -- 172.21.15.44:0/1002293992 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec012620 con 0x7fbff414dd10 2020-11-20T12:19:55.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.038+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff40a17a0 con 0x7fbff414dd10 2020-11-20T12:19:55.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.039+0000 7fbfe17fa700 1 -- 172.21.15.44:0/1002293992 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fbfec022070 con 0x7fbff414dd10 2020-11-20T12:19:55.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.039+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff40616c0 con 0x7fbff414dd10 2020-11-20T12:19:55.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.039+0000 7fbfe17fa700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfcc056cd0 0x7fbfcc059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:55.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.039+0000 7fbfe17fa700 1 -- 172.21.15.44:0/1002293992 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fbfec0709d0 con 0x7fbff414dd10 2020-11-20T12:19:55.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.039+0000 7fbff90d9700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfcc056cd0 0x7fbfcc059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:55.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.040+0000 7fbff90d9700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfcc056cd0 0x7fbfcc059190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fbff00035f0 tx=0x7fbff0007b90).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:55.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.043+0000 7fbfe17fa700 1 -- 172.21.15.44:0/1002293992 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbfec045c40 con 0x7fbff414dd10 2020-11-20T12:19:55.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.334+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbff414da00 con 0x7fbff414dd10 2020-11-20T12:19:55.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.335+0000 7fbfe17fa700 1 -- 172.21.15.44:0/1002293992 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fbfec016070 con 0x7fbff414dd10 2020-11-20T12:19:55.338 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:55.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfcc056cd0 msgr2=0x7fbfcc059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:55.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfcc056cd0 0x7fbfcc059190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fbff00035f0 tx=0x7fbff0007b90).stop 2020-11-20T12:19:55.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff414dd10 msgr2=0x7fbff414c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:55.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff414dd10 0x7fbff414c910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fbfec00ef30 tx=0x7fbfec0091d0).stop 2020-11-20T12:19:55.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 shutdown_connections 2020-11-20T12:19:55.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fbfcc056cd0 0x7fbfcc059190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fbff414dd10 0x7fbff414c910 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbff40b17d0 0x7fbff40a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 --2- 172.21.15.44:0/1002293992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbff40b1bb0 0x7fbff40afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:55.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.336+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 >> 172.21.15.44:0/1002293992 conn(0x7fbff4060c40 msgr2=0x7fbff40aac70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:55.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.337+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 shutdown_connections 2020-11-20T12:19:55.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:55.337+0000 7fbffa0db700 1 -- 172.21.15.44:0/1002293992 wait complete. 2020-11-20T12:19:58.353 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:19:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.494+0000 7f4a71cf2700 1 Processor -- start 2020-11-20T12:19:58.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.494+0000 7f4a71cf2700 1 -- start start 2020-11-20T12:19:58.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a71cf2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c060330 0x7f4a6c0a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a71cf2700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6c0a6260 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a71cf2700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6c0a6490 con 0x7f4a6c0a4c30 2020-11-20T12:19:58.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a71cf2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0a52e0 con 0x7f4a6c060330 2020-11-20T12:19:58.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c060330 0x7f4a6c0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a63fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4a6c0a4c30 0x7f4a6c13a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49576/0 (socket says 172.21.15.44:49576) 2020-11-20T12:19:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a63fff700 1 -- 172.21.15.44:0/1715915523 learned_addr learned my addr 172.21.15.44:0/1715915523 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:19:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a6bfff700 1 -- 172.21.15.44:0/1715915523 >> v1:172.21.15.44:6789/0 conn(0x7f4a6c13b0a0 legacy=0x7f4a6c141810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:19:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a6affd700 1 -- 172.21.15.44:0/1715915523 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536044699 0 0) 0x7f4a6c0a6490 con 0x7f4a6c0a4c30 2020-11-20T12:19:58.501 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a6affd700 1 -- 172.21.15.44:0/1715915523 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a4c003240 con 0x7f4a6c0a4c30 2020-11-20T12:19:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a6b7fe700 1 -- 172.21.15.44:0/1715915523 >> v1:172.21.15.79:6789/0 conn(0x7f4a6c0a4c30 legacy=0x7f4a6c13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a6b7fe700 1 -- 172.21.15.44:0/1715915523 >> v1:172.21.15.44:6789/0 conn(0x7f4a6c13b0a0 legacy=0x7f4a6c141810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:19:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.495+0000 7f4a6b7fe700 1 -- 172.21.15.44:0/1715915523 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0a6730 con 0x7f4a6c060330 2020-11-20T12:19:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a6b7fe700 1 --2- 172.21.15.44:0/1715915523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c060330 0x7f4a6c0a4740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4a5c00b9d0 tx=0x7f4a5c006df0).ready entity=mon.2 client_cookie=ba121e098e9e574c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:58.502 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a6affd700 1 -- 172.21.15.44:0/1715915523 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c014070 con 0x7f4a6c060330 2020-11-20T12:19:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a6affd700 1 -- 172.21.15.44:0/1715915523 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c009770 con 0x7f4a6c060330 2020-11-20T12:19:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a6affd700 1 -- 172.21.15.44:0/1715915523 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c009af0 con 0x7f4a6c060330 2020-11-20T12:19:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a71cf2700 1 -- 172.21.15.44:0/1715915523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c060330 msgr2=0x7f4a6c0a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/1715915523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c060330 0x7f4a6c0a4740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4a5c00b9d0 tx=0x7f4a5c006df0).stop 2020-11-20T12:19:58.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a71cf2700 1 -- 172.21.15.44:0/1715915523 shutdown_connections 2020-11-20T12:19:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/1715915523 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c060330 0x7f4a6c0a4740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a71cf2700 1 -- 172.21.15.44:0/1715915523 >> 172.21.15.44:0/1715915523 conn(0x7f4a6c062690 msgr2=0x7f4a6c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a71cf2700 1 -- 172.21.15.44:0/1715915523 shutdown_connections 2020-11-20T12:19:58.504 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.496+0000 7f4a71cf2700 1 -- 172.21.15.44:0/1715915523 wait complete. 2020-11-20T12:19:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.497+0000 7f4a71cf2700 1 Processor -- start 2020-11-20T12:19:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.497+0000 7f4a71cf2700 1 -- start start 2020-11-20T12:19:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a71cf2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a6c060330 0x7f4a6c14ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.507 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a71cf2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c0a4c30 0x7f4a6c0ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a71cf2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c13b0a0 0x7f4a6c137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a71cf2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c14a2c0 con 0x7f4a6c0a4c30 2020-11-20T12:19:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a71cf2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0add40 con 0x7f4a6c060330 2020-11-20T12:19:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a71cf2700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c137e50 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a63fff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c0a4c30 msgr2=0x7f4a6c0ac080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:19:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.498+0000 7f4a63fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c0a4c30 0x7f4a6c0ac080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:19:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c13b0a0 0x7f4a6c137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a6c060330 0x7f4a6c14ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c13b0a0 0x7f4a6c137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54398/0 (socket says 172.21.15.44:54398) 2020-11-20T12:19:58.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 -- 172.21.15.44:0/2626513813 learned_addr learned my addr 172.21.15.44:0/2626513813 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:19:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a6c060330 0x7f4a6c14ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35006/0 (socket says 172.21.15.44:35006) 2020-11-20T12:19:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 -- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a6c060330 msgr2=0x7f4a6c14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a6c060330 0x7f4a6c14ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 -- 172.21.15.44:0/2626513813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c0a4c30 msgr2=0x7f4a6c0ac080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:19:58.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c0a4c30 0x7f4a6c0ac080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 -- 172.21.15.44:0/2626513813 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c14a4b0 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6b7fe700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a6c060330 0x7f4a6c14ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a6bfff700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c13b0a0 0x7f4a6c137960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4a6400bae0 tx=0x7f4a640090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:58.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a637fe700 1 -- 172.21.15.44:0/2626513813 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64003c30 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a637fe700 1 -- 172.21.15.44:0/2626513813 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a64003d90 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a637fe700 1 -- 172.21.15.44:0/2626513813 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64006170 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c14a710 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.499+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c14aa10 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.512 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.501+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c065570 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.501+0000 7f4a637fe700 1 -- 172.21.15.44:0/2626513813 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f4a64006560 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.501+0000 7f4a637fe700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a48056cd0 0x7f4a48059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.501+0000 7f4a637fe700 1 -- 172.21.15.44:0/2626513813 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f4a64011020 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.501+0000 7f4a6b7fe700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a48056cd0 0x7f4a48059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.502+0000 7f4a6b7fe700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a48056cd0 0x7f4a48059190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f4a5c00b150 tx=0x7f4a5c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:58.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.505+0000 7f4a637fe700 1 -- 172.21.15.44:0/2626513813 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4a64044bb0 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.790+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4a6c065810 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.790+0000 7f4a637fe700 1 -- 172.21.15.44:0/2626513813 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f4a640481d0 con 0x7f4a6c13b0a0 2020-11-20T12:19:58.794 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:19:58.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a48056cd0 msgr2=0x7f4a48059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a48056cd0 0x7f4a48059190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f4a5c00b150 tx=0x7f4a5c016000).stop 2020-11-20T12:19:58.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c13b0a0 msgr2=0x7f4a6c137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c13b0a0 0x7f4a6c137960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4a6400bae0 tx=0x7f4a640090f0).stop 2020-11-20T12:19:58.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 shutdown_connections 2020-11-20T12:19:58.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4a48056cd0 0x7f4a48059190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f4a6c13b0a0 0x7f4a6c137960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a6c060330 0x7f4a6c14ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 --2- 172.21.15.44:0/2626513813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c0a4c30 0x7f4a6c0ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:58.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 >> 172.21.15.44:0/2626513813 conn(0x7f4a6c062690 msgr2=0x7f4a6c1394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:58.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 shutdown_connections 2020-11-20T12:19:58.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:19:58.792+0000 7f4a71cf2700 1 -- 172.21.15.44:0/2626513813 wait complete. 2020-11-20T12:20:01.807 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.966+0000 7f88fbdb5700 1 Processor -- start 2020-11-20T12:20:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.967+0000 7f88fbdb5700 1 -- start start 2020-11-20T12:20:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.967+0000 7f88fbdb5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f40a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.967+0000 7f88fbdb5700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40a6260 con 0x7f88f413b0a0 2020-11-20T12:20:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.967+0000 7f88fbdb5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40a6490 con 0x7f88f40a4c30 2020-11-20T12:20:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.967+0000 7f88fbdb5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40a52e0 con 0x7f88f4060330 2020-11-20T12:20:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.967+0000 7f88fa352700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f88f413b0a0 legacy=0x7f88f4141810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:20:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f9350700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f88f40a4c30 0x7f88f413a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49588/0 (socket says 172.21.15.44:49588) 2020-11-20T12:20:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f9350700 1 -- 172.21.15.44:0/1006634994 learned_addr learned my addr 172.21.15.44:0/1006634994 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:20:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f9b51700 1 --2- 172.21.15.44:0/1006634994 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f40a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f9b51700 1 -- 172.21.15.44:0/1006634994 >> v1:172.21.15.79:6789/0 conn(0x7f88f40a4c30 legacy=0x7f88f413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:01.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f9b51700 1 -- 172.21.15.44:0/1006634994 >> v1:172.21.15.44:6789/0 conn(0x7f88f413b0a0 legacy=0x7f88f4141810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:01.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f9b51700 1 -- 172.21.15.44:0/1006634994 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40a6730 con 0x7f88f4060330 2020-11-20T12:20:01.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f9b51700 1 --2- 172.21.15.44:0/1006634994 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f40a4740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f88f000b9d0 tx=0x7f88f0006df0).ready entity=mon.2 client_cookie=e10f8ce4039b1ee6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:01.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f8b4f700 1 -- 172.21.15.44:0/1006634994 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0014070 con 0x7f88f4060330 2020-11-20T12:20:01.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f8b4f700 1 -- 172.21.15.44:0/1006634994 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f0009830 con 0x7f88f4060330 2020-11-20T12:20:01.974 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.968+0000 7f88f8b4f700 1 -- 172.21.15.44:0/1006634994 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0009bb0 con 0x7f88f4060330 2020-11-20T12:20:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.969+0000 7f88fbdb5700 1 -- 172.21.15.44:0/1006634994 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 msgr2=0x7f88f40a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.969+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/1006634994 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f40a4740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f88f000b9d0 tx=0x7f88f0006df0).stop 2020-11-20T12:20:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.969+0000 7f88fbdb5700 1 -- 172.21.15.44:0/1006634994 shutdown_connections 2020-11-20T12:20:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.969+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/1006634994 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f40a4740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:01.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.969+0000 7f88fbdb5700 1 -- 172.21.15.44:0/1006634994 >> 172.21.15.44:0/1006634994 conn(0x7f88f4062690 msgr2=0x7f88f40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:01.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.969+0000 7f88fbdb5700 1 -- 172.21.15.44:0/1006634994 shutdown_connections 2020-11-20T12:20:01.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.969+0000 7f88fbdb5700 1 -- 172.21.15.44:0/1006634994 wait complete. 2020-11-20T12:20:01.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.970+0000 7f88fbdb5700 1 Processor -- start 2020-11-20T12:20:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.970+0000 7f88fbdb5700 1 -- start start 2020-11-20T12:20:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88fbdb5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f4139220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88fbdb5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88f40a4c30 0x7f88f4137580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:01.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88fbdb5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f413b0a0 0x7f88f414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88fbdb5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f414e730 con 0x7f88f40a4c30 2020-11-20T12:20:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88fbdb5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f414dd50 con 0x7f88f413b0a0 2020-11-20T12:20:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88fbdb5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f414dff0 con 0x7f88f4060330 2020-11-20T12:20:01.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88f9350700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88f40a4c30 msgr2=0x7f88f4137580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88f9350700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88f40a4c30 0x7f88f4137580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88f9b51700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f4139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88fa352700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f413b0a0 0x7f88f414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88f9b51700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f4139220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54408/0 (socket says 172.21.15.44:54408) 2020-11-20T12:20:01.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.971+0000 7f88f9b51700 1 -- 172.21.15.44:0/2616837716 learned_addr learned my addr 172.21.15.44:0/2616837716 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:01.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88fa352700 1 -- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 msgr2=0x7f88f4139220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:01.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88fa352700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f4139220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:01.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88fa352700 1 -- 172.21.15.44:0/2616837716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88f40a4c30 msgr2=0x7f88f4137580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:01.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88fa352700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88f40a4c30 0x7f88f4137580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:01.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88fa352700 1 -- 172.21.15.44:0/2616837716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40a2e70 con 0x7f88f413b0a0 2020-11-20T12:20:01.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88f9b51700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f4139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:01.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88fa352700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f413b0a0 0x7f88f414a270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f88ec00bae0 tx=0x7f88ec009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:01.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88e1ffb700 1 -- 172.21.15.44:0/2616837716 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec003c30 con 0x7f88f413b0a0 2020-11-20T12:20:01.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88e1ffb700 1 -- 172.21.15.44:0/2616837716 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec003db0 con 0x7f88f413b0a0 2020-11-20T12:20:01.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.972+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f414d840 con 0x7f88f413b0a0 2020-11-20T12:20:01.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.973+0000 7f88e1ffb700 1 -- 172.21.15.44:0/2616837716 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec006170 con 0x7f88f413b0a0 2020-11-20T12:20:01.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.973+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f40ac750 con 0x7f88f413b0a0 2020-11-20T12:20:01.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.973+0000 7f88e1ffb700 1 -- 172.21.15.44:0/2616837716 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f88ec006560 con 0x7f88f413b0a0 2020-11-20T12:20:01.985 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.974+0000 7f88e1ffb700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88cc056cd0 0x7f88cc059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.974+0000 7f88e1ffb700 1 -- 172.21.15.44:0/2616837716 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f88ec011020 con 0x7f88f413b0a0 2020-11-20T12:20:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.974+0000 7f88f9b51700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88cc056cd0 0x7f88cc059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.974+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f414dbc0 con 0x7f88f413b0a0 2020-11-20T12:20:01.986 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.974+0000 7f88f9b51700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88cc056cd0 0x7f88cc059190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f88f0006e30 tx=0x7f88f0016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:01.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:01.979+0000 7f88e1ffb700 1 -- 172.21.15.44:0/2616837716 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f88ec044480 con 0x7f88f413b0a0 2020-11-20T12:20:02.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.269+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f88f4061650 con 0x7f88f413b0a0 2020-11-20T12:20:02.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.270+0000 7f88e1ffb700 1 -- 172.21.15.44:0/2616837716 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f88ec047aa0 con 0x7f88f413b0a0 2020-11-20T12:20:02.274 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:02.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88cc056cd0 msgr2=0x7f88cc059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88cc056cd0 0x7f88cc059190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f88f0006e30 tx=0x7f88f0016000).stop 2020-11-20T12:20:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f413b0a0 msgr2=0x7f88f414a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f413b0a0 0x7f88f414a270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f88ec00bae0 tx=0x7f88ec009060).stop 2020-11-20T12:20:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 shutdown_connections 2020-11-20T12:20:02.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f88cc056cd0 0x7f88cc059190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f88f413b0a0 0x7f88f414a270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f88f4060330 0x7f88f4139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 --2- 172.21.15.44:0/2616837716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88f40a4c30 0x7f88f4137580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 >> 172.21.15.44:0/2616837716 conn(0x7f88f4062690 msgr2=0x7f88f40a3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:02.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 shutdown_connections 2020-11-20T12:20:02.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:02.272+0000 7f88fbdb5700 1 -- 172.21.15.44:0/2616837716 wait complete. 2020-11-20T12:20:05.289 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:05.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.439+0000 7ff272f45700 1 Processor -- start 2020-11-20T12:20:05.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.439+0000 7ff272f45700 1 -- start start 2020-11-20T12:20:05.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.440+0000 7ff272f45700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:05.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.440+0000 7ff272f45700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0a7880 con 0x7ff26c0a9480 2020-11-20T12:20:05.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.440+0000 7ff272f45700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0a5b00 con 0x7ff26c0a4330 2020-11-20T12:20:05.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.440+0000 7ff272f45700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0a5da0 con 0x7ff26c0aa1e0 2020-11-20T12:20:05.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.440+0000 7ff2714e2700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7ff26c0a9480 legacy=0x7ff26c1497d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:20:05.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff270ce1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff270ce1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35028/0 (socket says 172.21.15.44:35028) 2020-11-20T12:20:05.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 learned_addr learned my addr 172.21.15.44:0/4118662681 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:05.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff26bfff700 1 --1- 172.21.15.44:0/4118662681 >> v1:172.21.15.79:6789/0 conn(0x7ff26c0a4330 0x7ff26c1427e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49602/0 (socket says 172.21.15.44:49602) 2020-11-20T12:20:05.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:05.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:05.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:05.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.441+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.641+0000 7ff2714e2700 1 -- 172.21.15.44:0/4118662681 >> v1:172.21.15.44:6789/0 conn(0x7ff26c0a9480 legacy=0x7ff26c1497d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:20:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.642+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.642+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.642+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.642+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:05.642+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.042+0000 7ff2714e2700 1 -- 172.21.15.44:0/4118662681 >> v1:172.21.15.44:6789/0 conn(0x7ff26c0a9480 legacy=0x7ff26c1497d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:20:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.043+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.043+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.043+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.043+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:06.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.043+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:06.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.843+0000 7ff2714e2700 1 -- 172.21.15.44:0/4118662681 >> v1:172.21.15.44:6789/0 conn(0x7ff26c0a9480 legacy=0x7ff26c1497d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:20:06.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.844+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.844+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:06.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.844+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:06.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.844+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:06.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:06.844+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:20:08.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 msgr2=0x7ff26c0a37c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:08.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.79:3300/0 conn(0x7ff26c0aa1e0 0x7ff26c0a37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:08.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 -- 172.21.15.44:0/4118662681 >> v1:172.21.15.79:6789/0 conn(0x7ff26c0a4330 legacy=0x7ff26c1427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:08.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 -- 172.21.15.44:0/4118662681 >> v1:172.21.15.44:6789/0 conn(0x7ff26c0a9480 legacy=0x7ff26c1497d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:08.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:08.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 0x7ff254008350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:08.445 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 -- 172.21.15.44:0/4118662681 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0a5da0 con 0x7ff254005eb0 2020-11-20T12:20:08.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26a7fc700 1 -- 172.21.15.44:0/4118662681 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540089b0 con 0x7ff254003390 2020-11-20T12:20:08.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26bfff700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 msgr2=0x7ff254008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:08.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.440+0000 7ff26bfff700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 0x7ff254008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:08.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.441+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:08.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.441+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 msgr2=0x7ff2540057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:08.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.441+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 msgr2=0x7ff2540057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:08.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.441+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:08.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.441+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:08.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.641+0000 7ff26bfff700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 msgr2=0x7ff254008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:08.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.641+0000 7ff26bfff700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 0x7ff254008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:08.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.642+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:08.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.642+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 msgr2=0x7ff2540057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:08.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.642+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 msgr2=0x7ff2540057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:08.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.642+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:08.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:08.642+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:09.016 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:20:09.020 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:20:09.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.042+0000 7ff26bfff700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 msgr2=0x7ff254008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:09.045 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.042+0000 7ff26bfff700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 0x7ff254008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:09.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.043+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.047 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.043+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 msgr2=0x7ff2540057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:09.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.043+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 msgr2=0x7ff2540057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:09.048 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.043+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:09.049 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.043+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:09.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.843+0000 7ff26bfff700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 msgr2=0x7ff254008350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:09.846 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.843+0000 7ff26bfff700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 0x7ff254008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:20:09.847 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.844+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.844+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 msgr2=0x7ff254008350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:09.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.844+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> v2:172.21.15.44:3300/0 conn(0x7ff254005eb0 0x7ff254008350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:09.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.844+0000 7ff270ce1700 1 -- 172.21.15.44:0/4118662681 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0a6040 con 0x7ff254003390 2020-11-20T12:20:09.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.844+0000 7ff270ce1700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff260006cc0 tx=0x7ff260008040).ready entity=mon.2 client_cookie=636ef75006a31b5d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.848 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.844+0000 7ff26b7fe700 1 -- 172.21.15.44:0/4118662681 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260019d70 con 0x7ff254003390 2020-11-20T12:20:09.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff26b7fe700 1 -- 172.21.15.44:0/4118662681 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff260022440 con 0x7ff254003390 2020-11-20T12:20:09.849 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff26b7fe700 1 -- 172.21.15.44:0/4118662681 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2600227c0 con 0x7ff254003390 2020-11-20T12:20:09.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff272f45700 1 -- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 msgr2=0x7ff2540057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.851 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff272f45700 1 --2- 172.21.15.44:0/4118662681 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff254003390 0x7ff2540057e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ff260006cc0 tx=0x7ff260008040).stop 2020-11-20T12:20:09.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff2714e2700 1 -- 172.21.15.44:0/4118662681 reap_dead start 2020-11-20T12:20:09.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff272f45700 1 -- 172.21.15.44:0/4118662681 shutdown_connections 2020-11-20T12:20:09.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff272f45700 1 -- 172.21.15.44:0/4118662681 >> 172.21.15.44:0/4118662681 conn(0x7ff26c062690 msgr2=0x7ff26c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:09.852 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff272f45700 1 -- 172.21.15.44:0/4118662681 shutdown_connections 2020-11-20T12:20:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.845+0000 7ff272f45700 1 -- 172.21.15.44:0/4118662681 wait complete. 2020-11-20T12:20:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.846+0000 7ff272f45700 1 Processor -- start 2020-11-20T12:20:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff272f45700 1 -- start start 2020-11-20T12:20:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff272f45700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff254003390 0x7ff26c0aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.853 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff272f45700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c14e670 0x7ff26c0668a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff272f45700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff26c0655b0 0x7ff26c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff272f45700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0adba0 con 0x7ff254003390 2020-11-20T12:20:09.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff272f45700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c066160 con 0x7ff26c14e670 2020-11-20T12:20:09.854 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff272f45700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c066400 con 0x7ff26c0655b0 2020-11-20T12:20:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff270ce1700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff254003390 msgr2=0x7ff26c0aafb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff270ce1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff254003390 0x7ff26c0aafb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff2714e2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff26c0655b0 0x7ff26c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.855 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c14e670 0x7ff26c0668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff2714e2700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff26c0655b0 0x7ff26c151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54454/0 (socket says 172.21.15.44:54454) 2020-11-20T12:20:09.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c14e670 0x7ff26c0668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35064/0 (socket says 172.21.15.44:35064) 2020-11-20T12:20:09.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.847+0000 7ff2714e2700 1 -- 172.21.15.44:0/2243737687 learned_addr learned my addr 172.21.15.44:0/2243737687 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:09.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff26bfff700 1 -- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff26c0655b0 msgr2=0x7ff26c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.856 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff26bfff700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff26c0655b0 0x7ff26c151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff26bfff700 1 -- 172.21.15.44:0/2243737687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff254003390 msgr2=0x7ff26c0aafb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:09.857 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff26bfff700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff254003390 0x7ff26c0aafb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:09.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff26bfff700 1 -- 172.21.15.44:0/2243737687 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c066d90 con 0x7ff26c14e670 2020-11-20T12:20:09.858 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff2714e2700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff26c0655b0 0x7ff26c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:09.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff26bfff700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c14e670 0x7ff26c0668a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff258007fd0 tx=0x7ff258007a80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.859 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff268ff9700 1 -- 172.21.15.44:0/2243737687 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580072e0 con 0x7ff26c14e670 2020-11-20T12:20:09.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff268ff9700 1 -- 172.21.15.44:0/2243737687 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff258005530 con 0x7ff26c14e670 2020-11-20T12:20:09.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff268ff9700 1 -- 172.21.15.44:0/2243737687 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580058d0 con 0x7ff26c14e670 2020-11-20T12:20:09.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c065a10 con 0x7ff26c14e670 2020-11-20T12:20:09.860 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.848+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c1522f0 con 0x7ff26c14e670 2020-11-20T12:20:09.861 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.850+0000 7ff268ff9700 1 -- 172.21.15.44:0/2243737687 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff258010070 con 0x7ff26c14e670 2020-11-20T12:20:09.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.850+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c0a3560 con 0x7ff26c14e670 2020-11-20T12:20:09.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.850+0000 7ff268ff9700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff244056d00 0x7ff2440591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.850+0000 7ff268ff9700 1 -- 172.21.15.44:0/2243737687 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff25806f6e0 con 0x7ff26c14e670 2020-11-20T12:20:09.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.850+0000 7ff270ce1700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff244056d00 0x7ff2440591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.862 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.851+0000 7ff270ce1700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff244056d00 0x7ff2440591c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7ff260019a70 tx=0x7ff260022000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.863 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:09.854+0000 7ff268ff9700 1 -- 172.21.15.44:0/2243737687 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff2580449b0 con 0x7ff26c14e670 2020-11-20T12:20:10.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.133+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff26c0665c0 con 0x7ff26c14e670 2020-11-20T12:20:10.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.133+0000 7ff268ff9700 1 -- 172.21.15.44:0/2243737687 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7ff258047fd0 con 0x7ff26c14e670 2020-11-20T12:20:10.138 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":160,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff244056d00 msgr2=0x7ff2440591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff244056d00 0x7ff2440591c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7ff260019a70 tx=0x7ff260022000).stop 2020-11-20T12:20:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c14e670 msgr2=0x7ff26c0668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c14e670 0x7ff26c0668a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff258007fd0 tx=0x7ff258007a80).stop 2020-11-20T12:20:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 shutdown_connections 2020-11-20T12:20:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff244056d00 0x7ff2440591c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff26c0655b0 0x7ff26c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff254003390 0x7ff26c0aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:10.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 --2- 172.21.15.44:0/2243737687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c14e670 0x7ff26c0668a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 >> 172.21.15.44:0/2243737687 conn(0x7ff26c062690 msgr2=0x7ff26c142d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 shutdown_connections 2020-11-20T12:20:10.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.135+0000 7ff272f45700 1 -- 172.21.15.44:0/2243737687 wait complete. 2020-11-20T12:20:10.150 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:20:10.150 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:20:10.305 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.302+0000 7fa1b004f700 1 Processor -- start 2020-11-20T12:20:10.306 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.302+0000 7fa1b004f700 1 -- start start 2020-11-20T12:20:10.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1b004f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa1a80b1970 0x7fa1a813ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.309 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1b004f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1b004f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a80ab650 con 0x7fa1a80ab220 2020-11-20T12:20:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1b004f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a80aae30 con 0x7fa1a80b1970 2020-11-20T12:20:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1b004f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a80aaa00 con 0x7fa1a80b1d50 2020-11-20T12:20:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1ae5ec700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fa1a80ab220 legacy=0x7fa1a813b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:20:10.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1ad5ea700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1addeb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa1a80b1970 0x7fa1a813ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1ad5ea700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54458/0 (socket says 172.21.15.44:54458) 2020-11-20T12:20:10.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.303+0000 7fa1ad5ea700 1 -- 172.21.15.44:0/2619486107 learned_addr learned my addr 172.21.15.44:0/2619486107 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:10.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1ad5ea700 1 -- 172.21.15.44:0/2619486107 >> v2:172.21.15.79:3300/0 conn(0x7fa1a80b1970 msgr2=0x7fa1a813ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:10.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1ad5ea700 1 --2- 172.21.15.44:0/2619486107 >> v2:172.21.15.79:3300/0 conn(0x7fa1a80b1970 0x7fa1a813ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1ad5ea700 1 -- 172.21.15.44:0/2619486107 >> v1:172.21.15.44:6789/0 conn(0x7fa1a80ab220 legacy=0x7fa1a813b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:10.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1ad5ea700 1 -- 172.21.15.44:0/2619486107 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a80a9610 con 0x7fa1a80b1d50 2020-11-20T12:20:10.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1ad5ea700 1 --2- 172.21.15.44:0/2619486107 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a86a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa198006fd0 tx=0x7fa198006eb0).ready entity=mon.2 client_cookie=d82789ab36e3fc49 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1acde9700 1 -- 172.21.15.44:0/2619486107 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198009860 con 0x7fa1a80b1d50 2020-11-20T12:20:10.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1acde9700 1 -- 172.21.15.44:0/2619486107 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1980099c0 con 0x7fa1a80b1d50 2020-11-20T12:20:10.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1acde9700 1 -- 172.21.15.44:0/2619486107 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198009d40 con 0x7fa1a80b1d50 2020-11-20T12:20:10.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1b004f700 1 -- 172.21.15.44:0/2619486107 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 msgr2=0x7fa1a80a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1b004f700 1 --2- 172.21.15.44:0/2619486107 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a86a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa198006fd0 tx=0x7fa198006eb0).stop 2020-11-20T12:20:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1b004f700 1 -- 172.21.15.44:0/2619486107 shutdown_connections 2020-11-20T12:20:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1b004f700 1 --2- 172.21.15.44:0/2619486107 >> v2:172.21.15.79:3300/0 conn(0x7fa1a80b1970 0x7fa1a813ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1b004f700 1 --2- 172.21.15.44:0/2619486107 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a86a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.304+0000 7fa1b004f700 1 -- 172.21.15.44:0/2619486107 >> 172.21.15.44:0/2619486107 conn(0x7fa1a8060c40 msgr2=0x7fa1a80b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.305+0000 7fa1b004f700 1 -- 172.21.15.44:0/2619486107 shutdown_connections 2020-11-20T12:20:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.305+0000 7fa1b004f700 1 -- 172.21.15.44:0/2619486107 wait complete. 2020-11-20T12:20:10.319 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.306+0000 7fa1b004f700 1 Processor -- start 2020-11-20T12:20:10.320 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.306+0000 7fa1b004f700 1 -- start start 2020-11-20T12:20:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.306+0000 7fa1b004f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80ab220 0x7fa1a809e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.321 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.306+0000 7fa1b004f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1a80b1970 0x7fa1a80a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.306+0000 7fa1b004f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.306+0000 7fa1b004f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a814ad60 con 0x7fa1a80b1970 2020-11-20T12:20:10.322 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1b004f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a814a380 con 0x7fa1a80ab220 2020-11-20T12:20:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1b004f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a814a620 con 0x7fa1a80b1d50 2020-11-20T12:20:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1ad5ea700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1a80b1970 msgr2=0x7fa1a80a7890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1ad5ea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1a80b1970 0x7fa1a80a7890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:10.323 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1ae5ec700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1addeb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80ab220 0x7fa1a809e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1ae5ec700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54468/0 (socket says 172.21.15.44:54468) 2020-11-20T12:20:10.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1ae5ec700 1 -- 172.21.15.44:0/2859409601 learned_addr learned my addr 172.21.15.44:0/2859409601 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:10.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1addeb700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 msgr2=0x7fa1a80a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1addeb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1addeb700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1a80b1970 msgr2=0x7fa1a80a7890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1addeb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1a80b1970 0x7fa1a80a7890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:10.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1addeb700 1 -- 172.21.15.44:0/2859409601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a814a8c0 con 0x7fa1a80ab220 2020-11-20T12:20:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1ae5ec700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa1a80b1d50 0x7fa1a80a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.307+0000 7fa1addeb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80ab220 0x7fa1a809e500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa1a400eba0 tx=0x7fa1a4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.308+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400c970 con 0x7fa1a80ab220 2020-11-20T12:20:10.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.308+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a400cad0 con 0x7fa1a80ab220 2020-11-20T12:20:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.308+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a80a55b0 con 0x7fa1a80ab220 2020-11-20T12:20:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.308+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400ce50 con 0x7fa1a80ab220 2020-11-20T12:20:10.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.308+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a80a6820 con 0x7fa1a80ab220 2020-11-20T12:20:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.309+0000 7fa1b004f700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a8149e70 0x7fa1a80a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.309+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a80619a0 con 0x7fa1a8149e70 2020-11-20T12:20:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.309+0000 7fa1ad5ea700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a8149e70 0x7fa1a80a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.309+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa1a4012600 con 0x7fa1a80ab220 2020-11-20T12:20:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.309+0000 7fa1ad5ea700 1 -- 172.21.15.44:0/2859409601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1a80446d0 con 0x7fa1a8149e70 2020-11-20T12:20:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.309+0000 7fa1ad5ea700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a8149e70 0x7fa1a80a3a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa1a809dd70 tx=0x7fa198015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.309+0000 7fa195ffb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa180056cd0 0x7fa180059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa1a4070520 con 0x7fa1a80ab220 2020-11-20T12:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa1ae5ec700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa180056cd0 0x7fa180059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198011dd0 con 0x7fa1a8149e70 2020-11-20T12:20:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fa19801e460 con 0x7fa1a8149e70 2020-11-20T12:20:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a8149e70 msgr2=0x7fa1a80a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa195ffb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a8149e70 0x7fa1a80a3a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa1a809dd70 tx=0x7fa198015000).stop 2020-11-20T12:20:10.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a8149e70 msgr2=0x7fa1a80a3a80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:10.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa195ffb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a8149e70 0x7fa1a80a3a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.310+0000 7fa1ae5ec700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa180056cd0 0x7fa180059190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa1a8147a30 tx=0x7fa1a0009120).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.319+0000 7fa1b004f700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80a3f70 0x7fa1a814d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.319+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a80a4350 con 0x7fa1a80a3f70 2020-11-20T12:20:10.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.319+0000 7fa1ad5ea700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80a3f70 0x7fa1a814d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.319+0000 7fa1ad5ea700 1 -- 172.21.15.44:0/2859409601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa1a805c660 con 0x7fa1a80a3f70 2020-11-20T12:20:10.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.319+0000 7fa1ad5ea700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80a3f70 0x7fa1a814d440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa19800b1b0 tx=0x7fa19801e000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19801db20 con 0x7fa1a80a3f70 2020-11-20T12:20:10.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4767 (secure 0 0 0) 0x7fa198011bc0 con 0x7fa1a80a3f70 2020-11-20T12:20:10.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80a3f70 msgr2=0x7fa1a814d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa195ffb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80a3f70 0x7fa1a814d440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa19800b1b0 tx=0x7fa19801e000).stop 2020-11-20T12:20:10.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa195ffb700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80a3f70 msgr2=0x7fa1a814d440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:10.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa195ffb700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80a3f70 0x7fa1a814d440 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa180056cd0 msgr2=0x7fa180059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa180056cd0 0x7fa180059190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa1a8147a30 tx=0x7fa1a0009120).stop 2020-11-20T12:20:10.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80ab220 msgr2=0x7fa1a809e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 --2- 172.21.15.44:0/2859409601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1a80ab220 0x7fa1a809e500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa1a400eba0 tx=0x7fa1a4006df0).stop 2020-11-20T12:20:10.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1ae5ec700 1 -- 172.21.15.44:0/2859409601 reap_dead start 2020-11-20T12:20:10.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1ae5ec700 1 -- 172.21.15.44:0/2859409601 reap_dead start 2020-11-20T12:20:10.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 shutdown_connections 2020-11-20T12:20:10.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 >> 172.21.15.44:0/2859409601 conn(0x7fa1a8060c40 msgr2=0x7fa1a80a8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 shutdown_connections 2020-11-20T12:20:10.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.320+0000 7fa1b004f700 1 -- 172.21.15.44:0/2859409601 wait complete. 2020-11-20T12:20:10.340 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:20:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:20:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:20:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:20:10.341 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 160, 2020-11-20T12:20:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:20:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:20:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:20:10.342 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:20:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:20:10.343 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:20:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:10.344 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:10.345 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:20:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:20:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:10.347 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:20:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:20:10.349 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:20:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:20:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:20:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:20:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:10.350 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:20:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:20:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:20:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:20:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:20:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:20:10.352 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:10.353 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.354 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:20:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:20:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:20:10.355 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:20:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:10.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:20:10.357 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.358 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:20:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.359 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:20:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:20:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:10.360 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:20:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:20:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:20:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:20:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.364 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:20:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:20:10.365 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:10.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:20:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:20:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:10.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:10.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:10.368 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:20:10.369 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:10.370 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:20:10.370 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.370 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.371 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:20:10.371 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:20:10.371 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:10.372 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:10.372 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:10.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.372 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.373 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:20:10.373 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:20:10.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.374 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.374 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.374 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:10.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.375 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.375 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:20:10.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.376 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.376 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.376 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:10.376 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.377 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.377 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:20:10.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.377 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.378 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.378 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:20:10.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.379 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.379 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:20:10.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.379 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.380 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.380 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:20:10.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.380 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.381 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:20:10.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.381 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.382 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.382 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:10.382 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.382 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.383 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:20:10.383 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:20:10.385 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:20:10.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.525+0000 7f3d5f0b1700 1 Processor -- start 2020-11-20T12:20:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.525+0000 7f3d5f0b1700 1 -- start start 2020-11-20T12:20:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.526+0000 7f3d5f0b1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3d580aa020 0x7f3d580a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.526+0000 7f3d5f0b1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580a9770 con 0x7f3d580aa400 2020-11-20T12:20:10.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.526+0000 7f3d5f0b1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580a6da0 con 0x7f3d5813ac60 2020-11-20T12:20:10.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.526+0000 7f3d5f0b1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d580a56c0 con 0x7f3d580aa020 2020-11-20T12:20:10.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.526+0000 7f3d5ce4d700 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f3d580aa020 msgr2=0x7f3d580a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.526+0000 7f3d57fff700 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f3d580aa400 legacy=0x7f3d580a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2020-11-20T12:20:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.526+0000 7f3d5ce4d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3d580aa020 0x7f3d580a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:10.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d5d64e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3d5813ac60 0x7f3d580a3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49664/0 (socket says 172.21.15.44:49664) 2020-11-20T12:20:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d5d64e700 1 -- 172.21.15.44:0/861435788 learned_addr learned my addr 172.21.15.44:0/861435788 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:20:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451681001 0 0) 0x7f3d580a6da0 con 0x7f3d5813ac60 2020-11-20T12:20:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d3c003240 con 0x7f3d5813ac60 2020-11-20T12:20:10.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2436567129 0 0) 0x7f3d3c003240 con 0x7f3d5813ac60 2020-11-20T12:20:10.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 >> v2:172.21.15.44:3300/0 conn(0x7f3d580aa020 msgr2=0x7f3d580a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:10.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 --2- 172.21.15.44:0/861435788 >> v2:172.21.15.44:3300/0 conn(0x7f3d580aa020 0x7f3d580a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:10.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 >> v1:172.21.15.44:6789/0 conn(0x7f3d580aa400 legacy=0x7f3d580a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d580a7040 con 0x7f3d5813ac60 2020-11-20T12:20:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f3d50001860 con 0x7f3d5813ac60 2020-11-20T12:20:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 >> v1:172.21.15.79:6789/0 conn(0x7f3d5813ac60 legacy=0x7f3d580a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3c003730 0x7f3d3c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d3c004560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d3c004b60 0x7f3d3c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580a56c0 con 0x7f3d3c003730 2020-11-20T12:20:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c005200 con 0x7f3d3c004090 2020-11-20T12:20:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c00bc20 con 0x7f3d3c004b60 2020-11-20T12:20:10.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d5ce4d700 1 -- 172.21.15.44:0/861435788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3c003730 msgr2=0x7f3d3c003b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d5ce4d700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3c003730 0x7f3d3c003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.527+0000 7f3d57fff700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d3c004560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d5d64e700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d3c004b60 0x7f3d3c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d57fff700 1 -- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d3c004b60 msgr2=0x7f3d3c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d57fff700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d3c004b60 0x7f3d3c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d57fff700 1 -- 172.21.15.44:0/861435788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3c003730 msgr2=0x7f3d3c003b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d57fff700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d3c003730 0x7f3d3c003b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d57fff700 1 -- 172.21.15.44:0/861435788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c003240 con 0x7f3d3c004090 2020-11-20T12:20:10.544 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d5d64e700 1 -- 172.21.15.44:0/861435788 reap_dead start 2020-11-20T12:20:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d57fff700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d3c004560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3d440036a0 tx=0x7f3d44006e90).ready entity=mon.1 client_cookie=3e646595e901e2a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d440050d0 con 0x7f3d3c004090 2020-11-20T12:20:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d44005250 con 0x7f3d3c004090 2020-11-20T12:20:10.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.528+0000 7f3d577fe700 1 -- 172.21.15.44:0/861435788 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d440038e0 con 0x7f3d3c004090 2020-11-20T12:20:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.529+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 msgr2=0x7f3d3c004560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.529+0000 7f3d5f0b1700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d3c004560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f3d440036a0 tx=0x7f3d44006e90).stop 2020-11-20T12:20:10.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.529+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/861435788 shutdown_connections 2020-11-20T12:20:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.529+0000 7f3d5f0b1700 1 --2- 172.21.15.44:0/861435788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d3c004560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.529+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/861435788 >> 172.21.15.44:0/861435788 conn(0x7f3d58062690 msgr2=0x7f3d580a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.529+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/861435788 shutdown_connections 2020-11-20T12:20:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.529+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/861435788 wait complete. 2020-11-20T12:20:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 Processor -- start 2020-11-20T12:20:10.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 -- start start 2020-11-20T12:20:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d580674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d58146850 0x7f3d5809dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d580a2a40 0x7f3d580a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580a33b0 con 0x7f3d58146850 2020-11-20T12:20:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58065f40 con 0x7f3d3c004090 2020-11-20T12:20:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.530+0000 7f3d5f0b1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580661e0 con 0x7f3d580a2a40 2020-11-20T12:20:10.556 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d57fff700 1 -- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d58146850 msgr2=0x7f3d5809dfa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.44:3300/0 2020-11-20T12:20:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d57fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d58146850 0x7f3d5809dfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d580674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.557 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5d64e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d580a2a40 0x7f3d580a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d580674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35098/0 (socket says 172.21.15.44:35098) 2020-11-20T12:20:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5d64e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d580a2a40 0x7f3d580a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54488/0 (socket says 172.21.15.44:54488) 2020-11-20T12:20:10.558 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 -- 172.21.15.44:0/1210317132 learned_addr learned my addr 172.21.15.44:0/1210317132 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d580a2a40 msgr2=0x7f3d580a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d580a2a40 0x7f3d580a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d58146850 msgr2=0x7f3d5809dfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:10.559 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d58146850 0x7f3d5809dfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 -- 172.21.15.44:0/1210317132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d58066480 con 0x7f3d3c004090 2020-11-20T12:20:10.560 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5d64e700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d580a2a40 0x7f3d580a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d5ce4d700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d580674d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f3d4c00b7e0 tx=0x7f3d4c006ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c006170 con 0x7f3d3c004090 2020-11-20T12:20:10.561 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.531+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0062f0 con 0x7f3d3c004090 2020-11-20T12:20:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.532+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c003970 con 0x7f3d3c004090 2020-11-20T12:20:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.532+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d580679c0 con 0x7f3d3c004090 2020-11-20T12:20:10.562 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.532+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d580652e0 con 0x7f3d3c004090 2020-11-20T12:20:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.532+0000 7f3d5f0b1700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58146c70 0x7f3d58149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.532+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5809e650 con 0x7f3d58146c70 2020-11-20T12:20:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d57fff700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58146c70 0x7f3d58149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.563 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d57fff700 1 -- 172.21.15.44:0/1210317132 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d58066610 con 0x7f3d58146c70 2020-11-20T12:20:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3d4c017030 con 0x7f3d3c004090 2020-11-20T12:20:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d57fff700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58146c70 0x7f3d58149e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3d580add00 tx=0x7f3d44015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d54ff9700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d30056d00 0x7f3d300591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3d4c06f350 con 0x7f3d3c004090 2020-11-20T12:20:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d5d64e700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d30056d00 0x7f3d300591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.533+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44009c70 con 0x7f3d58146c70 2020-11-20T12:20:10.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.534+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f3d44018070 con 0x7f3d58146c70 2020-11-20T12:20:10.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.534+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58146c70 msgr2=0x7f3d58149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.534+0000 7f3d54ff9700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58146c70 0x7f3d58149e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3d580add00 tx=0x7f3d44015000).stop 2020-11-20T12:20:10.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.534+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58146c70 msgr2=0x7f3d58149e20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:10.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.534+0000 7f3d54ff9700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58146c70 0x7f3d58149e20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.534+0000 7f3d5d64e700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d30056d00 0x7f3d300591c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f3d58062aa0 tx=0x7f3d580a6f30).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.546+0000 7f3d5f0b1700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58066840 0x7f3d5814a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.546+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58066080 con 0x7f3d58066840 2020-11-20T12:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.546+0000 7f3d57fff700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58066840 0x7f3d5814a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d57fff700 1 -- 172.21.15.44:0/1210317132 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3d58061650 con 0x7f3d58066840 2020-11-20T12:20:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d57fff700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58066840 0x7f3d5814a310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3d44017000 tx=0x7f3d4401d000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44025070 con 0x7f3d58066840 2020-11-20T12:20:10.569 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4765 (secure 0 0 0) 0x7f3d44022640 con 0x7f3d58066840 2020-11-20T12:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58066840 msgr2=0x7f3d5814a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d54ff9700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58066840 0x7f3d5814a310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3d44017000 tx=0x7f3d4401d000).stop 2020-11-20T12:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d54ff9700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58066840 msgr2=0x7f3d5814a310 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:10.570 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.547+0000 7f3d54ff9700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f3d58066840 0x7f3d5814a310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d30056d00 msgr2=0x7f3d300591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f3d30056d00 0x7f3d300591c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f3d58062aa0 tx=0x7f3d580a6f30).stop 2020-11-20T12:20:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 msgr2=0x7f3d580674d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 --2- 172.21.15.44:0/1210317132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c004090 0x7f3d580674d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f3d4c00b7e0 tx=0x7f3d4c006ef0).stop 2020-11-20T12:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5d64e700 1 -- 172.21.15.44:0/1210317132 reap_dead start 2020-11-20T12:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5d64e700 1 -- 172.21.15.44:0/1210317132 reap_dead start 2020-11-20T12:20:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 shutdown_connections 2020-11-20T12:20:10.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 >> 172.21.15.44:0/1210317132 conn(0x7f3d58062690 msgr2=0x7f3d580a4460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.573 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 shutdown_connections 2020-11-20T12:20:10.574 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:10.548+0000 7f3d5f0b1700 1 -- 172.21.15.44:0/1210317132 wait complete. 2020-11-20T12:20:10.574 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:20:10.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:20:10.575 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:20:10.575 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:20:10.575 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 160, 2020-11-20T12:20:10.576 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:20:10.576 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:20:10.576 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:20:10.576 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.577 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:20:10.577 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:10.578 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:20:10.578 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:20:10.578 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:10.579 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:10.579 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:10.579 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:10.579 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:10.580 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:10.580 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:10.580 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:10.581 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.581 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:20:10.581 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:20:10.582 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:10.582 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:10.582 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:10.582 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:10.583 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:10.583 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:10.583 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:10.584 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:10.584 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.585 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:20:10.585 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:20:10.585 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:20:10.586 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:20:10.586 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:20:10.586 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:20:10.586 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:10.587 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:10.587 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:20:10.587 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:20:10.588 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:20:10.588 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:20:10.588 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:20:10.589 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:10.589 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:20:10.590 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:10.590 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:10.590 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:10.591 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:10.591 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:10.591 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:10.592 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:10.592 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:10.592 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.593 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:20:10.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.593 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:20:10.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.594 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:20:10.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:20:10.594 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:10.595 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:10.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:10.596 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:20:10.596 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:10.597 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:20:10.598 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.599 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.599 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:20:10.600 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:20:10.600 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:10.600 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:10.600 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:10.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.601 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:20:10.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:20:10.602 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:10.602 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:10.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.603 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:10.603 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:20:10.604 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:20:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.606 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.607 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:20:10.607 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:20:10.607 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:10.608 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:10.608 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:10.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.608 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:20:10.609 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:20:10.609 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:10.609 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:10.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.609 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:10.610 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:20:10.610 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.610 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:10.611 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:20:10.611 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:10.611 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.611 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.612 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:20:10.612 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:20:10.612 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:10.612 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:10.612 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:10.613 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.613 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.613 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:20:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:20:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 3 2020-11-20T12:20:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:20:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:20:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:20:10.622 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:20:10.623 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:20:30.624 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2020-11-20T12:20:30.624 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2020-11-20T12:20:30.624 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:20:30.625 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:20:30.637 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:20:30.637 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:20:30.638 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.788+0000 7f98f49a1700 1 Processor -- start 2020-11-20T12:20:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.788+0000 7f98f49a1700 1 -- start start 2020-11-20T12:20:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98f49a1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f98f00ae050 0x7f98f0060650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98f49a1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 0x7f98f0137b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98f49a1700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f013e080 con 0x7f98f00669d0 2020-11-20T12:20:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98f49a1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f0060b40 con 0x7f98f013ebf0 2020-11-20T12:20:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98f49a1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f0060d60 con 0x7f98f00ae050 2020-11-20T12:20:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98edd9b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f98f00669d0 0x7f98f013e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46816/0 (socket says 172.21.15.44:46816) 2020-11-20T12:20:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98eed9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 0x7f98f0137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98edd9b700 1 -- 172.21.15.44:0/2523257412 learned_addr learned my addr 172.21.15.44:0/2523257412 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:20:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98eed9d700 1 -- 172.21.15.44:0/2523257412 >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 msgr2=0x7f98f0137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:20:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98eed9d700 1 -- 172.21.15.44:0/2523257412 >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 msgr2=0x7f98f0137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98ee59c700 1 --2- 172.21.15.44:0/2523257412 >> v2:172.21.15.79:3300/0 conn(0x7f98f00ae050 0x7f98f0060650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98eed9d700 1 --2- 172.21.15.44:0/2523257412 >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 0x7f98f0137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98eed9d700 1 --2- 172.21.15.44:0/2523257412 >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 0x7f98f0137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98ee59c700 1 -- 172.21.15.44:0/2523257412 >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 msgr2=0x7f98f0137b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98ee59c700 1 --2- 172.21.15.44:0/2523257412 >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 0x7f98f0137b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98ee59c700 1 -- 172.21.15.44:0/2523257412 >> v1:172.21.15.44:6789/0 conn(0x7f98f00669d0 legacy=0x7f98f013e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98ee59c700 1 -- 172.21.15.44:0/2523257412 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f00ae430 con 0x7f98f00ae050 2020-11-20T12:20:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.789+0000 7f98ee59c700 1 --2- 172.21.15.44:0/2523257412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00ae050 0x7f98f0060650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f98f005d010 tx=0x7f98e0006df0).ready entity=mon.1 client_cookie=420dbe6820698c6a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98ed59a700 1 -- 172.21.15.44:0/2523257412 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e00097b0 con 0x7f98f00ae050 2020-11-20T12:20:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98ed59a700 1 -- 172.21.15.44:0/2523257412 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e0009910 con 0x7f98f00ae050 2020-11-20T12:20:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98ed59a700 1 -- 172.21.15.44:0/2523257412 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0009c90 con 0x7f98f00ae050 2020-11-20T12:20:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 -- 172.21.15.44:0/2523257412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00ae050 msgr2=0x7f98f0060650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 --2- 172.21.15.44:0/2523257412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00ae050 0x7f98f0060650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f98f005d010 tx=0x7f98e0006df0).stop 2020-11-20T12:20:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 -- 172.21.15.44:0/2523257412 shutdown_connections 2020-11-20T12:20:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 --2- 172.21.15.44:0/2523257412 >> v2:172.21.15.44:3300/0 conn(0x7f98f013ebf0 0x7f98f0137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 --2- 172.21.15.44:0/2523257412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00ae050 0x7f98f0060650 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 -- 172.21.15.44:0/2523257412 >> 172.21.15.44:0/2523257412 conn(0x7f98f0062bc0 msgr2=0x7f98f0062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 -- 172.21.15.44:0/2523257412 shutdown_connections 2020-11-20T12:20:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.790+0000 7f98f49a1700 1 -- 172.21.15.44:0/2523257412 wait complete. 2020-11-20T12:20:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.791+0000 7f98f49a1700 1 Processor -- start 2020-11-20T12:20:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.791+0000 7f98f49a1700 1 -- start start 2020-11-20T12:20:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98f49a1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00669d0 0x7f98f0139b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98f49a1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f98f00ae050 0x7f98f013f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98f49a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 0x7f98f00ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98f49a1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0135c70 con 0x7f98f013ebf0 2020-11-20T12:20:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98f49a1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0135f10 con 0x7f98f00669d0 2020-11-20T12:20:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98f49a1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f01358c0 con 0x7f98f00ae050 2020-11-20T12:20:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98eed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 0x7f98f00ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98eed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 0x7f98f00ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43076/0 (socket says 172.21.15.44:43076) 2020-11-20T12:20:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98eed9d700 1 -- 172.21.15.44:0/3094923084 learned_addr learned my addr 172.21.15.44:0/3094923084 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98ee59c700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00669d0 0x7f98f0139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98edd9b700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f98f00ae050 0x7f98f013f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98eed9d700 1 -- 172.21.15.44:0/3094923084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 msgr2=0x7f98f00ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:20:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98eed9d700 1 -- 172.21.15.44:0/3094923084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 msgr2=0x7f98f00ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:20:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98eed9d700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 0x7f98f00ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98eed9d700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 0x7f98f00ad400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98ee59c700 1 -- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f98f00ae050 msgr2=0x7f98f013f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98ee59c700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f98f00ae050 0x7f98f013f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98ee59c700 1 -- 172.21.15.44:0/3094923084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 msgr2=0x7f98f00ad400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98ee59c700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 0x7f98f00ad400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.792+0000 7f98ee59c700 1 -- 172.21.15.44:0/3094923084 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f013fd90 con 0x7f98f00669d0 2020-11-20T12:20:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.793+0000 7f98ee59c700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00669d0 0x7f98f0139b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f98e0012000 tx=0x7f98e0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.793+0000 7f98de7fc700 1 -- 172.21.15.44:0/3094923084 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0011dd0 con 0x7f98f00669d0 2020-11-20T12:20:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.793+0000 7f98de7fc700 1 -- 172.21.15.44:0/3094923084 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e001e460 con 0x7f98f00669d0 2020-11-20T12:20:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.793+0000 7f98de7fc700 1 -- 172.21.15.44:0/3094923084 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e001e800 con 0x7f98f00669d0 2020-11-20T12:20:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.793+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f013ffe0 con 0x7f98f00669d0 2020-11-20T12:20:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.793+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f00adb30 con 0x7f98f00669d0 2020-11-20T12:20:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.794+0000 7f98de7fc700 1 -- 172.21.15.44:0/3094923084 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f98e001a040 con 0x7f98f00669d0 2020-11-20T12:20:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.794+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f0135a80 con 0x7f98f00669d0 2020-11-20T12:20:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.794+0000 7f98de7fc700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f98c4056d50 0x7f98c4059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.794+0000 7f98de7fc700 1 -- 172.21.15.44:0/3094923084 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f98e00713b0 con 0x7f98f00669d0 2020-11-20T12:20:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.794+0000 7f98edd9b700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f98c4056d50 0x7f98c4059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.795+0000 7f98edd9b700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f98c4056d50 0x7f98c4059210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f98f00a9710 tx=0x7f98e4009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:30.798+0000 7f98de7fc700 1 -- 172.21.15.44:0/3094923084 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f98e0046710 con 0x7f98f00669d0 2020-11-20T12:20:31.087 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.084+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f98f00a50c0 con 0x7f98f00669d0 2020-11-20T12:20:31.088 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.084+0000 7f98de7fc700 1 -- 172.21.15.44:0/3094923084 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f98e0049d30 con 0x7f98f00669d0 2020-11-20T12:20:31.089 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":160,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:31.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f98c4056d50 msgr2=0x7f98c4059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.091 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f98c4056d50 0x7f98c4059210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f98f00a9710 tx=0x7f98e4009040).stop 2020-11-20T12:20:31.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00669d0 msgr2=0x7f98f0139b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00669d0 0x7f98f0139b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f98e0012000 tx=0x7f98e0015000).stop 2020-11-20T12:20:31.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 shutdown_connections 2020-11-20T12:20:31.092 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f98c4056d50 0x7f98c4059210 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98f013ebf0 0x7f98f00ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98f00669d0 0x7f98f0139b10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 --2- 172.21.15.44:0/3094923084 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f98f00ae050 0x7f98f013f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.093 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.087+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 >> 172.21.15.44:0/3094923084 conn(0x7f98f0062bc0 msgr2=0x7f98f005ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:31.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.088+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 shutdown_connections 2020-11-20T12:20:31.094 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:31.088+0000 7f98f49a1700 1 -- 172.21.15.44:0/3094923084 wait complete. 2020-11-20T12:20:34.106 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:34.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.256+0000 7ff9a1d1f700 1 Processor -- start 2020-11-20T12:20:34.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.256+0000 7ff9a1d1f700 1 -- start start 2020-11-20T12:20:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.257+0000 7ff9a1d1f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c0ab0e0 0x7ff99c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.257+0000 7ff9a1d1f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99c1385f0 con 0x7ff99c0aad00 2020-11-20T12:20:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.257+0000 7ff9a1d1f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99c0aca50 con 0x7ff99c0a86d0 2020-11-20T12:20:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.257+0000 7ff9a1d1f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c13e250 con 0x7ff99c0ab0e0 2020-11-20T12:20:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.257+0000 7ff99b7fe700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff99c0aad00 0x7ff99c0abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46842/0 (socket says 172.21.15.44:46842) 2020-11-20T12:20:34.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.257+0000 7ff99b7fe700 1 -- 172.21.15.44:0/1649241088 learned_addr learned my addr 172.21.15.44:0/1649241088 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:20:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.257+0000 7ff993fff700 1 --2- 172.21.15.44:0/1649241088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c0ab0e0 0x7ff99c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff99affd700 1 -- 172.21.15.44:0/1649241088 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955835641 0 0) 0x7ff99c0aca50 con 0x7ff99c0a86d0 2020-11-20T12:20:34.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff99affd700 1 -- 172.21.15.44:0/1649241088 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff97c003240 con 0x7ff99c0a86d0 2020-11-20T12:20:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff993fff700 1 -- 172.21.15.44:0/1649241088 >> v1:172.21.15.79:6789/0 conn(0x7ff99c0a86d0 legacy=0x7ff99c0ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff993fff700 1 -- 172.21.15.44:0/1649241088 >> v1:172.21.15.44:6789/0 conn(0x7ff99c0aad00 legacy=0x7ff99c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff993fff700 1 -- 172.21.15.44:0/1649241088 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c13e4f0 con 0x7ff99c0ab0e0 2020-11-20T12:20:34.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff993fff700 1 --2- 172.21.15.44:0/1649241088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c0ab0e0 0x7ff99c0a94d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff98400b9d0 tx=0x7ff984006df0).ready entity=mon.2 client_cookie=9780cf10b533f4dd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff99affd700 1 -- 172.21.15.44:0/1649241088 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984009840 con 0x7ff99c0ab0e0 2020-11-20T12:20:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff99affd700 1 -- 172.21.15.44:0/1649241088 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9840099a0 con 0x7ff99c0ab0e0 2020-11-20T12:20:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.258+0000 7ff99affd700 1 -- 172.21.15.44:0/1649241088 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984009d20 con 0x7ff99c0ab0e0 2020-11-20T12:20:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.259+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/1649241088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c0ab0e0 msgr2=0x7ff99c0a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.259+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/1649241088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c0ab0e0 0x7ff99c0a94d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff98400b9d0 tx=0x7ff984006df0).stop 2020-11-20T12:20:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.259+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/1649241088 shutdown_connections 2020-11-20T12:20:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.259+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/1649241088 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c0ab0e0 0x7ff99c0a94d0 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0x7ff98400b9d0 tx=0x7ff984006df0).stop 2020-11-20T12:20:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.259+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/1649241088 >> 172.21.15.44:0/1649241088 conn(0x7ff99c060170 msgr2=0x7ff99c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.259+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/1649241088 shutdown_connections 2020-11-20T12:20:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.259+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/1649241088 wait complete. 2020-11-20T12:20:34.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.260+0000 7ff9a1d1f700 1 Processor -- start 2020-11-20T12:20:34.270 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.260+0000 7ff9a1d1f700 1 -- start start 2020-11-20T12:20:34.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff9a1d1f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c0a86d0 0x7ff99c09def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff9a1d1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 0x7ff99c0a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.271 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff9a1d1f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c14d5c0 0x7ff99c1332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff9a1d1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0a5d40 con 0x7ff99c0aad00 2020-11-20T12:20:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff9a1d1f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0a5fe0 con 0x7ff99c0a86d0 2020-11-20T12:20:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff9a1d1f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0a6280 con 0x7ff99c14d5c0 2020-11-20T12:20:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff993fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 0x7ff99c0a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.261+0000 7ff99b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c0a86d0 0x7ff99c09def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff993fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 0x7ff99c0a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43102/0 (socket says 172.21.15.44:43102) 2020-11-20T12:20:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff993fff700 1 -- 172.21.15.44:0/2977034881 learned_addr learned my addr 172.21.15.44:0/2977034881 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c0a86d0 0x7ff99c09def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35150/0 (socket says 172.21.15.44:35150) 2020-11-20T12:20:34.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99bfff700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c14d5c0 0x7ff99c1332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff993fff700 1 -- 172.21.15.44:0/2977034881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 msgr2=0x7ff99c0a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:20:34.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff993fff700 1 -- 172.21.15.44:0/2977034881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 msgr2=0x7ff99c0a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:20:34.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff993fff700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 0x7ff99c0a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:34.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff993fff700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 0x7ff99c0a7080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:34.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99bfff700 1 -- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c0a86d0 msgr2=0x7ff99c09def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99bfff700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c0a86d0 0x7ff99c09def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99bfff700 1 -- 172.21.15.44:0/2977034881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 msgr2=0x7ff99c0a7080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:34.275 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99bfff700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 0x7ff99c0a7080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99bfff700 1 -- 172.21.15.44:0/2977034881 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0a6720 con 0x7ff99c14d5c0 2020-11-20T12:20:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99b7fe700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c0a86d0 0x7ff99c09def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff99bfff700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c14d5c0 0x7ff99c1332b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff99400ca60 tx=0x7ff994007d40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff9937fe700 1 -- 172.21.15.44:0/2977034881 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400a730 con 0x7ff99c14d5c0 2020-11-20T12:20:34.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff9937fe700 1 -- 172.21.15.44:0/2977034881 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99400a890 con 0x7ff99c14d5c0 2020-11-20T12:20:34.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c0a5a30 con 0x7ff99c14d5c0 2020-11-20T12:20:34.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff9937fe700 1 -- 172.21.15.44:0/2977034881 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400ac10 con 0x7ff99c14d5c0 2020-11-20T12:20:34.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.262+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c14da20 con 0x7ff99c14d5c0 2020-11-20T12:20:34.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.264+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c0a6440 con 0x7ff99c14d5c0 2020-11-20T12:20:34.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.264+0000 7ff9937fe700 1 -- 172.21.15.44:0/2977034881 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff994010070 con 0x7ff99c14d5c0 2020-11-20T12:20:34.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.264+0000 7ff9937fe700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff978056d20 0x7ff9780591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.264+0000 7ff9937fe700 1 -- 172.21.15.44:0/2977034881 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff99406fe00 con 0x7ff99c14d5c0 2020-11-20T12:20:34.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.264+0000 7ff99b7fe700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff978056d20 0x7ff9780591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.264+0000 7ff99b7fe700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff978056d20 0x7ff9780591e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7ff98c0015f0 tx=0x7ff98c000b80).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.268+0000 7ff9937fe700 1 -- 172.21.15.44:0/2977034881 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff9940450d0 con 0x7ff99c14d5c0 2020-11-20T12:20:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.594+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff99c069680 con 0x7ff99c14d5c0 2020-11-20T12:20:34.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.595+0000 7ff9937fe700 1 -- 172.21.15.44:0/2977034881 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7ff9940486f0 con 0x7ff99c14d5c0 2020-11-20T12:20:34.598 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":160,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:34.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff978056d20 msgr2=0x7ff9780591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.600 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff978056d20 0x7ff9780591e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7ff98c0015f0 tx=0x7ff98c000b80).stop 2020-11-20T12:20:34.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c14d5c0 msgr2=0x7ff99c1332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c14d5c0 0x7ff99c1332b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff99400ca60 tx=0x7ff994007d40).stop 2020-11-20T12:20:34.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 shutdown_connections 2020-11-20T12:20:34.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff978056d20 0x7ff9780591e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff99c14d5c0 0x7ff99c1332b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff99c0a86d0 0x7ff99c09def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 --2- 172.21.15.44:0/2977034881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff99c0aad00 0x7ff99c0a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.597+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 >> 172.21.15.44:0/2977034881 conn(0x7ff99c060170 msgr2=0x7ff99c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.598+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 shutdown_connections 2020-11-20T12:20:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:34.598+0000 7ff9a1d1f700 1 -- 172.21.15.44:0/2977034881 wait complete. 2020-11-20T12:20:37.620 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:37.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.784+0000 7f25f3d91700 1 Processor -- start 2020-11-20T12:20:37.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.784+0000 7f25f3d91700 1 -- start start 2020-11-20T12:20:37.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f3d91700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f3d91700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f3d91700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec0a8b10 con 0x7f25ec0b1970 2020-11-20T12:20:37.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f3d91700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec0aae30 con 0x7f25ec0a86a0 2020-11-20T12:20:37.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f3d91700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec0aaa00 con 0x7f25ec0b1d50 2020-11-20T12:20:37.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f232e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f132c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f232e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43116/0 (socket says 172.21.15.44:43116) 2020-11-20T12:20:37.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f1b2d700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f25ec0b1970 0x7f25ec13b1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46860/0 (socket says 172.21.15.44:46860) 2020-11-20T12:20:37.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 learned_addr learned my addr 172.21.15.44:0/3563921503 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:37.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:37.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:20:37.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:37.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:37.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:37.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:37.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:37.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.785+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:37.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:37.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:37.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:37.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:20:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:37.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:37.986+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:38.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:38.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:20:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:38.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:38.393 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:38.387+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:39.090 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:20:39.093 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:20:39.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:20:39.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:39.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:39.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:20:39.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:39.193 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f132c700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:39.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:39.194 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:39.188+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:20:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.749+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> v1:172.21.15.44:6789/0 conn(0x7f25ec0b1970 legacy=0x7f25ec13b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2020-11-20T12:20:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.749+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> v1:172.21.15.44:6789/0 conn(0x7f25ec0b1970 legacy=0x7f25ec13b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:20:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.749+0000 7f25f1b2d700 1 --1- 172.21.15.44:0/3563921503 >> v1:172.21.15.44:6789/0 conn(0x7f25ec0b1970 0x7f25ec13b1e0 :-1 s=OPENED pgs=26 cs=1 l=1).handle_message read tag failed 2020-11-20T12:20:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.749+0000 7f25f1b2d700 1 --1- 172.21.15.44:0/3563921503 >> v1:172.21.15.44:6789/0 conn(0x7f25ec0b1970 0x7f25ec13b1e0 :-1 s=OPENED pgs=26 cs=1 l=1).fault on lossy channel, failing 2020-11-20T12:20:40.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 msgr2=0x7f25ec13ab60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:40.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.79:3300/0 conn(0x7f25ec0b1d50 0x7f25ec13ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 -- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 msgr2=0x7f25ec0ab220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:40.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 --2- 172.21.15.44:0/3563921503 >> v2:172.21.15.44:3300/0 conn(0x7f25ec0a86a0 0x7f25ec0ab220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 -- 172.21.15.44:0/3563921503 >> v1:172.21.15.44:6789/0 conn(0x7f25ec0b1970 legacy=0x7f25ec13b1e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:40.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 -- 172.21.15.44:0/3563921503 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec0a9610 con 0x7f25cc005d20 2020-11-20T12:20:40.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25eb7fe700 1 -- 172.21.15.44:0/3563921503 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec0aaa00 con 0x7f25cc0032c0 2020-11-20T12:20:40.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.785+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.786+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 msgr2=0x7f25cc005710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.786+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 msgr2=0x7f25cc005710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.786+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:40.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.786+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:40.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.987+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.987+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 msgr2=0x7f25cc005710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:40.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.987+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 msgr2=0x7f25cc005710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:40.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.987+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:40.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:40.987+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:41.391 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.388+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.388+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 msgr2=0x7f25cc005710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:41.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.388+0000 7f25f1b2d700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 msgr2=0x7f25cc005710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:41.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.388+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:41.392 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.388+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275081399 0 0) 0x7f25ec0a9610 con 0x7f25cc005d20 2020-11-20T12:20:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d8003240 con 0x7f25cc005d20 2020-11-20T12:20:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3930356183 0 0) 0x7f25d8003240 con 0x7f25cc005d20 2020-11-20T12:20:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 msgr2=0x7f25cc005710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25cc0032c0 0x7f25cc005710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc00abd0 con 0x7f25cc005d20 2020-11-20T12:20:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f25e000d1c0 con 0x7f25cc005d20 2020-11-20T12:20:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 >> v1:172.21.15.79:6789/0 conn(0x7f25cc005d20 legacy=0x7f25cc0081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 reap_dead start 2020-11-20T12:20:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25d8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d80044e0 0x7f25d8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d8004e50 0x7f25d800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec0aaa00 con 0x7f25d8004e50 2020-11-20T12:20:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8004290 con 0x7f25d8003730 2020-11-20T12:20:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d800bc20 con 0x7f25d80044e0 2020-11-20T12:20:41.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d8004e50 0x7f25d800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.983+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25d8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f132c700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d80044e0 0x7f25d8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d80044e0 msgr2=0x7f25d8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:41.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d80044e0 0x7f25d8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 msgr2=0x7f25d8003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:41.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25d8003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f232e700 1 -- 172.21.15.44:0/3563921503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8003240 con 0x7f25d8004e50 2020-11-20T12:20:41.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25d8003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:41.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.984+0000 7f25f232e700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d8004e50 0x7f25d800b730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f25ec0a5020 tx=0x7f25e4008040).ready entity=mon.0 client_cookie=ed6ef174e7a47081 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:41.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.986+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40149c0 con 0x7f25d8004e50 2020-11-20T12:20:41.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.986+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e400c040 con 0x7f25d8004e50 2020-11-20T12:20:41.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.986+0000 7f25f0b2b700 1 -- 172.21.15.44:0/3563921503 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4014d30 con 0x7f25d8004e50 2020-11-20T12:20:41.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 -- 172.21.15.44:0/3563921503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d8004e50 msgr2=0x7f25d800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d8004e50 0x7f25d800b730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f25ec0a5020 tx=0x7f25e4008040).stop 2020-11-20T12:20:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 -- 172.21.15.44:0/3563921503 shutdown_connections 2020-11-20T12:20:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d8004e50 0x7f25d800b730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d80044e0 0x7f25d8004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 --2- 172.21.15.44:0/3563921503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25d8003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 -- 172.21.15.44:0/3563921503 >> 172.21.15.44:0/3563921503 conn(0x7f25ec060c40 msgr2=0x7f25ec0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:41.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 -- 172.21.15.44:0/3563921503 shutdown_connections 2020-11-20T12:20:41.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.987+0000 7f25f3d91700 1 -- 172.21.15.44:0/3563921503 wait complete. 2020-11-20T12:20:41.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.988+0000 7f25f3d91700 1 Processor -- start 2020-11-20T12:20:41.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.989+0000 7f25f3d91700 1 -- start start 2020-11-20T12:20:41.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.989+0000 7f25f3d91700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25ec140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.989+0000 7f25f3d91700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d80044e0 0x7f25ec09e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.989+0000 7f25f3d91700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 0x7f25ec149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.989+0000 7f25f3d91700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec140150 con 0x7f25d80044e0 2020-11-20T12:20:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.989+0000 7f25f3d91700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec14a180 con 0x7f25d8003730 2020-11-20T12:20:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.989+0000 7f25f3d91700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec13bcc0 con 0x7f25d8004e50 2020-11-20T12:20:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d80044e0 0x7f25ec09e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d80044e0 0x7f25ec09e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43146/0 (socket says 172.21.15.44:43146) 2020-11-20T12:20:42.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 -- 172.21.15.44:0/2615725809 learned_addr learned my addr 172.21.15.44:0/2615725809 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:42.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25ec140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f232e700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 0x7f25ec149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f232e700 1 -- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 msgr2=0x7f25ec149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2020-11-20T12:20:42.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f232e700 1 -- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 msgr2=0x7f25ec149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:20:42.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f232e700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 0x7f25ec149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:42.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f232e700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 0x7f25ec149c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:42.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 -- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 msgr2=0x7f25ec149c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:42.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 0x7f25ec149c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 -- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 msgr2=0x7f25ec140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25ec140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 -- 172.21.15.44:0/2615725809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec140380 con 0x7f25d80044e0 2020-11-20T12:20:42.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25ec140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:42.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f132c700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d80044e0 0x7f25ec09e7f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f25e0016d50 tx=0x7f25e0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:42.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25e9ffb700 1 -- 172.21.15.44:0/2615725809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0006920 con 0x7f25d80044e0 2020-11-20T12:20:42.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25e9ffb700 1 -- 172.21.15.44:0/2615725809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e0014d50 con 0x7f25d80044e0 2020-11-20T12:20:42.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25e9ffb700 1 -- 172.21.15.44:0/2615725809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0017440 con 0x7f25d80044e0 2020-11-20T12:20:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.990+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec140620 con 0x7f25d80044e0 2020-11-20T12:20:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.991+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec141780 con 0x7f25d80044e0 2020-11-20T12:20:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.992+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec0a4a00 con 0x7f25d80044e0 2020-11-20T12:20:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.993+0000 7f25e9ffb700 1 -- 172.21.15.44:0/2615725809 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f25e0027070 con 0x7f25d80044e0 2020-11-20T12:20:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.993+0000 7f25e9ffb700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f25c4056d20 0x7f25c40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.993+0000 7f25e9ffb700 1 -- 172.21.15.44:0/2615725809 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f25e00714b0 con 0x7f25d80044e0 2020-11-20T12:20:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.997+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f25c4056d20 0x7f25c40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.997+0000 7f25e9ffb700 1 -- 172.21.15.44:0/2615725809 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f25e0046800 con 0x7f25d80044e0 2020-11-20T12:20:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:41.997+0000 7f25f1b2d700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f25c4056d20 0x7f25c40591e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f25dc007fd0 tx=0x7f25dc009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:42.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.307+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f25ec0ad5b0 con 0x7f25d80044e0 2020-11-20T12:20:42.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.308+0000 7f25e9ffb700 1 -- 172.21.15.44:0/2615725809 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f25e0049e20 con 0x7f25d80044e0 2020-11-20T12:20:42.311 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":164,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f25c4056d20 msgr2=0x7f25c40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f25c4056d20 0x7f25c40591e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f25dc007fd0 tx=0x7f25dc009040).stop 2020-11-20T12:20:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d80044e0 msgr2=0x7f25ec09e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d80044e0 0x7f25ec09e7f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f25e0016d50 tx=0x7f25e0008040).stop 2020-11-20T12:20:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 shutdown_connections 2020-11-20T12:20:42.314 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f25c4056d20 0x7f25c40591e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f25d8004e50 0x7f25ec149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25d8003730 0x7f25ec140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 --2- 172.21.15.44:0/2615725809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25d80044e0 0x7f25ec09e7f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 >> 172.21.15.44:0/2615725809 conn(0x7f25ec060c40 msgr2=0x7f25ec13c310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 shutdown_connections 2020-11-20T12:20:42.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:42.310+0000 7f25f3d91700 1 -- 172.21.15.44:0/2615725809 wait complete. 2020-11-20T12:20:45.326 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:45.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.511+0000 7fc4a85ac700 1 Processor -- start 2020-11-20T12:20:45.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.511+0000 7fc4a85ac700 1 -- start start 2020-11-20T12:20:45.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.512+0000 7fc4a85ac700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:45.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.512+0000 7fc4a85ac700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:45.516 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a6348700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a85ac700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a00a9920 con 0x7fc4a013f500 2020-11-20T12:20:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a85ac700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a00aa250 con 0x7fc4a00ae2a0 2020-11-20T12:20:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a85ac700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a0060730 con 0x7fc4a00adec0 2020-11-20T12:20:45.517 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a6348700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54592/0 (socket says 172.21.15.44:54592) 2020-11-20T12:20:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 learned_addr learned my addr 172.21.15.44:0/4193464901 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:45.518 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:45.519 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.513+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:45.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:45.717 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:45.718 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:45.719 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:45.714+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:46.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:46.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.115+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:46.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:46.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a6348700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:46.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:46.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:20:46.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:20:46.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a5b47700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:46.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:46.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:46.916+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:20:47.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.993+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000476420 0 0) 0x7fc4a00a9920 con 0x7fc4a013f500 2020-11-20T12:20:47.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.993+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc488003240 con 0x7fc4a013f500 2020-11-20T12:20:47.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2352491200 0 0) 0x7fc488003240 con 0x7fc4a013f500 2020-11-20T12:20:47.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4a00a9010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:47.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4a00a9010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 msgr2=0x7fc4a013f010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 --2- 172.21.15.44:0/4193464901 >> v2:172.21.15.79:3300/0 conn(0x7fc4a00ae2a0 0x7fc4a013f010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a00609d0 con 0x7fc4a013f500 2020-11-20T12:20:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fc49c0014e0 con 0x7fc4a013f500 2020-11-20T12:20:47.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 >> v1:172.21.15.44:6789/0 conn(0x7fc4a013f500 legacy=0x7fc4a013f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4880037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc488003d10 0x7fc4880044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4880049d0 0x7fc488004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a00aa250 con 0x7fc4880049d0 2020-11-20T12:20:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a0060730 con 0x7fc488003d10 2020-11-20T12:20:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880042d0 con 0x7fc4a00adec0 2020-11-20T12:20:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a6348700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4880037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a6b49700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4880049d0 0x7fc488004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.994+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc488003d10 0x7fc4880044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.995+0000 7fc4a6b49700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 msgr2=0x7fc4880037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.995+0000 7fc4a6b49700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4a00adec0 0x7fc4880037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.995+0000 7fc4a6b49700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc488003d10 msgr2=0x7fc4880044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.995+0000 7fc4a6b49700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc488003d10 0x7fc4880044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.995+0000 7fc4a6b49700 1 -- 172.21.15.44:0/4193464901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488003240 con 0x7fc4880049d0 2020-11-20T12:20:48.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.995+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc488003d10 0x7fc4880044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:48.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.995+0000 7fc4a6b49700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4880049d0 0x7fc488004e50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc49c00c190 tx=0x7fc4a0060b60).ready entity=mon.0 client_cookie=b02d39fc91c1ba64 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:48.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.997+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00cdd0 con 0x7fc4880049d0 2020-11-20T12:20:48.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.997+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c012400 con 0x7fc4880049d0 2020-11-20T12:20:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.997+0000 7fc4a5346700 1 -- 172.21.15.44:0/4193464901 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0127c0 con 0x7fc4880049d0 2020-11-20T12:20:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.998+0000 7fc4a85ac700 1 -- 172.21.15.44:0/4193464901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4880049d0 msgr2=0x7fc488004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.998+0000 7fc4a85ac700 1 --2- 172.21.15.44:0/4193464901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4880049d0 0x7fc488004e50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc49c00c190 tx=0x7fc4a0060b60).stop 2020-11-20T12:20:48.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.998+0000 7fc4a6b49700 1 -- 172.21.15.44:0/4193464901 reap_dead start 2020-11-20T12:20:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.998+0000 7fc4a85ac700 1 -- 172.21.15.44:0/4193464901 shutdown_connections 2020-11-20T12:20:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.998+0000 7fc4a85ac700 1 -- 172.21.15.44:0/4193464901 >> 172.21.15.44:0/4193464901 conn(0x7fc4a00658a0 msgr2=0x7fc4a0062bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.998+0000 7fc4a85ac700 1 -- 172.21.15.44:0/4193464901 shutdown_connections 2020-11-20T12:20:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.998+0000 7fc4a85ac700 1 -- 172.21.15.44:0/4193464901 wait complete. 2020-11-20T12:20:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.999+0000 7fc4a85ac700 1 Processor -- start 2020-11-20T12:20:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:47.999+0000 7fc4a85ac700 1 -- start start 2020-11-20T12:20:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a85ac700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 0x7fc4a0143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a85ac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a00a61c0 0x7fc4a00a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a85ac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4a0145e40 0x7fc4a0151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a85ac700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01462c0 con 0x7fc4a00a61c0 2020-11-20T12:20:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a85ac700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a00a5800 con 0x7fc4a0145e40 2020-11-20T12:20:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a85ac700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a00a8050 con 0x7fc4880049d0 2020-11-20T12:20:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a5b47700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a00a61c0 0x7fc4a00a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a5b47700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a00a61c0 0x7fc4a00a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43194/0 (socket says 172.21.15.44:43194) 2020-11-20T12:20:48.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a5b47700 1 -- 172.21.15.44:0/643360949 learned_addr learned my addr 172.21.15.44:0/643360949 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.000+0000 7fc4a6348700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 0x7fc4a0143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a6b49700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4a0145e40 0x7fc4a0151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a6348700 1 -- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 msgr2=0x7fc4a0143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:20:48.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a6348700 1 -- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 msgr2=0x7fc4a0143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:20:48.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a6348700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 0x7fc4a0143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:48.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a6348700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 0x7fc4a0143600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:48.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a5b47700 1 -- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 msgr2=0x7fc4a0143600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:48.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 0x7fc4a0143600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a5b47700 1 -- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4a0145e40 msgr2=0x7fc4a0151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4a0145e40 0x7fc4a0151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a5b47700 1 -- 172.21.15.44:0/643360949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a00a5d20 con 0x7fc4a00a61c0 2020-11-20T12:20:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a5b47700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a00a61c0 0x7fc4a00a79b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc494015ae0 tx=0x7fc494008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4927fc700 1 -- 172.21.15.44:0/643360949 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494011a50 con 0x7fc4a00a61c0 2020-11-20T12:20:48.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4927fc700 1 -- 172.21.15.44:0/643360949 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc494011bb0 con 0x7fc4a00a61c0 2020-11-20T12:20:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a01450c0 con 0x7fc4a00a61c0 2020-11-20T12:20:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4927fc700 1 -- 172.21.15.44:0/643360949 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494017440 con 0x7fc4a00a61c0 2020-11-20T12:20:48.013 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.001+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a0152010 con 0x7fc4a00a61c0 2020-11-20T12:20:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.003+0000 7fc4927fc700 1 -- 172.21.15.44:0/643360949 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc494026060 con 0x7fc4a00a61c0 2020-11-20T12:20:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.003+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a00a7ea0 con 0x7fc4a00a61c0 2020-11-20T12:20:48.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.003+0000 7fc4927fc700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc478056d20 0x7fc4780591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.003+0000 7fc4927fc700 1 -- 172.21.15.44:0/643360949 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc494070b00 con 0x7fc4a00a61c0 2020-11-20T12:20:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.003+0000 7fc4a6348700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc478056d20 0x7fc4780591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.004+0000 7fc4a6348700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc478056d20 0x7fc4780591e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fc498006ed0 tx=0x7fc498008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.009+0000 7fc4927fc700 1 -- 172.21.15.44:0/643360949 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc494045c60 con 0x7fc4a00a61c0 2020-11-20T12:20:48.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.409+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc4a00a8210 con 0x7fc4a00a61c0 2020-11-20T12:20:49.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.410+0000 7fc4927fc700 1 -- 172.21.15.44:0/643360949 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fc494024020 con 0x7fc4a00a61c0 2020-11-20T12:20:49.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.414+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc478056d20 msgr2=0x7fc4780591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.414+0000 7fc4a85ac700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc478056d20 0x7fc4780591e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fc498006ed0 tx=0x7fc498008040).stop 2020-11-20T12:20:49.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.414+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a00a61c0 msgr2=0x7fc4a00a79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.414+0000 7fc4a85ac700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a00a61c0 0x7fc4a00a79b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc494015ae0 tx=0x7fc494008040).stop 2020-11-20T12:20:49.804 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 shutdown_connections 2020-11-20T12:20:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc478056d20 0x7fc4780591e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4a0145e40 0x7fc4a0151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc4880049d0 0x7fc4a0143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 --2- 172.21.15.44:0/643360949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4a00a61c0 0x7fc4a00a79b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 >> 172.21.15.44:0/643360949 conn(0x7fc4a00658a0 msgr2=0x7fc4a00ac480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 shutdown_connections 2020-11-20T12:20:49.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:48.415+0000 7fc4a85ac700 1 -- 172.21.15.44:0/643360949 wait complete. 2020-11-20T12:20:49.807 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":168,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:52.808 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.001+0000 7f56b9f6a700 1 Processor -- start 2020-11-20T12:20:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.001+0000 7f56b9f6a700 1 -- start start 2020-11-20T12:20:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b9f6a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b9f6a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b40a4f70 con 0x7f56b40a9460 2020-11-20T12:20:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b9f6a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b40ab310 con 0x7f56b40a4af0 2020-11-20T12:20:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b9f6a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b413ac80 con 0x7f56b413b780 2020-11-20T12:20:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b2ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b2ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35256/0 (socket says 172.21.15.44:35256) 2020-11-20T12:20:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1049050108 learned_addr learned my addr 172.21.15.44:0/1049050108 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:53.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b37fe700 1 --1- 172.21.15.44:0/1049050108 >> v1:172.21.15.44:6789/0 conn(0x7f56b40a9460 0x7f56b413b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46956/0 (socket says 172.21.15.44:46956) 2020-11-20T12:20:53.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 msgr2=0x7f56b40aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:20:53.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 msgr2=0x7f56b40aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:53.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:53.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.002+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:53.207 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.203+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.203+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 msgr2=0x7f56b40aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:20:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.203+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 msgr2=0x7f56b40aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:53.885 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.203+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:53.886 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.203+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:20:53.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.604+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:53.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.604+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 msgr2=0x7f56b40aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:20:53.887 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.604+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 msgr2=0x7f56b40aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:20:53.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.604+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:53.888 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.604+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:20:53.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126358112 0 0) 0x7f56b40ab310 con 0x7f56b40a4af0 2020-11-20T12:20:53.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a0003240 con 0x7f56b40a4af0 2020-11-20T12:20:53.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 <== mon.1 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3678761293 0 0) 0x7f56a0003240 con 0x7f56b40a4af0 2020-11-20T12:20:53.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 msgr2=0x7f56b40aad20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:53.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 --2- 172.21.15.44:0/1049050108 >> v2:172.21.15.79:3300/0 conn(0x7f56b413b780 0x7f56b40aad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:53.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 >> v1:172.21.15.44:6789/0 conn(0x7f56b40a9460 legacy=0x7f56b413b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:53.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40ab5b0 con 0x7f56b40a4af0 2020-11-20T12:20:53.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 <== mon.1 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7f56a40014e0 con 0x7f56b40a4af0 2020-11-20T12:20:53.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 >> v1:172.21.15.79:6789/0 conn(0x7f56b40a4af0 legacy=0x7f56b4137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a0003730 0x7f56a0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a00044e0 0x7f56a0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a0004e50 0x7f56a000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a0004260 con 0x7f56a0003730 2020-11-20T12:20:53.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a000bc20 con 0x7f56a0004e50 2020-11-20T12:20:53.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a000bd60 con 0x7f56a00044e0 2020-11-20T12:20:53.999 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a0003730 0x7f56a0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a00044e0 0x7f56a0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.992+0000 7f56b3fff700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a0004e50 0x7f56a000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b37fe700 1 -- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a00044e0 msgr2=0x7f56a0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a00044e0 0x7f56a0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b37fe700 1 -- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a0004e50 msgr2=0x7f56a000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a0004e50 0x7f56a000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b37fe700 1 -- 172.21.15.44:0/1049050108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0003240 con 0x7f56a0003730 2020-11-20T12:20:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a00044e0 0x7f56a0004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b3fff700 1 -- 172.21.15.44:0/1049050108 reap_dead start 2020-11-20T12:20:54.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a0003730 0x7f56a0003b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f56ac001e80 tx=0x7f56ac007f00).ready entity=mon.0 client_cookie=71381464eca75093 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.993+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac005ac0 con 0x7f56a0003730 2020-11-20T12:20:54.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.995+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac005c20 con 0x7f56a0003730 2020-11-20T12:20:54.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.995+0000 7f56b27fc700 1 -- 172.21.15.44:0/1049050108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0070d0 con 0x7f56a0003730 2020-11-20T12:20:54.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.996+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1049050108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a0003730 msgr2=0x7f56a0003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.996+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a0003730 0x7f56a0003b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f56ac001e80 tx=0x7f56ac007f00).stop 2020-11-20T12:20:54.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.996+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1049050108 shutdown_connections 2020-11-20T12:20:54.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.996+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1049050108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56a0003730 0x7f56a0003b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.996+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1049050108 >> 172.21.15.44:0/1049050108 conn(0x7f56b40621c0 msgr2=0x7f56b4060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:54.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.996+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1049050108 shutdown_connections 2020-11-20T12:20:54.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.996+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1049050108 wait complete. 2020-11-20T12:20:54.005 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.998+0000 7f56b9f6a700 1 Processor -- start 2020-11-20T12:20:54.006 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.998+0000 7f56b9f6a700 1 -- start start 2020-11-20T12:20:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b9f6a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 0x7f56b4139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b9f6a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b4151e90 0x7f56b4152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b9f6a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56b4130ff0 0x7f56b4131430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b9f6a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4152800 con 0x7f56b4151e90 2020-11-20T12:20:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b9f6a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40a76a0 con 0x7f56b4130ff0 2020-11-20T12:20:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b9f6a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40657e0 con 0x7f56a0003730 2020-11-20T12:20:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b4151e90 0x7f56b4152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b4151e90 0x7f56b4152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43226/0 (socket says 172.21.15.44:43226) 2020-11-20T12:20:54.008 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1890995261 learned_addr learned my addr 172.21.15.44:0/1890995261 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:54.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b3fff700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56b4130ff0 0x7f56b4131430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 0x7f56b4139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b37fe700 1 -- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 msgr2=0x7f56b4139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2020-11-20T12:20:54.009 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b37fe700 1 -- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 msgr2=0x7f56b4139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:20:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 0x7f56b4139d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:20:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:53.999+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 0x7f56b4139d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:20:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.001+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 msgr2=0x7f56b4139d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:20:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.001+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 0x7f56b4139d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.001+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56b4130ff0 msgr2=0x7f56b4131430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.001+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56b4130ff0 0x7f56b4131430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.001+0000 7f56b2ffd700 1 -- 172.21.15.44:0/1890995261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4065a80 con 0x7f56b4151e90 2020-11-20T12:20:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.001+0000 7f56b2ffd700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b4151e90 0x7f56b4152310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f56a8014d00 tx=0x7f56a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.002+0000 7f56937fe700 1 -- 172.21.15.44:0/1890995261 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8012b20 con 0x7f56b4151e90 2020-11-20T12:20:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.002+0000 7f56937fe700 1 -- 172.21.15.44:0/1890995261 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a8012c80 con 0x7f56b4151e90 2020-11-20T12:20:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.002+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b40654d0 con 0x7f56b4151e90 2020-11-20T12:20:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.002+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b413a5e0 con 0x7f56b4151e90 2020-11-20T12:20:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.002+0000 7f56937fe700 1 -- 172.21.15.44:0/1890995261 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8016430 con 0x7f56b4151e90 2020-11-20T12:20:54.012 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.003+0000 7f56937fe700 1 -- 172.21.15.44:0/1890995261 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f56a8012040 con 0x7f56b4151e90 2020-11-20T12:20:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.004+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b4060880 con 0x7f56b4151e90 2020-11-20T12:20:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.004+0000 7f56937fe700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f568c056d70 0x7f568c059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.004+0000 7f56937fe700 1 -- 172.21.15.44:0/1890995261 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f56a8071080 con 0x7f56b4151e90 2020-11-20T12:20:54.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.005+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f568c056d70 0x7f568c059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.015 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.005+0000 7f56b37fe700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f568c056d70 0x7f568c059230 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f56ac002210 tx=0x7f56ac015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.016 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.012+0000 7f56937fe700 1 -- 172.21.15.44:0/1890995261 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f56a8046260 con 0x7f56b4151e90 2020-11-20T12:20:54.446 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.443+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f56b40a9840 con 0x7f56b4151e90 2020-11-20T12:20:54.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.443+0000 7f56937fe700 1 -- 172.21.15.44:0/1890995261 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f56a8049880 con 0x7f56b4151e90 2020-11-20T12:20:54.447 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":172,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:54.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f568c056d70 msgr2=0x7f568c059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f568c056d70 0x7f568c059230 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f56ac002210 tx=0x7f56ac015000).stop 2020-11-20T12:20:54.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b4151e90 msgr2=0x7f56b4152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b4151e90 0x7f56b4152310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f56a8014d00 tx=0x7f56a8008040).stop 2020-11-20T12:20:54.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 shutdown_connections 2020-11-20T12:20:54.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f568c056d70 0x7f568c059230 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56b4130ff0 0x7f56b4131430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f56a0003730 0x7f56b4139d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 --2- 172.21.15.44:0/1890995261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b4151e90 0x7f56b4152310 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0x7f56a8014d00 tx=0x7f56a8008040).stop 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.446+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 >> 172.21.15.44:0/1890995261 conn(0x7f56b40621c0 msgr2=0x7f56b4060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:54.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.447+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 shutdown_connections 2020-11-20T12:20:54.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:54.447+0000 7f56b9f6a700 1 -- 172.21.15.44:0/1890995261 wait complete. 2020-11-20T12:20:57.469 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:20:57.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.643+0000 7f322505a700 1 Processor -- start 2020-11-20T12:20:57.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.643+0000 7f322505a700 1 -- start start 2020-11-20T12:20:57.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f322505a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f32200abc70 0x7f32200671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f322505a700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3220067dc0 0x7f32200a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f322505a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32200a8cb0 con 0x7f32200a72d0 2020-11-20T12:20:57.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f322505a700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200ac550 con 0x7f3220067dc0 2020-11-20T12:20:57.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f322505a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200ab340 con 0x7f32200abc70 2020-11-20T12:20:57.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f321f59e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3220067dc0 0x7f32200a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f3217fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f32200a72d0 0x7f32200676b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46982/0 (socket says 172.21.15.44:46982) 2020-11-20T12:20:57.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f3217fff700 1 -- 172.21.15.44:0/817048743 learned_addr learned my addr 172.21.15.44:0/817048743 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:20:57.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f321ed9d700 1 --2- 172.21.15.44:0/817048743 >> v2:172.21.15.79:3300/0 conn(0x7f32200abc70 0x7f32200671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.644+0000 7f321e59c700 1 -- 172.21.15.44:0/817048743 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386916876 0 0) 0x7f32200a8cb0 con 0x7f32200a72d0 2020-11-20T12:20:57.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321e59c700 1 -- 172.21.15.44:0/817048743 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3200003240 con 0x7f32200a72d0 2020-11-20T12:20:57.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321f59e700 1 -- 172.21.15.44:0/817048743 >> v2:172.21.15.79:3300/0 conn(0x7f32200abc70 msgr2=0x7f32200671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:57.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321f59e700 1 --2- 172.21.15.44:0/817048743 >> v2:172.21.15.79:3300/0 conn(0x7f32200abc70 0x7f32200671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321f59e700 1 -- 172.21.15.44:0/817048743 >> v1:172.21.15.44:6789/0 conn(0x7f32200a72d0 legacy=0x7f32200676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321f59e700 1 -- 172.21.15.44:0/817048743 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322013c650 con 0x7f3220067dc0 2020-11-20T12:20:57.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321f59e700 1 --2- 172.21.15.44:0/817048743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f32200a87c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f322005c330 tx=0x7f32180091d0).ready entity=mon.0 client_cookie=a9a9ae3c5f380e36 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321e59c700 1 -- 172.21.15.44:0/817048743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321800ba20 con 0x7f3220067dc0 2020-11-20T12:20:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321e59c700 1 -- 172.21.15.44:0/817048743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321800bb80 con 0x7f3220067dc0 2020-11-20T12:20:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f321e59c700 1 -- 172.21.15.44:0/817048743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011400 con 0x7f3220067dc0 2020-11-20T12:20:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f322505a700 1 -- 172.21.15.44:0/817048743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 msgr2=0x7f32200a87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f322505a700 1 --2- 172.21.15.44:0/817048743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f32200a87c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f322005c330 tx=0x7f32180091d0).stop 2020-11-20T12:20:57.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f322505a700 1 -- 172.21.15.44:0/817048743 shutdown_connections 2020-11-20T12:20:57.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f322505a700 1 --2- 172.21.15.44:0/817048743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f32200a87c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.645+0000 7f322505a700 1 --2- 172.21.15.44:0/817048743 >> v2:172.21.15.79:3300/0 conn(0x7f32200abc70 0x7f32200671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.646+0000 7f322505a700 1 -- 172.21.15.44:0/817048743 >> 172.21.15.44:0/817048743 conn(0x7f3220060170 msgr2=0x7f3220060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:57.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.646+0000 7f322505a700 1 -- 172.21.15.44:0/817048743 shutdown_connections 2020-11-20T12:20:57.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.646+0000 7f322505a700 1 -- 172.21.15.44:0/817048743 wait complete. 2020-11-20T12:20:57.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.646+0000 7f322505a700 1 Processor -- start 2020-11-20T12:20:57.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f322505a700 1 -- start start 2020-11-20T12:20:57.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f322505a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f3220116cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f322505a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32200a72d0 0x7f32200a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f322505a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f32200abc70 0x7f32200a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f322505a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200a4260 con 0x7f3220067dc0 2020-11-20T12:20:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f322505a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32201171c0 con 0x7f32200a72d0 2020-11-20T12:20:57.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f322505a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3220063430 con 0x7f32200abc70 2020-11-20T12:20:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f321ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f3220116cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f321ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f3220116cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43238/0 (socket says 172.21.15.44:43238) 2020-11-20T12:20:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.647+0000 7f321ed9d700 1 -- 172.21.15.44:0/316772478 learned_addr learned my addr 172.21.15.44:0/316772478 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:57.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f321f59e700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f32200abc70 0x7f32200a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f3217fff700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32200a72d0 0x7f32200a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f321ed9d700 1 -- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f32200abc70 msgr2=0x7f32200a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f321ed9d700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f32200abc70 0x7f32200a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f321ed9d700 1 -- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32200a72d0 msgr2=0x7f32200a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f321ed9d700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32200a72d0 0x7f32200a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f321ed9d700 1 -- 172.21.15.44:0/316772478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200636d0 con 0x7f3220067dc0 2020-11-20T12:20:57.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f321ed9d700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f3220116cd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f321000eba0 tx=0x7f3210008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f3216ffd700 1 -- 172.21.15.44:0/316772478 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000c820 con 0x7f3220067dc0 2020-11-20T12:20:57.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f3216ffd700 1 -- 172.21.15.44:0/316772478 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321000c980 con 0x7f3220067dc0 2020-11-20T12:20:57.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3220063970 con 0x7f3220067dc0 2020-11-20T12:20:57.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f3216ffd700 1 -- 172.21.15.44:0/316772478 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000cd00 con 0x7f3220067dc0 2020-11-20T12:20:57.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.648+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32200a1d70 con 0x7f3220067dc0 2020-11-20T12:20:57.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.650+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3220061650 con 0x7f3220067dc0 2020-11-20T12:20:57.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.650+0000 7f3216ffd700 1 -- 172.21.15.44:0/316772478 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f3210022070 con 0x7f3220067dc0 2020-11-20T12:20:57.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.650+0000 7f3216ffd700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f31fc056d20 0x7f31fc0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.650+0000 7f3216ffd700 1 -- 172.21.15.44:0/316772478 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f3210070350 con 0x7f3220067dc0 2020-11-20T12:20:57.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.651+0000 7f3217fff700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f31fc056d20 0x7f31fc0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.651+0000 7f3217fff700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f31fc056d20 0x7f31fc0591e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f32200aa9e0 tx=0x7f3208009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.656+0000 7f3216ffd700 1 -- 172.21.15.44:0/316772478 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3210045620 con 0x7f3220067dc0 2020-11-20T12:20:57.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.974+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3220065590 con 0x7f3220067dc0 2020-11-20T12:20:57.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.974+0000 7f3216ffd700 1 -- 172.21.15.44:0/316772478 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f3210016020 con 0x7f3220067dc0 2020-11-20T12:20:57.978 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:20:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f31fc056d20 msgr2=0x7f31fc0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f31fc056d20 0x7f31fc0591e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f32200aa9e0 tx=0x7f3208009040).stop 2020-11-20T12:20:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 msgr2=0x7f3220116cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f3220116cd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f321000eba0 tx=0x7f3210008040).stop 2020-11-20T12:20:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 shutdown_connections 2020-11-20T12:20:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f31fc056d20 0x7f31fc0591e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f32200abc70 0x7f32200a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3220067dc0 0x7f3220116cd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 --2- 172.21.15.44:0/316772478 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32200a72d0 0x7f32200a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 >> 172.21.15.44:0/316772478 conn(0x7f3220060170 msgr2=0x7f32200ad740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:57.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 shutdown_connections 2020-11-20T12:20:57.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:57.977+0000 7f322505a700 1 -- 172.21.15.44:0/316772478 wait complete. 2020-11-20T12:20:57.997 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:20:57.997 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:20:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.164+0000 7f7b3b539700 1 Processor -- start 2020-11-20T12:20:58.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.164+0000 7f7b3b539700 1 -- start start 2020-11-20T12:20:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b3b539700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b341381c0 0x7f7b341463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b3b539700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b340a92d0 con 0x7f7b340ae070 2020-11-20T12:20:58.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b3b539700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b340a9570 con 0x7f7b34060e40 2020-11-20T12:20:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b3b539700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340a9810 con 0x7f7b341381c0 2020-11-20T12:20:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b392d5700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f7b340ae070 0x7f7b34060730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46994/0 (socket says 172.21.15.44:46994) 2020-11-20T12:20:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b38ad4700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7b34060e40 0x7f7b34137ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49872/0 (socket says 172.21.15.44:49872) 2020-11-20T12:20:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b392d5700 1 -- 172.21.15.44:0/1655509024 learned_addr learned my addr 172.21.15.44:0/1655509024 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:20:58.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b39ad6700 1 --2- 172.21.15.44:0/1655509024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b341381c0 0x7f7b341463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b39ad6700 1 -- 172.21.15.44:0/1655509024 >> v1:172.21.15.79:6789/0 conn(0x7f7b34060e40 legacy=0x7f7b34137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b39ad6700 1 -- 172.21.15.44:0/1655509024 >> v1:172.21.15.44:6789/0 conn(0x7f7b340ae070 legacy=0x7f7b34060730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b23fff700 1 -- 172.21.15.44:0/1655509024 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092186327 0 0) 0x7f7b340a92d0 con 0x7f7b340ae070 2020-11-20T12:20:58.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.165+0000 7f7b39ad6700 1 -- 172.21.15.44:0/1655509024 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b340ae450 con 0x7f7b341381c0 2020-11-20T12:20:58.171 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.166+0000 7f7b39ad6700 1 --2- 172.21.15.44:0/1655509024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b341381c0 0x7f7b341463b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7b2c009cc0 tx=0x7f7b2c0091d0).ready entity=mon.2 client_cookie=c6f30a96e0af6f90 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.166+0000 7f7b23fff700 1 -- 172.21.15.44:0/1655509024 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00bbc0 con 0x7f7b341381c0 2020-11-20T12:20:58.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.166+0000 7f7b23fff700 1 -- 172.21.15.44:0/1655509024 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c00bd20 con 0x7f7b341381c0 2020-11-20T12:20:58.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.166+0000 7f7b23fff700 1 -- 172.21.15.44:0/1655509024 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011610 con 0x7f7b341381c0 2020-11-20T12:20:58.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.167+0000 7f7b3b539700 1 -- 172.21.15.44:0/1655509024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b341381c0 msgr2=0x7f7b341463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.167+0000 7f7b3b539700 1 --2- 172.21.15.44:0/1655509024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b341381c0 0x7f7b341463b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7b2c009cc0 tx=0x7f7b2c0091d0).stop 2020-11-20T12:20:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.167+0000 7f7b3b539700 1 -- 172.21.15.44:0/1655509024 shutdown_connections 2020-11-20T12:20:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.167+0000 7f7b3b539700 1 --2- 172.21.15.44:0/1655509024 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b341381c0 0x7f7b341463b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.167+0000 7f7b3b539700 1 -- 172.21.15.44:0/1655509024 >> 172.21.15.44:0/1655509024 conn(0x7f7b34062bc0 msgr2=0x7f7b34062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.167+0000 7f7b3b539700 1 -- 172.21.15.44:0/1655509024 shutdown_connections 2020-11-20T12:20:58.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.167+0000 7f7b3b539700 1 -- 172.21.15.44:0/1655509024 wait complete. 2020-11-20T12:20:58.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.168+0000 7f7b3b539700 1 Processor -- start 2020-11-20T12:20:58.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.168+0000 7f7b3b539700 1 -- start start 2020-11-20T12:20:58.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b3b539700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b34060e40 0x7f7b3413e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b3b539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340ae070 0x7f7b3413be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b3b539700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b341381c0 0x7f7b34133280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b3b539700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3413c320 con 0x7f7b340ae070 2020-11-20T12:20:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b3b539700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3413c540 con 0x7f7b341381c0 2020-11-20T12:20:58.177 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b3b539700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3413c7e0 con 0x7f7b34060e40 2020-11-20T12:20:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b38ad4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340ae070 0x7f7b3413be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b38ad4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340ae070 0x7f7b3413be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43254/0 (socket says 172.21.15.44:43254) 2020-11-20T12:20:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b38ad4700 1 -- 172.21.15.44:0/305426213 learned_addr learned my addr 172.21.15.44:0/305426213 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b39ad6700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b341381c0 0x7f7b34133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.178 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.169+0000 7f7b392d5700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b34060e40 0x7f7b3413e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b38ad4700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b34060e40 msgr2=0x7f7b3413e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b38ad4700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7b34060e40 0x7f7b3413e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b38ad4700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b341381c0 msgr2=0x7f7b34133280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.179 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b38ad4700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b341381c0 0x7f7b34133280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b38ad4700 1 -- 172.21.15.44:0/305426213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34149b60 con 0x7f7b340ae070 2020-11-20T12:20:58.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b39ad6700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b341381c0 0x7f7b34133280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:58.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b38ad4700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340ae070 0x7f7b3413be30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7b3000c710 tx=0x7f7b30007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b300072e0 con 0x7f7b340ae070 2020-11-20T12:20:58.180 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b30005530 con 0x7f7b340ae070 2020-11-20T12:20:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b300058b0 con 0x7f7b340ae070 2020-11-20T12:20:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3413e2b0 con 0x7f7b340ae070 2020-11-20T12:20:58.181 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.170+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b341339b0 con 0x7f7b340ae070 2020-11-20T12:20:58.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.171+0000 7f7b3b539700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b34149f00 0x7f7b3414a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.171+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34133c90 con 0x7f7b34149f00 2020-11-20T12:20:58.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.171+0000 7f7b392d5700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b34149f00 0x7f7b3414a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.182 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.172+0000 7f7b392d5700 1 -- 172.21.15.44:0/305426213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b34149cf0 con 0x7f7b34149f00 2020-11-20T12:20:58.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.172+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f7b30010070 con 0x7f7b340ae070 2020-11-20T12:20:58.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.172+0000 7f7b392d5700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b34149f00 0x7f7b3414a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7b24000bc0 tx=0x7f7b240079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.172+0000 7f7b217fa700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7b0c05f530 0x7f7b0c0619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.172+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f7b3006f780 con 0x7f7b340ae070 2020-11-20T12:20:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.172+0000 7f7b39ad6700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7b0c05f530 0x7f7b0c0619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.172+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24011650 con 0x7f7b34149f00 2020-11-20T12:20:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.173+0000 7f7b39ad6700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7b0c05f530 0x7f7b0c0619f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f7b341385f0 tx=0x7f7b2c016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.173+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f7b240117b0 con 0x7f7b34149f00 2020-11-20T12:20:58.184 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.173+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b34149f00 msgr2=0x7f7b3414a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.173+0000 7f7b217fa700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b34149f00 0x7f7b3414a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7b24000bc0 tx=0x7f7b240079e0).stop 2020-11-20T12:20:58.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.173+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b34149f00 msgr2=0x7f7b3414a2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:58.185 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.173+0000 7f7b217fa700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b34149f00 0x7f7b3414a2e0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0x7f7b24000bc0 tx=0x7f7b240079e0).stop 2020-11-20T12:20:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.184+0000 7f7b3b539700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340a1390 0x7f7b3414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.187 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.184+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3413c9a0 con 0x7f7b340a1390 2020-11-20T12:20:58.188 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.184+0000 7f7b392d5700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340a1390 0x7f7b3414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.185+0000 7f7b392d5700 1 -- 172.21.15.44:0/305426213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b340a50e0 con 0x7f7b340a1390 2020-11-20T12:20:58.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.185+0000 7f7b392d5700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340a1390 0x7f7b3414d130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7b24001e90 tx=0x7f7b24018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.189 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.185+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24013de0 con 0x7f7b340a1390 2020-11-20T12:20:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.185+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7f7b24011440 con 0x7f7b340a1390 2020-11-20T12:20:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.185+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340a1390 msgr2=0x7f7b3414d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.185+0000 7f7b217fa700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340a1390 0x7f7b3414d130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7b24001e90 tx=0x7f7b24018000).stop 2020-11-20T12:20:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.186+0000 7f7b217fa700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340a1390 msgr2=0x7f7b3414d130 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:58.190 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.186+0000 7f7b217fa700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340a1390 0x7f7b3414d130 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0x7f7b24001e90 tx=0x7f7b24018000).stop 2020-11-20T12:20:58.191 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:20:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:20:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:20:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:20:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 176, 2020-11-20T12:20:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:20:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:20:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:20:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:20:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:20:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:20:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:20:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:20:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:20:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:20:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:20:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:20:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:20:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:20:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:20:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:20:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:20:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:20:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:20:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:20:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:20:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:20:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:20:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:20:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:20:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:20:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:20:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:20:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:20:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:20:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:20:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:20:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:20:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:20:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:20:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:20:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:20:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:20:58.216 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.217 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:20:58.217 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.217 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.218 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:20:58.218 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:20:58.218 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.218 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.219 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.219 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.219 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:58.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.221 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.221 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:20:58.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.221 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.222 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.222 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:58.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.222 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.222 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:20:58.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.223 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.223 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.223 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:20:58.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.226 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:20:58.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.226 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.226 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:20:58.228 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:20:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7b0c05f530 msgr2=0x7f7b0c0619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b3b539700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f7b0c05f530 0x7f7b0c0619f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f7b341385f0 tx=0x7f7b2c016000).stop 2020-11-20T12:20:58.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340ae070 msgr2=0x7f7b3413be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b3b539700 1 --2- 172.21.15.44:0/305426213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b340ae070 0x7f7b3413be30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7b3000c710 tx=0x7f7b30007c40).stop 2020-11-20T12:20:58.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b39ad6700 1 -- 172.21.15.44:0/305426213 reap_dead start 2020-11-20T12:20:58.230 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b39ad6700 1 -- 172.21.15.44:0/305426213 reap_dead start 2020-11-20T12:20:58.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 shutdown_connections 2020-11-20T12:20:58.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.187+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 >> 172.21.15.44:0/305426213 conn(0x7f7b34062bc0 msgr2=0x7f7b34137880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.188+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 shutdown_connections 2020-11-20T12:20:58.231 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.188+0000 7f7b3b539700 1 -- 172.21.15.44:0/305426213 wait complete. 2020-11-20T12:20:58.232 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:20:58.401 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.398+0000 7f8ccfb3a700 1 Processor -- start 2020-11-20T12:20:58.402 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.398+0000 7f8ccfb3a700 1 -- start start 2020-11-20T12:20:58.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.399+0000 7f8ccfb3a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.399+0000 7f8ccfb3a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8cc80a4c30 0x7f8cc813a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.399+0000 7f8ccfb3a700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80a6260 con 0x7f8cc813ae80 2020-11-20T12:20:58.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.399+0000 7f8ccfb3a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc80a52e0 con 0x7f8cc80a4c30 2020-11-20T12:20:58.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.399+0000 7f8ccfb3a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80a6490 con 0x7f8cc8060330 2020-11-20T12:20:58.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd0d5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8cc80a4c30 0x7f8cc813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54702/0 (socket says 172.21.15.44:54702) 2020-11-20T12:20:58.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd0d5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8cc80a4c30 0x7f8cc813a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35316/0 (socket says 172.21.15.44:35316) 2020-11-20T12:20:58.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 -- 172.21.15.44:0/3234241468 learned_addr learned my addr 172.21.15.44:0/3234241468 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:58.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 -- 172.21.15.44:0/3234241468 >> v2:172.21.15.79:3300/0 conn(0x7f8cc80a4c30 msgr2=0x7f8cc813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 --2- 172.21.15.44:0/3234241468 >> v2:172.21.15.79:3300/0 conn(0x7f8cc80a4c30 0x7f8cc813a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 -- 172.21.15.44:0/3234241468 >> v1:172.21.15.44:6789/0 conn(0x7f8cc813ae80 legacy=0x7f8cc8136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 -- 172.21.15.44:0/3234241468 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc80a6730 con 0x7f8cc8060330 2020-11-20T12:20:58.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd0d5700 1 --2- 172.21.15.44:0/3234241468 >> v2:172.21.15.79:3300/0 conn(0x7f8cc80a4c30 0x7f8cc813a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:58.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccd8d6700 1 --2- 172.21.15.44:0/3234241468 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80a4740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8cb8006fd0 tx=0x7f8cb8006eb0).ready entity=mon.2 client_cookie=f93e38d763168697 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccc8d4700 1 -- 172.21.15.44:0/3234241468 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8009810 con 0x7f8cc8060330 2020-11-20T12:20:58.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccc8d4700 1 -- 172.21.15.44:0/3234241468 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8009970 con 0x7f8cc8060330 2020-11-20T12:20:58.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.400+0000 7f8ccc8d4700 1 -- 172.21.15.44:0/3234241468 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8009cf0 con 0x7f8cc8060330 2020-11-20T12:20:58.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.401+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/3234241468 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 msgr2=0x7f8cc80a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.408 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.401+0000 7f8ccfb3a700 1 --2- 172.21.15.44:0/3234241468 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80a4740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f8cb8006fd0 tx=0x7f8cb8006eb0).stop 2020-11-20T12:20:58.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.401+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/3234241468 shutdown_connections 2020-11-20T12:20:58.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.401+0000 7f8ccfb3a700 1 --2- 172.21.15.44:0/3234241468 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80a4740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.401+0000 7f8ccfb3a700 1 --2- 172.21.15.44:0/3234241468 >> v2:172.21.15.79:3300/0 conn(0x7f8cc80a4c30 0x7f8cc813a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.401+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/3234241468 >> 172.21.15.44:0/3234241468 conn(0x7f8cc8062690 msgr2=0x7f8cc80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.402+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/3234241468 shutdown_connections 2020-11-20T12:20:58.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.402+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/3234241468 wait complete. 2020-11-20T12:20:58.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.404+0000 7f8ccfb3a700 1 Processor -- start 2020-11-20T12:20:58.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.404+0000 7f8ccfb3a700 1 -- start start 2020-11-20T12:20:58.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.404+0000 7f8ccfb3a700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.404+0000 7f8ccfb3a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80a4c30 0x7f8cc80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8ccfb3a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc813ae80 0x7f8cc80aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8ccfb3a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80add90 con 0x7f8cc813ae80 2020-11-20T12:20:58.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8ccfb3a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81376a0 con 0x7f8cc80a4c30 2020-11-20T12:20:58.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8ccfb3a700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8137940 con 0x7f8cc8060330 2020-11-20T12:20:58.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc813ae80 0x7f8cc80aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc813ae80 0x7f8cc80aaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43274/0 (socket says 172.21.15.44:43274) 2020-11-20T12:20:58.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 -- 172.21.15.44:0/2615337200 learned_addr learned my addr 172.21.15.44:0/2615337200 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:58.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8ccd8d6700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8ccd0d5700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80a4c30 0x7f8cc80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 msgr2=0x7f8cc80acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8cc8060330 0x7f8cc80acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80a4c30 msgr2=0x7f8cc80ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80a4c30 0x7f8cc80ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 -- 172.21.15.44:0/2615337200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8124970 con 0x7f8cc813ae80 2020-11-20T12:20:58.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.405+0000 7f8cce0d7700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc813ae80 0x7f8cc80aaec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8cc000ca90 tx=0x7f8cc0007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.406+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000ce60 con 0x7f8cc813ae80 2020-11-20T12:20:58.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.406+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc000a730 con 0x7f8cc813ae80 2020-11-20T12:20:58.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.406+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000aad0 con 0x7f8cc813ae80 2020-11-20T12:20:58.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.406+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc8137c40 con 0x7f8cc813ae80 2020-11-20T12:20:58.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.406+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc81371d0 con 0x7f8cc813ae80 2020-11-20T12:20:58.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.407+0000 7f8ccfb3a700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80ab3b0 0x7f8cc8124db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.407+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8137b00 con 0x7f8cc80ab3b0 2020-11-20T12:20:58.416 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.408+0000 7f8ccd8d6700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80ab3b0 0x7f8cc8124db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.408+0000 7f8ccd8d6700 1 -- 172.21.15.44:0/2615337200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cc8061650 con 0x7f8cc80ab3b0 2020-11-20T12:20:58.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.408+0000 7f8ccd8d6700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80ab3b0 0x7f8cc8124db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8cb8006ef0 tx=0x7f8cb8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.408+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8009cb0 con 0x7f8cc80ab3b0 2020-11-20T12:20:58.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.408+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8cc0019030 con 0x7f8cc813ae80 2020-11-20T12:20:58.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.409+0000 7f8cb5ffb700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8ca0056dc0 0x7f8ca0059280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.409+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8cc006f750 con 0x7f8cc813ae80 2020-11-20T12:20:58.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.409+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f8cb8011c80 con 0x7f8cc80ab3b0 2020-11-20T12:20:58.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.409+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80ab3b0 msgr2=0x7f8cc8124db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.409+0000 7f8cb5ffb700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80ab3b0 0x7f8cc8124db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8cb8006ef0 tx=0x7f8cb8015000).stop 2020-11-20T12:20:58.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.409+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80ab3b0 msgr2=0x7f8cc8124db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:58.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.409+0000 7f8cb5ffb700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc80ab3b0 0x7f8cc8124db0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0x7f8cb8006ef0 tx=0x7f8cb8015000).stop 2020-11-20T12:20:58.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.410+0000 7f8ccd0d5700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8ca0056dc0 0x7f8ca0059280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.410+0000 7f8ccd0d5700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8ca0056dc0 0x7f8ca0059280 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f8cc4006fd0 tx=0x7f8cc4006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.421+0000 7f8ccfb3a700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc81252a0 0x7f8cc8125680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.421+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80446d0 con 0x7f8cc81252a0 2020-11-20T12:20:58.425 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.421+0000 7f8ccd8d6700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc81252a0 0x7f8cc8125680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.421+0000 7f8ccd8d6700 1 -- 172.21.15.44:0/2615337200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8cc80680f0 con 0x7f8cc81252a0 2020-11-20T12:20:58.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.422+0000 7f8ccd8d6700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc81252a0 0x7f8cc8125680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f8cc80a6a50 tx=0x7f8cb8021070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.422+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb802e9d0 con 0x7f8cc81252a0 2020-11-20T12:20:58.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.422+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4776 (secure 0 0 0) 0x7f8cb8011a70 con 0x7f8cc81252a0 2020-11-20T12:20:58.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.422+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc81252a0 msgr2=0x7f8cc8125680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.422+0000 7f8cb5ffb700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc81252a0 0x7f8cc8125680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f8cc80a6a50 tx=0x7f8cb8021070).stop 2020-11-20T12:20:58.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.422+0000 7f8cb5ffb700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc81252a0 msgr2=0x7f8cc8125680 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:58.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.422+0000 7f8cb5ffb700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8cc81252a0 0x7f8cc8125680 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0x7f8cc80a6a50 tx=0x7f8cb8021070).stop 2020-11-20T12:20:58.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8ca0056dc0 msgr2=0x7f8ca0059280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8ca0056dc0 0x7f8ca0059280 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f8cc4006fd0 tx=0x7f8cc4006df0).stop 2020-11-20T12:20:58.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc813ae80 msgr2=0x7f8cc80aaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 --2- 172.21.15.44:0/2615337200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc813ae80 0x7f8cc80aaec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8cc000ca90 tx=0x7f8cc0007f50).stop 2020-11-20T12:20:58.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8cce0d7700 1 -- 172.21.15.44:0/2615337200 reap_dead start 2020-11-20T12:20:58.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8cce0d7700 1 -- 172.21.15.44:0/2615337200 reap_dead start 2020-11-20T12:20:58.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 shutdown_connections 2020-11-20T12:20:58.429 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 >> 172.21.15.44:0/2615337200 conn(0x7f8cc8062690 msgr2=0x7f8cc8139ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 shutdown_connections 2020-11-20T12:20:58.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.423+0000 7f8ccfb3a700 1 -- 172.21.15.44:0/2615337200 wait complete. 2020-11-20T12:20:58.430 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:20:58.430 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:20:58.430 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:20:58.431 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:20:58.431 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 176, 2020-11-20T12:20:58.431 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:20:58.431 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:20:58.431 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:20:58.432 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:20:58.432 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.432 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:20:58.432 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:58.433 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:20:58.433 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:20:58.433 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.433 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.433 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.434 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.434 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.434 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.434 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.434 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.435 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.435 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:20:58.435 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:20:58.435 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.435 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.436 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.436 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.436 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.436 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.436 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.437 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.437 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.437 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:20:58.437 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:20:58.438 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:20:58.438 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:20:58.438 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:20:58.438 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:20:58.439 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:58.439 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:58.439 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:20:58.439 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:20:58.439 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:20:58.440 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:20:58.440 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:20:58.440 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:58.440 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:20:58.440 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.441 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.441 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.441 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.441 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.441 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.442 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.442 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.442 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.442 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:20:58.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.443 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:20:58.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.443 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:20:58.443 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:20:58.443 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.444 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.444 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.444 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:20:58.444 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.445 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:20:58.445 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.446 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.446 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.446 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:20:58.446 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:20:58.446 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.447 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.447 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.447 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:20:58.448 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:20:58.448 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.448 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.449 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:20:58.449 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.450 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:20:58.450 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.451 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.451 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:20:58.451 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:20:58.451 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.452 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.452 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.452 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:20:58.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:20:58.453 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.453 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.454 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:20:58.454 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.455 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:20:58.455 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.455 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.455 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.456 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:20:58.456 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:20:58.456 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.456 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.456 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.456 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.457 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.457 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:20:58.457 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:20:58.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.458 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.458 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.458 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:58.458 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.459 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.459 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:20:58.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.459 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.459 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.460 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:58.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.460 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.460 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:20:58.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.461 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.461 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.461 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:20:58.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.461 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.462 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:20:58.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.462 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.462 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.462 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:20:58.463 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.463 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.463 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:20:58.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.463 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.463 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.464 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:58.464 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.464 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.464 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:20:58.465 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:20:58.466 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:20:58.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.662+0000 7f1250ee5700 1 Processor -- start 2020-11-20T12:20:58.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.662+0000 7f1250ee5700 1 -- start start 2020-11-20T12:20:58.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1250ee5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c066e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1250ee5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f124c0aad50 0x7f124c0a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1250ee5700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0676b0 con 0x7f124c13e260 2020-11-20T12:20:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1250ee5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c067410 con 0x7f124c0aad50 2020-11-20T12:20:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1250ee5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0a95d0 con 0x7f124c061340 2020-11-20T12:20:58.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1243fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f124c13e260 0x7f124c0a8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47032/0 (socket says 172.21.15.44:47032) 2020-11-20T12:20:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1243fff700 1 -- 172.21.15.44:0/463114439 learned_addr learned my addr 172.21.15.44:0/463114439 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:20:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124ad9d700 1 --2- 172.21.15.44:0/463114439 >> v2:172.21.15.79:3300/0 conn(0x7f124c0aad50 0x7f124c0a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124a59c700 1 --2- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1249d9b700 1 -- 172.21.15.44:0/463114439 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176783462 0 0) 0x7f124c0676b0 con 0x7f124c13e260 2020-11-20T12:20:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1249d9b700 1 -- 172.21.15.44:0/463114439 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f122c003240 con 0x7f124c13e260 2020-11-20T12:20:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124ad9d700 1 -- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 msgr2=0x7f124c066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124ad9d700 1 --2- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c066e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124ad9d700 1 -- 172.21.15.44:0/463114439 >> v1:172.21.15.44:6789/0 conn(0x7f124c13e260 legacy=0x7f124c0a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f1249d9b700 1 -- 172.21.15.44:0/463114439 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 579699288 0 0) 0x7f122c003240 con 0x7f124c13e260 2020-11-20T12:20:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124ad9d700 1 -- 172.21.15.44:0/463114439 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0a9f60 con 0x7f124c0aad50 2020-11-20T12:20:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124a59c700 1 --2- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c066e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.663+0000 7f124ad9d700 1 --2- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 0x7f124c0a9a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f1244009c90 tx=0x7f12440091d0).ready entity=mon.1 client_cookie=c935372c64a1173e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1249d9b700 1 -- 172.21.15.44:0/463114439 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12340014e0 con 0x7f124c0aad50 2020-11-20T12:20:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1249d9b700 1 -- 172.21.15.44:0/463114439 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124400b740 con 0x7f124c0aad50 2020-11-20T12:20:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1249d9b700 1 -- 172.21.15.44:0/463114439 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400bac0 con 0x7f124c0aad50 2020-11-20T12:20:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 -- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 msgr2=0x7f124c0a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 --2- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 0x7f124c0a9a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f1244009c90 tx=0x7f12440091d0).stop 2020-11-20T12:20:58.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 -- 172.21.15.44:0/463114439 shutdown_connections 2020-11-20T12:20:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 --2- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 0x7f124c0a9a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 --2- 172.21.15.44:0/463114439 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c066e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 -- 172.21.15.44:0/463114439 >> 172.21.15.44:0/463114439 conn(0x7f124c0ae170 msgr2=0x7f124c0aa170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 -- 172.21.15.44:0/463114439 shutdown_connections 2020-11-20T12:20:58.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.664+0000 7f1250ee5700 1 -- 172.21.15.44:0/463114439 wait complete. 2020-11-20T12:20:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 Processor -- start 2020-11-20T12:20:58.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 -- start start 2020-11-20T12:20:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c0a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 0x7f124c1406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c13e260 0x7f124c0a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c141670 con 0x7f124c13e260 2020-11-20T12:20:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c1418a0 con 0x7f124c0aad50 2020-11-20T12:20:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1250ee5700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c141b40 con 0x7f124c061340 2020-11-20T12:20:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f124a59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c0a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.680 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f124a59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c0a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54726/0 (socket says 172.21.15.44:54726) 2020-11-20T12:20:58.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f124a59c700 1 -- 172.21.15.44:0/1346367239 learned_addr learned my addr 172.21.15.44:0/1346367239 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:58.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f1243fff700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 0x7f124c1406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.666+0000 7f124ad9d700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c13e260 0x7f124c0a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.681 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f124a59c700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 msgr2=0x7f124c1406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f124a59c700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 0x7f124c1406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f124a59c700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c13e260 msgr2=0x7f124c0a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f124a59c700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f124c13e260 0x7f124c0a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f124a59c700 1 -- 172.21.15.44:0/1346367239 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c141de0 con 0x7f124c061340 2020-11-20T12:20:58.682 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f1243fff700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f124c0aad50 0x7f124c1406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f124a59c700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c0a4c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f123c00ef50 tx=0x7f123c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00ca80 con 0x7f124c061340 2020-11-20T12:20:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c00cbe0 con 0x7f124c061340 2020-11-20T12:20:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c140bd0 con 0x7f124c061340 2020-11-20T12:20:58.683 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0123f0 con 0x7f124c061340 2020-11-20T12:20:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.667+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c0a7490 con 0x7f124c061340 2020-11-20T12:20:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.668+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f123c012760 con 0x7f124c061340 2020-11-20T12:20:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f1250ee5700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c14d230 0x7f124c14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c069750 con 0x7f124c14d230 2020-11-20T12:20:58.684 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f12427fc700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1228056d20 0x7f12280591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f1243fff700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c14d230 0x7f124c14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f123c0706e0 con 0x7f124c061340 2020-11-20T12:20:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f124ad9d700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1228056d20 0x7f12280591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f1243fff700 1 -- 172.21.15.44:0/1346367239 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f124c0ae580 con 0x7f124c14d230 2020-11-20T12:20:58.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f1243fff700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c14d230 0x7f124c14d610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f124c14d190 tx=0x7f1234009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.669+0000 7f124ad9d700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1228056d20 0x7f12280591e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f124400b650 tx=0x7f1244016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.670+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400dca0 con 0x7f124c14d230 2020-11-20T12:20:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.670+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f123400de00 con 0x7f124c14d230 2020-11-20T12:20:58.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.670+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c14d230 msgr2=0x7f124c14d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.670+0000 7f12427fc700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c14d230 0x7f124c14d610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f124c14d190 tx=0x7f1234009040).stop 2020-11-20T12:20:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.670+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c14d230 msgr2=0x7f124c14d610 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:58.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.670+0000 7f12427fc700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c14d230 0x7f124c14d610 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0x7f124c14d190 tx=0x7f1234009040).stop 2020-11-20T12:20:58.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.681+0000 7f1250ee5700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c149cc0 0x7f124c14a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.681+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c061720 con 0x7f124c149cc0 2020-11-20T12:20:58.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.681+0000 7f1243fff700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c149cc0 0x7f124c14a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.689 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.681+0000 7f1243fff700 1 -- 172.21.15.44:0/1346367239 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f124c0a79d0 con 0x7f124c149cc0 2020-11-20T12:20:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.681+0000 7f1243fff700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c149cc0 0x7f124c14a0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f124c0a76e0 tx=0x7f1234018000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12340150d0 con 0x7f124c149cc0 2020-11-20T12:20:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7f1234015230 con 0x7f124c149cc0 2020-11-20T12:20:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c149cc0 msgr2=0x7f124c14a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.690 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f12427fc700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c149cc0 0x7f124c14a0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f124c0a76e0 tx=0x7f1234018000).stop 2020-11-20T12:20:58.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f12427fc700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c149cc0 msgr2=0x7f124c14a0a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:20:58.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f12427fc700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c149cc0 0x7f124c14a0a0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0x7f124c0a76e0 tx=0x7f1234018000).stop 2020-11-20T12:20:58.691 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1228056d20 msgr2=0x7f12280591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f1250ee5700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1228056d20 0x7f12280591e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f124400b650 tx=0x7f1244016000).stop 2020-11-20T12:20:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 msgr2=0x7f124c0a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.682+0000 7f1250ee5700 1 --2- 172.21.15.44:0/1346367239 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f124c061340 0x7f124c0a4c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f123c00ef50 tx=0x7f123c006df0).stop 2020-11-20T12:20:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.683+0000 7f124ad9d700 1 -- 172.21.15.44:0/1346367239 reap_dead start 2020-11-20T12:20:58.692 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.683+0000 7f124ad9d700 1 -- 172.21.15.44:0/1346367239 reap_dead start 2020-11-20T12:20:58.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.683+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 shutdown_connections 2020-11-20T12:20:58.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.683+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 >> 172.21.15.44:0/1346367239 conn(0x7f124c0ae170 msgr2=0x7f124c0ab500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.683+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 shutdown_connections 2020-11-20T12:20:58.693 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.683+0000 7f1250ee5700 1 -- 172.21.15.44:0/1346367239 wait complete. 2020-11-20T12:20:58.694 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:20:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:20:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:20:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:20:58.694 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 176, 2020-11-20T12:20:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:20:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:20:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:20:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:20:58.695 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:20:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:20:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:20:58.696 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.697 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.698 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:20:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:20:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.700 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:20:58.701 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:20:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:20:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:20:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:20:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:20:58.702 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:20:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:20:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:20:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:20:58.703 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:20:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:20:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:20:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:20:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:20:58.704 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:20:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:20:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:20:58.705 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:20:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:20:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:20:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:20:58.706 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:20:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:20:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.707 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:20:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:20:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:20:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:20:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:20:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:20:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.711 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:20:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:20:58.712 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:20:58.713 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:20:58.714 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:20:58.715 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:20:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:20:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:20:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:20:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:20:58.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:20:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:20:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:20:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:20:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:20:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.719 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:20:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:20:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:20:58.720 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:20:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:20:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:20:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:20:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.722 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:20:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:20:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:20:58.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:20:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:20:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:20:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:20:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:20:58.724 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:20:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:20:58.725 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:20:58.725 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:20:58.726 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:20:58.726 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:20:58.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.913+0000 7f0a13569700 1 Processor -- start 2020-11-20T12:20:58.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a13569700 1 -- start start 2020-11-20T12:20:58.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a13569700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c13c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a13569700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a0c066250 0x7f0a0c13c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a13569700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0a8bf0 con 0x7f0a0c0aa1e0 2020-11-20T12:20:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a13569700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a86b0 con 0x7f0a0c066250 2020-11-20T12:20:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a13569700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a8950 con 0x7f0a0c0a7810 2020-11-20T12:20:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a11b06700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a0c066250 0x7f0a0c13c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.914+0000 7f0a11b06700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0a0c066250 0x7f0a0c13c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43304/0 (socket says 172.21.15.44:43304) 2020-11-20T12:20:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a11b06700 1 -- 172.21.15.44:0/664183661 learned_addr learned my addr 172.21.15.44:0/664183661 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a10b04700 1 --2- 172.21.15.44:0/664183661 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c13c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a11b06700 1 -- 172.21.15.44:0/664183661 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 msgr2=0x7f0a0c13c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a11b06700 1 --2- 172.21.15.44:0/664183661 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c13c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a11b06700 1 -- 172.21.15.44:0/664183661 >> v1:172.21.15.79:6789/0 conn(0x7f0a0c0aa1e0 legacy=0x7f0a0c065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a11b06700 1 -- 172.21.15.44:0/664183661 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0aa5c0 con 0x7f0a0c066250 2020-11-20T12:20:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a11b06700 1 --2- 172.21.15.44:0/664183661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c13c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0a04009d50 tx=0x7f0a04009290).ready entity=mon.0 client_cookie=d3feef60bc3445a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a03fff700 1 -- 172.21.15.44:0/664183661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400bc80 con 0x7f0a0c066250 2020-11-20T12:20:58.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a03fff700 1 -- 172.21.15.44:0/664183661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0400bde0 con 0x7f0a0c066250 2020-11-20T12:20:58.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.915+0000 7f0a03fff700 1 -- 172.21.15.44:0/664183661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04011610 con 0x7f0a0c066250 2020-11-20T12:20:58.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 -- 172.21.15.44:0/664183661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 msgr2=0x7f0a0c13c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 --2- 172.21.15.44:0/664183661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c13c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0a04009d50 tx=0x7f0a04009290).stop 2020-11-20T12:20:58.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 -- 172.21.15.44:0/664183661 shutdown_connections 2020-11-20T12:20:58.923 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 --2- 172.21.15.44:0/664183661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c13c660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 --2- 172.21.15.44:0/664183661 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c13c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 -- 172.21.15.44:0/664183661 >> 172.21.15.44:0/664183661 conn(0x7f0a0c062690 msgr2=0x7f0a0c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 -- 172.21.15.44:0/664183661 shutdown_connections 2020-11-20T12:20:58.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.916+0000 7f0a13569700 1 -- 172.21.15.44:0/664183661 wait complete. 2020-11-20T12:20:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.917+0000 7f0a13569700 1 Processor -- start 2020-11-20T12:20:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.917+0000 7f0a13569700 1 -- start start 2020-11-20T12:20:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.917+0000 7f0a13569700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c0a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a13569700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c0a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a13569700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a0c0aa1e0 0x7f0a0c149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a13569700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a0f90 con 0x7f0a0c066250 2020-11-20T12:20:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a13569700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a05b0 con 0x7f0a0c0aa1e0 2020-11-20T12:20:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a13569700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a0850 con 0x7f0a0c0a7810 2020-11-20T12:20:58.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c0a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c0a3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43306/0 (socket says 172.21.15.44:43306) 2020-11-20T12:20:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 -- 172.21.15.44:0/3458583626 learned_addr learned my addr 172.21.15.44:0/3458583626 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:20:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11b06700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a0c0aa1e0 0x7f0a0c149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a10b04700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c0a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 -- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 msgr2=0x7f0a0c0a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c0a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 -- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a0c0aa1e0 msgr2=0x7f0a0c149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a0c0aa1e0 0x7f0a0c149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 -- 172.21.15.44:0/3458583626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0a0af0 con 0x7f0a0c066250 2020-11-20T12:20:58.930 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a11305700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c0a3580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0a0800c6e0 tx=0x7f0a08007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a017fa700 1 -- 172.21.15.44:0/3458583626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08007020 con 0x7f0a0c066250 2020-11-20T12:20:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a017fa700 1 -- 172.21.15.44:0/3458583626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08007180 con 0x7f0a0c066250 2020-11-20T12:20:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0a02a0 con 0x7f0a0c066250 2020-11-20T12:20:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.918+0000 7f0a017fa700 1 -- 172.21.15.44:0/3458583626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080064f0 con 0x7f0a0c066250 2020-11-20T12:20:58.931 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.919+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0a1fa0 con 0x7f0a0c066250 2020-11-20T12:20:58.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.920+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c061650 con 0x7f0a0c066250 2020-11-20T12:20:58.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.920+0000 7f0a017fa700 1 -- 172.21.15.44:0/3458583626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0a0801a060 con 0x7f0a0c066250 2020-11-20T12:20:58.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.921+0000 7f0a017fa700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09e4056d20 0x7f09e40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.921+0000 7f0a017fa700 1 -- 172.21.15.44:0/3458583626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0a0806f700 con 0x7f0a0c066250 2020-11-20T12:20:58.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.921+0000 7f0a10b04700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09e4056d20 0x7f09e40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.921+0000 7f0a10b04700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09e4056d20 0x7f09e40591e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f0a0c0a6510 tx=0x7f09fc008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:58.925+0000 7f0a017fa700 1 -- 172.21.15.44:0/3458583626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0a080449d0 con 0x7f0a0c066250 2020-11-20T12:20:59.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.206+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f0a0c0a1fa0 con 0x7f0a0c066250 2020-11-20T12:20:59.210 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.208+0000 7f0a017fa700 1 -- 172.21.15.44:0/3458583626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f0a0c0a1fa0 con 0x7f0a0c066250 2020-11-20T12:20:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09e4056d20 msgr2=0x7f09e40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.215 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09e4056d20 0x7f09e40591e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f0a0c0a6510 tx=0x7f09fc008040).stop 2020-11-20T12:20:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 msgr2=0x7f0a0c0a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c0a3580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0a0800c6e0 tx=0x7f0a08007c50).stop 2020-11-20T12:20:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 shutdown_connections 2020-11-20T12:20:59.216 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f09e4056d20 0x7f09e40591e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0a0c0aa1e0 0x7f0a0c149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.212+0000 7f0a13569700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a0c066250 0x7f0a0c0a3580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.217 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.213+0000 7f0a13569700 1 --2- 172.21.15.44:0/3458583626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0a0c0a7810 0x7f0a0c0a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.213+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 >> 172.21.15.44:0/3458583626 conn(0x7f0a0c062690 msgr2=0x7f0a0c0672c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.213+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 shutdown_connections 2020-11-20T12:20:59.218 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:20:59.213+0000 7f0a13569700 1 -- 172.21.15.44:0/3458583626 wait complete. 2020-11-20T12:20:59.232 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:21:09.197 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:21:09.217 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:21:09.233 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:21:09.234 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:21:09.473 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.470+0000 7ff963778700 1 Processor -- start 2020-11-20T12:21:09.474 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.470+0000 7ff963778700 1 -- start start 2020-11-20T12:21:09.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.471+0000 7ff963778700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff95c0abc70 0x7ff95c0671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.471+0000 7ff963778700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff95c0a72d0 0x7ff95c0676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.471+0000 7ff963778700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c13c0d0 0x7ff95c067ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.471+0000 7ff963778700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c068090 con 0x7ff95c0abc70 2020-11-20T12:21:09.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.471+0000 7ff963778700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c0ac550 con 0x7ff95c0a72d0 2020-11-20T12:21:09.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.471+0000 7ff963778700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0ab340 con 0x7ff95c13c0d0 2020-11-20T12:21:09.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff95c0abc70 0x7ff95c0671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff960d13700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff95c0a72d0 0x7ff95c0676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff960d13700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff95c0a72d0 0x7ff95c0676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35366/0 (socket says 172.21.15.44:35366) 2020-11-20T12:21:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff960d13700 1 -- 172.21.15.44:0/2556694103 learned_addr learned my addr 172.21.15.44:0/2556694103 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff95c0abc70 0x7ff95c0671c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43314/0 (socket says 172.21.15.44:43314) 2020-11-20T12:21:09.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961d15700 1 --2- 172.21.15.44:0/2556694103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c13c0d0 0x7ff95c067ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 -- 172.21.15.44:0/2556694103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c13c0d0 msgr2=0x7ff95c067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 --2- 172.21.15.44:0/2556694103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c13c0d0 0x7ff95c067ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 -- 172.21.15.44:0/2556694103 >> v2:172.21.15.79:3300/0 conn(0x7ff95c0a72d0 msgr2=0x7ff95c0676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:09.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 --2- 172.21.15.44:0/2556694103 >> v2:172.21.15.79:3300/0 conn(0x7ff95c0a72d0 0x7ff95c0676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.480 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 -- 172.21.15.44:0/2556694103 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c13c650 con 0x7ff95c0abc70 2020-11-20T12:21:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961d15700 1 --2- 172.21.15.44:0/2556694103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c13c0d0 0x7ff95c067ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.472+0000 7ff961514700 1 --2- 172.21.15.44:0/2556694103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0abc70 0x7ff95c0671c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff95c0b0190 tx=0x7ff94c006d90).ready entity=mon.0 client_cookie=2036dd17e2ff93a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.474+0000 7ff94bfff700 1 -- 172.21.15.44:0/2556694103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c006030 con 0x7ff95c0abc70 2020-11-20T12:21:09.484 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.474+0000 7ff94bfff700 1 -- 172.21.15.44:0/2556694103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c006190 con 0x7ff95c0abc70 2020-11-20T12:21:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.474+0000 7ff94bfff700 1 -- 172.21.15.44:0/2556694103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c006510 con 0x7ff95c0abc70 2020-11-20T12:21:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.475+0000 7ff963778700 1 -- 172.21.15.44:0/2556694103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0abc70 msgr2=0x7ff95c0671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.475+0000 7ff963778700 1 --2- 172.21.15.44:0/2556694103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0abc70 0x7ff95c0671c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff95c0b0190 tx=0x7ff94c006d90).stop 2020-11-20T12:21:09.485 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.476+0000 7ff963778700 1 -- 172.21.15.44:0/2556694103 shutdown_connections 2020-11-20T12:21:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.476+0000 7ff963778700 1 --2- 172.21.15.44:0/2556694103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0abc70 0x7ff95c0671c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.476+0000 7ff963778700 1 --2- 172.21.15.44:0/2556694103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c13c0d0 0x7ff95c067ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.476+0000 7ff963778700 1 --2- 172.21.15.44:0/2556694103 >> v2:172.21.15.79:3300/0 conn(0x7ff95c0a72d0 0x7ff95c0676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.476+0000 7ff963778700 1 -- 172.21.15.44:0/2556694103 >> 172.21.15.44:0/2556694103 conn(0x7ff95c060170 msgr2=0x7ff95c060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.476+0000 7ff963778700 1 -- 172.21.15.44:0/2556694103 shutdown_connections 2020-11-20T12:21:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.476+0000 7ff963778700 1 -- 172.21.15.44:0/2556694103 wait complete. 2020-11-20T12:21:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.477+0000 7ff963778700 1 Processor -- start 2020-11-20T12:21:09.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.477+0000 7ff963778700 1 -- start start 2020-11-20T12:21:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff963778700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0a72d0 0x7ff95c138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff963778700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c0abc70 0x7ff95c13a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff963778700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff95c13c0d0 0x7ff95c13a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff963778700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c13adf0 con 0x7ff95c0a72d0 2020-11-20T12:21:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff963778700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c1360a0 con 0x7ff95c13c0d0 2020-11-20T12:21:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff963778700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c136340 con 0x7ff95c0abc70 2020-11-20T12:21:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff961514700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0a72d0 0x7ff95c138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff961514700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0a72d0 0x7ff95c138f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43320/0 (socket says 172.21.15.44:43320) 2020-11-20T12:21:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff961514700 1 -- 172.21.15.44:0/2495846761 learned_addr learned my addr 172.21.15.44:0/2495846761 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.478+0000 7ff960d13700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c0abc70 0x7ff95c13a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff961d15700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff95c13c0d0 0x7ff95c13a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff961514700 1 -- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c0abc70 msgr2=0x7ff95c13a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff961514700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c0abc70 0x7ff95c13a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff961514700 1 -- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff95c13c0d0 msgr2=0x7ff95c13a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff961514700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff95c13c0d0 0x7ff95c13a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff961514700 1 -- 172.21.15.44:0/2495846761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c1365e0 con 0x7ff95c0a72d0 2020-11-20T12:21:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff960d13700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c0abc70 0x7ff95c13a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff961514700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0a72d0 0x7ff95c138f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7ff95c05c300 tx=0x7ff94c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff9497fa700 1 -- 172.21.15.44:0/2495846761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c01fa30 con 0x7ff95c0a72d0 2020-11-20T12:21:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff9497fa700 1 -- 172.21.15.44:0/2495846761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c023070 con 0x7ff95c0a72d0 2020-11-20T12:21:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff9497fa700 1 -- 172.21.15.44:0/2495846761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c01fb90 con 0x7ff95c0a72d0 2020-11-20T12:21:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c139640 con 0x7ff95c0a72d0 2020-11-20T12:21:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.479+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c0a3850 con 0x7ff95c0a72d0 2020-11-20T12:21:09.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.481+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c0a3b30 con 0x7ff95c0a72d0 2020-11-20T12:21:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.483+0000 7ff9497fa700 1 -- 172.21.15.44:0/2495846761 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff94c00f070 con 0x7ff95c0a72d0 2020-11-20T12:21:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.483+0000 7ff9497fa700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff934056d20 0x7ff9340591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.483+0000 7ff9497fa700 1 -- 172.21.15.44:0/2495846761 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff94c075c60 con 0x7ff95c0a72d0 2020-11-20T12:21:09.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.483+0000 7ff960d13700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff934056d20 0x7ff9340591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.484+0000 7ff960d13700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff934056d20 0x7ff9340591e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7ff954006d90 tx=0x7ff954008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.488+0000 7ff9497fa700 1 -- 172.21.15.44:0/2495846761 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff94c04aff0 con 0x7ff95c0a72d0 2020-11-20T12:21:09.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.894+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff95c061650 con 0x7ff95c0a72d0 2020-11-20T12:21:09.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.894+0000 7ff9497fa700 1 -- 172.21.15.44:0/2495846761 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7ff94c02d020 con 0x7ff95c0a72d0 2020-11-20T12:21:09.898 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:21:09.900 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff934056d20 msgr2=0x7ff9340591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff934056d20 0x7ff9340591e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7ff954006d90 tx=0x7ff954008040).stop 2020-11-20T12:21:09.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0a72d0 msgr2=0x7ff95c138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0a72d0 0x7ff95c138f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7ff95c05c300 tx=0x7ff94c015000).stop 2020-11-20T12:21:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 shutdown_connections 2020-11-20T12:21:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff934056d20 0x7ff9340591e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff95c13c0d0 0x7ff95c13a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff95c0a72d0 0x7ff95c138f10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 --2- 172.21.15.44:0/2495846761 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff95c0abc70 0x7ff95c13a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 >> 172.21.15.44:0/2495846761 conn(0x7ff95c060170 msgr2=0x7ff95c0ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:09.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 shutdown_connections 2020-11-20T12:21:09.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:09.897+0000 7ff963778700 1 -- 172.21.15.44:0/2495846761 wait complete. 2020-11-20T12:21:09.916 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:21:09.916 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:21:09.917 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:21:10.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.102+0000 7f1380c68700 1 Processor -- start 2020-11-20T12:21:10.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.102+0000 7f1380c68700 1 -- start start 2020-11-20T12:21:10.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f1380c68700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f1380c68700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f137c0a9480 0x7f137c142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f1380c68700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137c0a7880 con 0x7f137c0aa1e0 2020-11-20T12:21:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f1380c68700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0a5b00 con 0x7f137c0a9480 2020-11-20T12:21:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f1380c68700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0a5da0 con 0x7f137c0a4330 2020-11-20T12:21:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f137ad9d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f137c0a9480 0x7f137c142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f137a59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f137c0aa1e0 0x7f137c0a37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49952/0 (socket says 172.21.15.44:49952) 2020-11-20T12:21:10.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.103+0000 7f137a59c700 1 -- 172.21.15.44:0/1268254215 learned_addr learned my addr 172.21.15.44:0/1268254215 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:21:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f1379d9b700 1 -- 172.21.15.44:0/1268254215 >> v2:172.21.15.44:3300/0 conn(0x7f137c0a9480 msgr2=0x7f137c142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f1379d9b700 1 --2- 172.21.15.44:0/1268254215 >> v2:172.21.15.44:3300/0 conn(0x7f137c0a9480 0x7f137c142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f1379d9b700 1 -- 172.21.15.44:0/1268254215 >> v1:172.21.15.79:6789/0 conn(0x7f137c0aa1e0 legacy=0x7f137c0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f1379d9b700 1 -- 172.21.15.44:0/1268254215 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0a6040 con 0x7f137c0a4330 2020-11-20T12:21:10.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f1379d9b700 1 --2- 172.21.15.44:0/1268254215 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c1427e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f136c00b9a0 tx=0x7f136c006df0).ready entity=mon.2 client_cookie=48457b68723d7670 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f137959a700 1 -- 172.21.15.44:0/1268254215 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c009840 con 0x7f137c0a4330 2020-11-20T12:21:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f137959a700 1 -- 172.21.15.44:0/1268254215 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c0099a0 con 0x7f137c0a4330 2020-11-20T12:21:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.104+0000 7f137959a700 1 -- 172.21.15.44:0/1268254215 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c009d20 con 0x7f137c0a4330 2020-11-20T12:21:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 -- 172.21.15.44:0/1268254215 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 msgr2=0x7f137c1427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 --2- 172.21.15.44:0/1268254215 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c1427e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f136c00b9a0 tx=0x7f136c006df0).stop 2020-11-20T12:21:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 -- 172.21.15.44:0/1268254215 shutdown_connections 2020-11-20T12:21:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 --2- 172.21.15.44:0/1268254215 >> v2:172.21.15.44:3300/0 conn(0x7f137c0a9480 0x7f137c142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 --2- 172.21.15.44:0/1268254215 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c1427e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 -- 172.21.15.44:0/1268254215 >> 172.21.15.44:0/1268254215 conn(0x7f137c062690 msgr2=0x7f137c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 -- 172.21.15.44:0/1268254215 shutdown_connections 2020-11-20T12:21:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.105+0000 7f1380c68700 1 -- 172.21.15.44:0/1268254215 wait complete. 2020-11-20T12:21:10.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.107+0000 7f1380c68700 1 Processor -- start 2020-11-20T12:21:10.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.107+0000 7f1380c68700 1 -- start start 2020-11-20T12:21:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1380c68700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c067890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1380c68700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a9480 0x7f137c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1380c68700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa1e0 0x7f137c1397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1380c68700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c09f170 con 0x7f137c0a9480 2020-11-20T12:21:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1380c68700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c151e10 con 0x7f137c0aa1e0 2020-11-20T12:21:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1380c68700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0acab0 con 0x7f137c0a4330 2020-11-20T12:21:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a9480 0x7f137c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a9480 0x7f137c0ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43334/0 (socket says 172.21.15.44:43334) 2020-11-20T12:21:10.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f137ad9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa1e0 0x7f137c1397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f137a59c700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c067890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 -- 172.21.15.44:0/417107302 learned_addr learned my addr 172.21.15.44:0/417107302 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 msgr2=0x7f137c067890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c067890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa1e0 msgr2=0x7f137c1397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa1e0 0x7f137c1397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f1379d9b700 1 -- 172.21.15.44:0/417107302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c09f360 con 0x7f137c0a9480 2020-11-20T12:21:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f137ad9d700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f137c0aa1e0 0x7f137c1397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:10.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.108+0000 7f137a59c700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f137c0a4330 0x7f137c067890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.109+0000 7f1379d9b700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a9480 0x7f137c0ad010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f136c006e30 tx=0x7f136c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.109+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c011de0 con 0x7f137c0a9480 2020-11-20T12:21:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.109+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c01e460 con 0x7f137c0a9480 2020-11-20T12:21:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.109+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c09f600 con 0x7f137c0a9480 2020-11-20T12:21:10.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.109+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c01e800 con 0x7f137c0a9480 2020-11-20T12:21:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.109+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c09d470 con 0x7f137c0a9480 2020-11-20T12:21:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.110+0000 7f1380c68700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0ad500 0x7f137c139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.110+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c13a1b0 con 0x7f137c0ad500 2020-11-20T12:21:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.110+0000 7f137a59c700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0ad500 0x7f137c139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.132 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.110+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f136c025030 con 0x7f137c0a9480 2020-11-20T12:21:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.110+0000 7f137a59c700 1 -- 172.21.15.44:0/417107302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f137c09d6f0 con 0x7f137c0ad500 2020-11-20T12:21:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f137a59c700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0ad500 0x7f137c139cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1370004ab0 tx=0x7f1370000b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f136c0719f0 con 0x7f137c0a9480 2020-11-20T12:21:10.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f137ad9d700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137000f9f0 con 0x7f137c0ad500 2020-11-20T12:21:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f137000fb50 con 0x7f137c0ad500 2020-11-20T12:21:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0ad500 msgr2=0x7f137c139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0ad500 0x7f137c139cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1370004ab0 tx=0x7f1370000b80).stop 2020-11-20T12:21:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0ad500 msgr2=0x7f137c139cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f136a7fc700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0ad500 0x7f137c139cc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.111+0000 7f137ad9d700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f137c09f090 tx=0x7f13740091d0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.127+0000 7f1380c68700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aac60 0x7f137c0ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.127+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0ac0b0 con 0x7f137c0aac60 2020-11-20T12:21:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.127+0000 7f137a59c700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aac60 0x7f137c0ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.127+0000 7f137a59c700 1 -- 172.21.15.44:0/417107302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f137c0a9cd0 con 0x7f137c0aac60 2020-11-20T12:21:10.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.127+0000 7f137a59c700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aac60 0x7f137c0ab070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1370017040 tx=0x7f1370009000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.128+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700150d0 con 0x7f137c0aac60 2020-11-20T12:21:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.130+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4779 (secure 0 0 0) 0x7f1370015230 con 0x7f137c0aac60 2020-11-20T12:21:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.130+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aac60 msgr2=0x7f137c0ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.130+0000 7f136a7fc700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aac60 0x7f137c0ab070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1370017040 tx=0x7f1370009000).stop 2020-11-20T12:21:10.141 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.130+0000 7f136a7fc700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aac60 msgr2=0x7f137c0ab070 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.130+0000 7f136a7fc700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0aac60 0x7f137c0ab070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 msgr2=0x7f13500591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f1380c68700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f1350056d20 0x7f13500591e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f137c09f090 tx=0x7f13740091d0).stop 2020-11-20T12:21:10.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a9480 msgr2=0x7f137c0ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f1380c68700 1 --2- 172.21.15.44:0/417107302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c0a9480 0x7f137c0ad010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f136c006e30 tx=0x7f136c015000).stop 2020-11-20T12:21:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f137ad9d700 1 -- 172.21.15.44:0/417107302 reap_dead start 2020-11-20T12:21:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f137ad9d700 1 -- 172.21.15.44:0/417107302 reap_dead start 2020-11-20T12:21:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 shutdown_connections 2020-11-20T12:21:10.143 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.132+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 >> 172.21.15.44:0/417107302 conn(0x7f137c062690 msgr2=0x7f137c142d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.133+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 shutdown_connections 2020-11-20T12:21:10.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.133+0000 7f1380c68700 1 -- 172.21.15.44:0/417107302 wait complete. 2020-11-20T12:21:10.144 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:21:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:21:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:21:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:21:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 176, 2020-11-20T12:21:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:21:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:21:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:21:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:21:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 13, 2020-11-20T12:21:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:21:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:21:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:21:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:21:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:21:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:21:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:21:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:21:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:21:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:21:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:21:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:21:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:21:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:21:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:21:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:21:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:21:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:21:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:21:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:21:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:21:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:21:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:21:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:21:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:21:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:21:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:21:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:21:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:21:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:21:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:21:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:21:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:21:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:21:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:21:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:21:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:21:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:21:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:21:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:21:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:21:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:21:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:21:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:21:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:21:10.184 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:21:10.185 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:21:10.437 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.434+0000 7fb8e359e700 1 Processor -- start 2020-11-20T12:21:10.438 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.434+0000 7fb8e359e700 1 -- start start 2020-11-20T12:21:10.439 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e359e700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb8e40a43d0 0x7fb8e40aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e359e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e407f450 0x7fb8e407f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e359e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb8e40a96c0 0x7fb8e40a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e359e700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40a8450 con 0x7fb8e40a43d0 2020-11-20T12:21:10.440 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e359e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40a8680 con 0x7fb8e40a96c0 2020-11-20T12:21:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e359e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40a6550 con 0x7fb8e407f450 2020-11-20T12:21:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e1d9b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e407f450 0x7fb8e407f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e1d9b700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e407f450 0x7fb8e407f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54786/0 (socket says 172.21.15.44:54786) 2020-11-20T12:21:10.441 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.435+0000 7fb8e1d9b700 1 -- 172.21.15.44:0/2220973850 learned_addr learned my addr 172.21.15.44:0/2220973850 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e259c700 1 --2- 172.21.15.44:0/2220973850 >> v2:172.21.15.44:3300/0 conn(0x7fb8e40a43d0 0x7fb8e40aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e1d9b700 1 -- 172.21.15.44:0/2220973850 >> v2:172.21.15.79:3300/0 conn(0x7fb8e40a96c0 msgr2=0x7fb8e40a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e1d9b700 1 --2- 172.21.15.44:0/2220973850 >> v2:172.21.15.79:3300/0 conn(0x7fb8e40a96c0 0x7fb8e40a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e1d9b700 1 -- 172.21.15.44:0/2220973850 >> v2:172.21.15.44:3300/0 conn(0x7fb8e40a43d0 msgr2=0x7fb8e40aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:10.442 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e1d9b700 1 --2- 172.21.15.44:0/2220973850 >> v2:172.21.15.44:3300/0 conn(0x7fb8e40a43d0 0x7fb8e40aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e1d9b700 1 -- 172.21.15.44:0/2220973850 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40adb50 con 0x7fb8e407f450 2020-11-20T12:21:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e259c700 1 --2- 172.21.15.44:0/2220973850 >> v2:172.21.15.44:3300/0 conn(0x7fb8e40a43d0 0x7fb8e40aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e1d9b700 1 --2- 172.21.15.44:0/2220973850 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e407f450 0x7fb8e407f8d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fb8e40a54d0 tx=0x7fb8d8006eb0).ready entity=mon.2 client_cookie=33d26739a5a4fb93 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e159a700 1 -- 172.21.15.44:0/2220973850 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8009960 con 0x7fb8e407f450 2020-11-20T12:21:10.443 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e159a700 1 -- 172.21.15.44:0/2220973850 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d8009ac0 con 0x7fb8e407f450 2020-11-20T12:21:10.444 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.436+0000 7fb8e159a700 1 -- 172.21.15.44:0/2220973850 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8009e40 con 0x7fb8e407f450 2020-11-20T12:21:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 -- 172.21.15.44:0/2220973850 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e407f450 msgr2=0x7fb8e407f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2220973850 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e407f450 0x7fb8e407f8d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fb8e40a54d0 tx=0x7fb8d8006eb0).stop 2020-11-20T12:21:10.447 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 -- 172.21.15.44:0/2220973850 shutdown_connections 2020-11-20T12:21:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2220973850 >> v2:172.21.15.44:3300/0 conn(0x7fb8e40a43d0 0x7fb8e40aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2220973850 >> v2:172.21.15.79:3300/0 conn(0x7fb8e40a96c0 0x7fb8e40a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2220973850 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e407f450 0x7fb8e407f8d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 -- 172.21.15.44:0/2220973850 >> 172.21.15.44:0/2220973850 conn(0x7fb8e40619a0 msgr2=0x7fb8e4061db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.448 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 -- 172.21.15.44:0/2220973850 shutdown_connections 2020-11-20T12:21:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.437+0000 7fb8e359e700 1 -- 172.21.15.44:0/2220973850 wait complete. 2020-11-20T12:21:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.439+0000 7fb8e359e700 1 Processor -- start 2020-11-20T12:21:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.439+0000 7fb8e359e700 1 -- start start 2020-11-20T12:21:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e359e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e407f450 0x7fb8e41486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.449 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e359e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e40a43d0 0x7fb8e4066ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e359e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e40a96c0 0x7fb8e4066270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e359e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40673e0 con 0x7fb8e407f450 2020-11-20T12:21:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e359e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4064350 con 0x7fb8e40a96c0 2020-11-20T12:21:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e359e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41457a0 con 0x7fb8e40a43d0 2020-11-20T12:21:10.450 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e407f450 0x7fb8e41486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e407f450 0x7fb8e41486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43352/0 (socket says 172.21.15.44:43352) 2020-11-20T12:21:10.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e259c700 1 -- 172.21.15.44:0/2586289406 learned_addr learned my addr 172.21.15.44:0/2586289406 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:10.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e2d9d700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e40a96c0 0x7fb8e4066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.451 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e2d9d700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e40a43d0 msgr2=0x7fb8e4066ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e2d9d700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fb8e40a43d0 0x7fb8e4066ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e2d9d700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e407f450 msgr2=0x7fb8e41486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e2d9d700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8e407f450 0x7fb8e41486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e2d9d700 1 -- 172.21.15.44:0/2586289406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e4145a40 con 0x7fb8e40a96c0 2020-11-20T12:21:10.452 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.440+0000 7fb8e2d9d700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e40a96c0 0x7fb8e4066270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb8dc00bb10 tx=0x7fb8dc0091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.441+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00c830 con 0x7fb8e40a96c0 2020-11-20T12:21:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.441+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00c990 con 0x7fb8e40a96c0 2020-11-20T12:21:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.441+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00cb20 con 0x7fb8e40a96c0 2020-11-20T12:21:10.453 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.442+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e4065dd0 con 0x7fb8e40a96c0 2020-11-20T12:21:10.454 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.442+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e41454c0 con 0x7fb8e40a96c0 2020-11-20T12:21:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.444+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fb8dc01c070 con 0x7fb8e40a96c0 2020-11-20T12:21:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.444+0000 7fb8d27fc700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8b8056d70 0x7fb8b8059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.456 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.444+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fb8dc06f770 con 0x7fb8e40a96c0 2020-11-20T12:21:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.444+0000 7fb8e259c700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8b8056d70 0x7fb8b8059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.445+0000 7fb8e259c700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8b8056d70 0x7fb8b8059230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fb8d400aae0 tx=0x7fb8d4008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.445+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4066760 0x7fb8e407deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.457 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.445+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4061180 con 0x7fb8e4066760 2020-11-20T12:21:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.446+0000 7fb8e1d9b700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4066760 0x7fb8e407deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.446+0000 7fb8e1d9b700 1 -- 172.21.15.44:0/2586289406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8e40abbc0 con 0x7fb8e4066760 2020-11-20T12:21:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.446+0000 7fb8e1d9b700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4066760 0x7fb8e407deb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fb8e40a54d0 tx=0x7fb8d8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.446+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8011d50 con 0x7fb8e4066760 2020-11-20T12:21:10.458 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.447+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fb8d801e460 con 0x7fb8e4066760 2020-11-20T12:21:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.447+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4066760 msgr2=0x7fb8e407deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.447+0000 7fb8d27fc700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4066760 0x7fb8e407deb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fb8e40a54d0 tx=0x7fb8d8015000).stop 2020-11-20T12:21:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.447+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4066760 msgr2=0x7fb8e407deb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:10.459 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.447+0000 7fb8d27fc700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4066760 0x7fb8e407deb0 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0x7fb8e40a54d0 tx=0x7fb8d8015000).stop 2020-11-20T12:21:10.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.460+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e407e3a0 0x7fb8e415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.463 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.460+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4057d80 con 0x7fb8e407e3a0 2020-11-20T12:21:10.464 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.461+0000 7fb8e1d9b700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e407e3a0 0x7fb8e415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8e1d9b700 1 -- 172.21.15.44:0/2586289406 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb8e4065f60 con 0x7fb8e407e3a0 2020-11-20T12:21:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8e1d9b700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e407e3a0 0x7fb8e415e4e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb8e4066b40 tx=0x7fb8d801e000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.465 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d801d8d0 con 0x7fb8e407e3a0 2020-11-20T12:21:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4777 (secure 0 0 0) 0x7fb8d8011b40 con 0x7fb8e407e3a0 2020-11-20T12:21:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e407e3a0 msgr2=0x7fb8e415e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.466 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8d27fc700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e407e3a0 0x7fb8e415e4e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb8e4066b40 tx=0x7fb8d801e000).stop 2020-11-20T12:21:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8d27fc700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e407e3a0 msgr2=0x7fb8e415e4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:10.467 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.462+0000 7fb8d27fc700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e407e3a0 0x7fb8e415e4e0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0x7fb8e4066b40 tx=0x7fb8d801e000).stop 2020-11-20T12:21:10.467 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:21:10.467 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:21:10.468 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:21:10.468 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:21:10.468 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 176, 2020-11-20T12:21:10.468 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:21:10.468 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:21:10.469 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:21:10.469 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:21:10.469 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.469 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:21:10.470 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:10.470 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:21:10.470 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:21:10.470 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.470 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.470 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.471 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.471 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.471 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.471 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.471 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.472 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.472 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:21:10.472 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:21:10.472 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.472 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.473 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.473 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.473 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.473 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.473 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.474 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.474 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.474 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:21:10.474 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:21:10.475 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:21:10.475 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:21:10.475 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:21:10.475 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:21:10.475 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:10.476 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:10.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:21:10.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:21:10.476 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:21:10.476 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:21:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:21:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:21:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:21:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:21:10.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.480 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:21:10.480 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:21:10.480 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.480 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.481 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:21:10.481 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.482 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.482 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:21:10.482 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.483 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.483 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:21:10.483 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:21:10.484 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.484 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.484 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.485 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:21:10.485 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:21:10.485 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.486 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.486 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:21:10.486 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.487 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:21:10.487 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.488 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.488 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:21:10.488 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:21:10.489 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.489 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.489 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.490 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:21:10.490 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:21:10.490 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.490 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.491 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:21:10.491 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.492 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:21:10.492 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.492 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.493 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.493 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:21:10.493 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:21:10.493 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.494 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.494 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.494 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.495 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:21:10.495 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:21:10.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.495 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.495 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:21:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.497 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.497 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.497 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.499 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.499 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:21:10.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.500 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.500 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.500 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:21:10.501 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.501 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.501 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:21:10.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.502 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.502 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.503 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 6 2020-11-20T12:21:10.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.504 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.504 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:21:10.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.504 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.505 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.505 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.505 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.506 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.506 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:21:10.506 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:21:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8b8056d70 msgr2=0x7fb8b8059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fb8b8056d70 0x7fb8b8059230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7fb8d400aae0 tx=0x7fb8d4008040).stop 2020-11-20T12:21:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e40a96c0 msgr2=0x7fb8e4066270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e359e700 1 --2- 172.21.15.44:0/2586289406 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e40a96c0 0x7fb8e4066270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fb8dc00bb10 tx=0x7fb8dc0091a0).stop 2020-11-20T12:21:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e2d9d700 1 -- 172.21.15.44:0/2586289406 reap_dead start 2020-11-20T12:21:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e2d9d700 1 -- 172.21.15.44:0/2586289406 reap_dead start 2020-11-20T12:21:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 shutdown_connections 2020-11-20T12:21:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.464+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 >> 172.21.15.44:0/2586289406 conn(0x7fb8e40619a0 msgr2=0x7fb8e414c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.465+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 shutdown_connections 2020-11-20T12:21:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.466+0000 7fb8e359e700 1 -- 172.21.15.44:0/2586289406 wait complete. 2020-11-20T12:21:10.512 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:21:10.769 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.749+0000 7f62d277e700 1 Processor -- start 2020-11-20T12:21:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.749+0000 7f62d277e700 1 -- start start 2020-11-20T12:21:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62d277e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc0a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62d277e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f62cc138330 0x7f62cc0a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62d277e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc0a94f0 con 0x7f62cc0a30c0 2020-11-20T12:21:10.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62d277e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc0a88c0 con 0x7f62cc138330 2020-11-20T12:21:10.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62d277e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0aa5c0 con 0x7f62cc0aa1e0 2020-11-20T12:21:10.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62cbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc0a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62cbfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc0a80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:54800/0 (socket says 172.21.15.44:54800) 2020-11-20T12:21:10.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62cb7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f62cc0a30c0 0x7f62cc137c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:49986/0 (socket says 172.21.15.44:49986) 2020-11-20T12:21:10.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.750+0000 7f62cbfff700 1 -- 172.21.15.44:0/965574113 learned_addr learned my addr 172.21.15.44:0/965574113 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:10.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62caffd700 1 -- 172.21.15.44:0/965574113 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873496565 0 0) 0x7f62cc0a94f0 con 0x7f62cc0a30c0 2020-11-20T12:21:10.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62cbfff700 1 -- 172.21.15.44:0/965574113 >> v2:172.21.15.79:3300/0 conn(0x7f62cc138330 msgr2=0x7f62cc0a3520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:10.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62cbfff700 1 --2- 172.21.15.44:0/965574113 >> v2:172.21.15.79:3300/0 conn(0x7f62cc138330 0x7f62cc0a3520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:10.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62cbfff700 1 -- 172.21.15.44:0/965574113 >> v1:172.21.15.79:6789/0 conn(0x7f62cc0a30c0 legacy=0x7f62cc137c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62cbfff700 1 -- 172.21.15.44:0/965574113 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0a8d60 con 0x7f62cc0aa1e0 2020-11-20T12:21:10.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62cbfff700 1 --2- 172.21.15.44:0/965574113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc0a80c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f62c0006fd0 tx=0x7f62c0006eb0).ready entity=mon.2 client_cookie=6786d3be6b556301 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62caffd700 1 -- 172.21.15.44:0/965574113 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00098d0 con 0x7f62cc0aa1e0 2020-11-20T12:21:10.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62caffd700 1 -- 172.21.15.44:0/965574113 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0009a30 con 0x7f62cc0aa1e0 2020-11-20T12:21:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.751+0000 7f62caffd700 1 -- 172.21.15.44:0/965574113 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0009db0 con 0x7f62cc0aa1e0 2020-11-20T12:21:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.752+0000 7f62d277e700 1 -- 172.21.15.44:0/965574113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 msgr2=0x7f62cc0a80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.752+0000 7f62d277e700 1 --2- 172.21.15.44:0/965574113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc0a80c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f62c0006fd0 tx=0x7f62c0006eb0).stop 2020-11-20T12:21:10.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.753+0000 7f62d277e700 1 -- 172.21.15.44:0/965574113 shutdown_connections 2020-11-20T12:21:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.753+0000 7f62d277e700 1 --2- 172.21.15.44:0/965574113 >> v2:172.21.15.79:3300/0 conn(0x7f62cc138330 0x7f62cc0a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.753+0000 7f62d277e700 1 --2- 172.21.15.44:0/965574113 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc0a80c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.753+0000 7f62d277e700 1 -- 172.21.15.44:0/965574113 >> 172.21.15.44:0/965574113 conn(0x7f62cc062600 msgr2=0x7f62cc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.753+0000 7f62d277e700 1 -- 172.21.15.44:0/965574113 shutdown_connections 2020-11-20T12:21:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.753+0000 7f62d277e700 1 -- 172.21.15.44:0/965574113 wait complete. 2020-11-20T12:21:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.755+0000 7f62d277e700 1 Processor -- start 2020-11-20T12:21:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.755+0000 7f62d277e700 1 -- start start 2020-11-20T12:21:10.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62d277e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc0a30c0 0x7f62cc067980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62d277e700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62d277e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc138330 0x7f62cc065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62d277e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc066480 con 0x7f62cc0a30c0 2020-11-20T12:21:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62d277e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc1380a0 con 0x7f62cc138330 2020-11-20T12:21:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62d277e700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc14df10 con 0x7f62cc0aa1e0 2020-11-20T12:21:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62cbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc0a30c0 0x7f62cc067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62cbfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc0a30c0 0x7f62cc067980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43366/0 (socket says 172.21.15.44:43366) 2020-11-20T12:21:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62d0d1b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc138330 0x7f62cc065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.756+0000 7f62cbfff700 1 -- 172.21.15.44:0/526858877 learned_addr learned my addr 172.21.15.44:0/526858877 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62d0d1b700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 msgr2=0x7f62cc14e650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2020-11-20T12:21:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62d0d1b700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc0aa1e0 0x7f62cc14e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62d0d1b700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc0a30c0 msgr2=0x7f62cc067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62d0d1b700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62cc0a30c0 0x7f62cc067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62d0d1b700 1 -- 172.21.15.44:0/526858877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc14e1b0 con 0x7f62cc138330 2020-11-20T12:21:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62d0d1b700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc138330 0x7f62cc065ac0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f62c400b8e0 tx=0x7f62c4009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400d750 con 0x7f62cc138330 2020-11-20T12:21:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400d8d0 con 0x7f62cc138330 2020-11-20T12:21:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.757+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400dc90 con 0x7f62cc138330 2020-11-20T12:21:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.758+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc14eb40 con 0x7f62cc138330 2020-11-20T12:21:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.758+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc0a53f0 con 0x7f62cc138330 2020-11-20T12:21:10.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.760+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f62c401e070 con 0x7f62cc138330 2020-11-20T12:21:10.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.760+0000 7f62abfff700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f62a4056d20 0x7f62a40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.760+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f62c4070120 con 0x7f62cc138330 2020-11-20T12:21:10.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.760+0000 7f62cbfff700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f62a4056d20 0x7f62a40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.761+0000 7f62d277e700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc069560 0x7f62cc13b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.761+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc137ea0 con 0x7f62cc069560 2020-11-20T12:21:10.785 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.761+0000 7f62cbfff700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f62a4056d20 0x7f62a40591e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f62cc067250 tx=0x7f62c0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.761+0000 7f62cb7fe700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc069560 0x7f62cc13b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.761+0000 7f62cb7fe700 1 -- 172.21.15.44:0/526858877 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62cc14e340 con 0x7f62cc069560 2020-11-20T12:21:10.786 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.762+0000 7f62cb7fe700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc069560 0x7f62cc13b170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f62cc1375d0 tx=0x7f62bc007980).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.762+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0072e0 con 0x7f62cc069560 2020-11-20T12:21:10.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.762+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f62bc005530 con 0x7f62cc069560 2020-11-20T12:21:10.787 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.762+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc069560 msgr2=0x7f62cc13b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.762+0000 7f62abfff700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc069560 0x7f62cc13b170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f62cc1375d0 tx=0x7f62bc007980).stop 2020-11-20T12:21:10.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.762+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc069560 msgr2=0x7f62cc13b170 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:10.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.762+0000 7f62abfff700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc069560 0x7f62cc13b170 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0x7f62cc1375d0 tx=0x7f62bc007980).stop 2020-11-20T12:21:10.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.776+0000 7f62d277e700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc13b660 0x7f62cc14a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.776+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc061650 con 0x7f62cc13b660 2020-11-20T12:21:10.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.776+0000 7f62cb7fe700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc13b660 0x7f62cc14a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.776+0000 7f62cb7fe700 1 -- 172.21.15.44:0/526858877 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f62cc0653d0 con 0x7f62cc13b660 2020-11-20T12:21:10.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.776+0000 7f62cb7fe700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc13b660 0x7f62cc14a800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f62bc0011a0 tx=0x7f62bc015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0149f0 con 0x7f62cc13b660 2020-11-20T12:21:10.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7f62bc0070d0 con 0x7f62cc13b660 2020-11-20T12:21:10.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc13b660 msgr2=0x7f62cc14a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62abfff700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc13b660 0x7f62cc14a800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f62bc0011a0 tx=0x7f62bc015000).stop 2020-11-20T12:21:10.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62abfff700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc13b660 msgr2=0x7f62cc14a800 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62abfff700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f62cc13b660 0x7f62cc14a800 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0x7f62bc0011a0 tx=0x7f62bc015000).stop 2020-11-20T12:21:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f62a4056d20 msgr2=0x7f62a40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62d277e700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f62a4056d20 0x7f62a40591e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f62cc067250 tx=0x7f62c0015000).stop 2020-11-20T12:21:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc138330 msgr2=0x7f62cc065ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62d277e700 1 --2- 172.21.15.44:0/526858877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62cc138330 0x7f62cc065ac0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f62c400b8e0 tx=0x7f62c4009060).stop 2020-11-20T12:21:10.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62d0d1b700 1 -- 172.21.15.44:0/526858877 reap_dead start 2020-11-20T12:21:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.777+0000 7f62d0d1b700 1 -- 172.21.15.44:0/526858877 reap_dead start 2020-11-20T12:21:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.778+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 shutdown_connections 2020-11-20T12:21:10.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.778+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 >> 172.21.15.44:0/526858877 conn(0x7f62cc062600 msgr2=0x7f62cc140a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.778+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 shutdown_connections 2020-11-20T12:21:10.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:10.778+0000 7f62d277e700 1 -- 172.21.15.44:0/526858877 wait complete. 2020-11-20T12:21:10.798 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:21:10.799 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:21:10.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:21:10.799 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:21:10.800 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 176, 2020-11-20T12:21:10.800 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:21:10.800 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:21:10.800 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:21:10.801 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:21:10.801 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.801 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:21:10.802 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:10.802 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:21:10.802 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:21:10.803 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.803 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.803 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.803 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.804 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.804 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.804 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.805 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.805 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.805 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:21:10.805 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:21:10.806 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.806 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.806 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.807 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.807 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.807 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.808 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.808 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.808 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.809 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:21:10.810 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:21:10.810 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:21:10.810 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:21:10.811 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:21:10.811 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:21:10.811 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:10.812 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:10.812 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:21:10.812 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:21:10.813 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:21:10.813 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:21:10.813 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:21:10.814 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:10.814 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:21:10.814 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:10.814 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:10.815 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:10.816 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:21:10.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:21:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.818 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:21:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:21:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.819 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:21:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.822 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:21:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.823 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:21:10.824 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:21:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.825 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:21:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:21:10.827 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:21:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.830 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:21:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.831 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:21:10.832 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:21:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.833 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.834 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:21:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:21:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:10.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:21:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:10.837 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:21:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:21:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:21:10.839 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.840 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:21:10.841 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:21:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.842 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.843 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:21:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:10.844 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:10.845 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:10.846 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:21:10.847 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:21:10.848 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:21:10.849 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:21:10.849 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:21:10.849 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:21:10.850 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:21:10.850 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:10.863 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:21:10.863 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:21:10.864 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:21:11.042 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.039+0000 7f41c36a0700 1 Processor -- start 2020-11-20T12:21:11.043 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.039+0000 7f41c36a0700 1 -- start start 2020-11-20T12:21:11.050 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c36a0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41bc0a7810 0x7f41bc13c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c36a0700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f41bc066250 0x7f41bc13c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c36a0700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc0a8bf0 con 0x7f41bc0aa1e0 2020-11-20T12:21:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c36a0700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc0a86b0 con 0x7f41bc066250 2020-11-20T12:21:11.051 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c36a0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc0a8950 con 0x7f41bc0a7810 2020-11-20T12:21:11.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c0c3b700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f41bc0a7810 msgr2=0x7f41bc13c170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:11.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f41bc066250 0x7f41bc13c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c0c3b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41bc0a7810 0x7f41bc13c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:11.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f41bc066250 0x7f41bc13c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43414/0 (socket says 172.21.15.44:43414) 2020-11-20T12:21:11.053 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c143c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f41bc0aa1e0 0x7f41bc065b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47158/0 (socket says 172.21.15.44:47158) 2020-11-20T12:21:11.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 -- 172.21.15.44:0/1999294594 learned_addr learned my addr 172.21.15.44:0/1999294594 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:11.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 -- 172.21.15.44:0/1999294594 >> v2:172.21.15.79:3300/0 conn(0x7f41bc0a7810 msgr2=0x7f41bc13c170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:11.054 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 --2- 172.21.15.44:0/1999294594 >> v2:172.21.15.79:3300/0 conn(0x7f41bc0a7810 0x7f41bc13c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:11.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 -- 172.21.15.44:0/1999294594 >> v1:172.21.15.44:6789/0 conn(0x7f41bc0aa1e0 legacy=0x7f41bc065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.055 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 -- 172.21.15.44:0/1999294594 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0aa5c0 con 0x7f41bc066250 2020-11-20T12:21:11.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41abfff700 1 -- 172.21.15.44:0/1999294594 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761717831 0 0) 0x7f41bc0a8bf0 con 0x7f41bc0aa1e0 2020-11-20T12:21:11.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41c1c3d700 1 --2- 172.21.15.44:0/1999294594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc066250 0x7f41bc13c660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f41bc062c40 tx=0x7f41b4009290).ready entity=mon.0 client_cookie=990b8fc027e977ad server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.056 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.040+0000 7f41abfff700 1 -- 172.21.15.44:0/1999294594 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b400bbc0 con 0x7f41bc066250 2020-11-20T12:21:11.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41abfff700 1 -- 172.21.15.44:0/1999294594 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b400bd20 con 0x7f41bc066250 2020-11-20T12:21:11.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41abfff700 1 -- 172.21.15.44:0/1999294594 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4011610 con 0x7f41bc066250 2020-11-20T12:21:11.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 -- 172.21.15.44:0/1999294594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc066250 msgr2=0x7f41bc13c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.057 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 --2- 172.21.15.44:0/1999294594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc066250 0x7f41bc13c660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f41bc062c40 tx=0x7f41b4009290).stop 2020-11-20T12:21:11.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 -- 172.21.15.44:0/1999294594 shutdown_connections 2020-11-20T12:21:11.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 --2- 172.21.15.44:0/1999294594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc066250 0x7f41bc13c660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 --2- 172.21.15.44:0/1999294594 >> v2:172.21.15.79:3300/0 conn(0x7f41bc0a7810 0x7f41bc13c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:11.058 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 -- 172.21.15.44:0/1999294594 >> 172.21.15.44:0/1999294594 conn(0x7f41bc062690 msgr2=0x7f41bc0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:11.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 -- 172.21.15.44:0/1999294594 shutdown_connections 2020-11-20T12:21:11.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.041+0000 7f41c36a0700 1 -- 172.21.15.44:0/1999294594 wait complete. 2020-11-20T12:21:11.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.042+0000 7f41c36a0700 1 Processor -- start 2020-11-20T12:21:11.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.042+0000 7f41c36a0700 1 -- start start 2020-11-20T12:21:11.059 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.043+0000 7f41c36a0700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41bc066250 0x7f41bc13f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c36a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc0a7810 0x7f41bc14d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c36a0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc0aa1e0 0x7f41bc137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.060 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c36a0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc14d9e0 con 0x7f41bc0a7810 2020-11-20T12:21:11.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c36a0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc14dc80 con 0x7f41bc0aa1e0 2020-11-20T12:21:11.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c36a0700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc14df20 con 0x7f41bc066250 2020-11-20T12:21:11.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc0a7810 0x7f41bc14d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c1c3d700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc0aa1e0 msgr2=0x7f41bc137c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:11.061 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c1c3d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc0aa1e0 0x7f41bc137c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:11.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc0a7810 0x7f41bc14d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43418/0 (socket says 172.21.15.44:43418) 2020-11-20T12:21:11.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 -- 172.21.15.44:0/2756689626 learned_addr learned my addr 172.21.15.44:0/2756689626 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:11.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c143c700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41bc066250 0x7f41bc13f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.062 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 -- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41bc066250 msgr2=0x7f41bc13f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41bc066250 0x7f41bc13f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 -- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc0aa1e0 msgr2=0x7f41bc137c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:11.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc0aa1e0 0x7f41bc137c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:11.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 -- 172.21.15.44:0/2756689626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0a6130 con 0x7f41bc0a7810 2020-11-20T12:21:11.064 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c0c3b700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc0a7810 0x7f41bc14d0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f41ac0097c0 tx=0x7f41ac006e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41a97fa700 1 -- 172.21.15.44:0/2756689626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac006170 con 0x7f41bc0a7810 2020-11-20T12:21:11.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41a97fa700 1 -- 172.21.15.44:0/2756689626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac0062f0 con 0x7f41bc0a7810 2020-11-20T12:21:11.065 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.044+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc14d5c0 con 0x7f41bc0a7810 2020-11-20T12:21:11.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.045+0000 7f41a97fa700 1 -- 172.21.15.44:0/2756689626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0038e0 con 0x7f41bc0a7810 2020-11-20T12:21:11.066 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.045+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1382f0 con 0x7f41bc0a7810 2020-11-20T12:21:11.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.046+0000 7f41a97fa700 1 -- 172.21.15.44:0/2756689626 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f41ac017030 con 0x7f41bc0a7810 2020-11-20T12:21:11.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.046+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc149b50 con 0x7f41bc0a7810 2020-11-20T12:21:11.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.046+0000 7f41a97fa700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4194056d00 0x7f41940591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.046+0000 7f41a97fa700 1 -- 172.21.15.44:0/2756689626 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f41ac06f3b0 con 0x7f41bc0a7810 2020-11-20T12:21:11.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.046+0000 7f41c143c700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4194056d00 0x7f41940591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.047+0000 7f41c143c700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4194056d00 0x7f41940591c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f41b8005560 tx=0x7f41b8007180).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.050+0000 7f41a97fa700 1 -- 172.21.15.44:0/2756689626 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f41bc149b50 con 0x7f41bc0a7810 2020-11-20T12:21:11.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.371+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f41bc149b50 con 0x7f41bc0a7810 2020-11-20T12:21:11.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.371+0000 7f41a97fa700 1 -- 172.21.15.44:0/2756689626 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f41bc149b50 con 0x7f41bc0a7810 2020-11-20T12:21:11.376 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:21:11.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4194056d00 msgr2=0x7f41940591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.379 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4194056d00 0x7f41940591c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f41b8005560 tx=0x7f41b8007180).stop 2020-11-20T12:21:11.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc0a7810 msgr2=0x7f41bc14d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.380 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc0a7810 0x7f41bc14d0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f41ac0097c0 tx=0x7f41ac006e60).stop 2020-11-20T12:21:11.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 shutdown_connections 2020-11-20T12:21:11.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f4194056d00 0x7f41940591c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.381 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41bc0aa1e0 0x7f41bc137c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:11.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f41bc066250 0x7f41bc13f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 --2- 172.21.15.44:0/2756689626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41bc0a7810 0x7f41bc14d0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.382 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 >> 172.21.15.44:0/2756689626 conn(0x7f41bc062690 msgr2=0x7f41bc065a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:11.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 shutdown_connections 2020-11-20T12:21:11.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:11.376+0000 7f41c36a0700 1 -- 172.21.15.44:0/2756689626 wait complete. 2020-11-20T12:21:14.399 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:21:14.589 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.586+0000 7f0eac2e1700 1 Processor -- start 2020-11-20T12:21:14.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.586+0000 7f0eac2e1700 1 -- start start 2020-11-20T12:21:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0eac2e1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea40aa020 0x7f0ea40a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.593 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0eac2e1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0ea40aa400 0x7f0ea40a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0eac2e1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea40a9770 con 0x7f0ea413ac60 2020-11-20T12:21:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0eac2e1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea40a6da0 con 0x7f0ea40aa400 2020-11-20T12:21:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0eac2e1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40a7040 con 0x7f0ea40aa020 2020-11-20T12:21:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0ea987c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0ea40aa400 0x7f0ea40a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0eaa87e700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f0ea413ac60 legacy=0x7f0ea40a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:21:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0ea987c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f0ea40aa400 0x7f0ea40a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43432/0 (socket says 172.21.15.44:43432) 2020-11-20T12:21:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0ea987c700 1 -- 172.21.15.44:0/2198760990 learned_addr learned my addr 172.21.15.44:0/2198760990 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.587+0000 7f0eaa07d700 1 --2- 172.21.15.44:0/2198760990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea40aa020 0x7f0ea40a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0eaa07d700 1 -- 172.21.15.44:0/2198760990 >> v2:172.21.15.44:3300/0 conn(0x7f0ea40aa400 msgr2=0x7f0ea40a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:14.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0eaa07d700 1 --2- 172.21.15.44:0/2198760990 >> v2:172.21.15.44:3300/0 conn(0x7f0ea40aa400 0x7f0ea40a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0eaa07d700 1 -- 172.21.15.44:0/2198760990 >> v1:172.21.15.79:6789/0 conn(0x7f0ea413ac60 legacy=0x7f0ea40a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:14.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0eaa07d700 1 -- 172.21.15.44:0/2198760990 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40a6a90 con 0x7f0ea40aa020 2020-11-20T12:21:14.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0ea987c700 1 --2- 172.21.15.44:0/2198760990 >> v2:172.21.15.44:3300/0 conn(0x7f0ea40aa400 0x7f0ea40a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:14.596 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0eaa07d700 1 --2- 172.21.15.44:0/2198760990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea40aa020 0x7f0ea40a9280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0e9c00f150 tx=0x7f0e9c00d670).ready entity=mon.2 client_cookie=f8129bd03b208c5e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0ea907b700 1 -- 172.21.15.44:0/2198760990 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c012070 con 0x7f0ea40aa020 2020-11-20T12:21:14.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0ea907b700 1 -- 172.21.15.44:0/2198760990 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00d2e0 con 0x7f0ea40aa020 2020-11-20T12:21:14.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.588+0000 7f0ea907b700 1 -- 172.21.15.44:0/2198760990 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00fe10 con 0x7f0ea40aa020 2020-11-20T12:21:14.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2198760990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea40aa020 msgr2=0x7f0ea40a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.597 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2198760990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea40aa020 0x7f0ea40a9280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0e9c00f150 tx=0x7f0e9c00d670).stop 2020-11-20T12:21:14.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2198760990 shutdown_connections 2020-11-20T12:21:14.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2198760990 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea40aa020 0x7f0ea40a9280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.598 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2198760990 >> v2:172.21.15.44:3300/0 conn(0x7f0ea40aa400 0x7f0ea40a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2198760990 >> 172.21.15.44:0/2198760990 conn(0x7f0ea4062690 msgr2=0x7f0ea40a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2198760990 shutdown_connections 2020-11-20T12:21:14.599 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.589+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2198760990 wait complete. 2020-11-20T12:21:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.592+0000 7f0eac2e1700 1 Processor -- start 2020-11-20T12:21:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.592+0000 7f0eac2e1700 1 -- start start 2020-11-20T12:21:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eac2e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ea40aa020 0x7f0ea4133140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eac2e1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea40aa400 0x7f0ea40ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eac2e1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea413ac60 0x7f0ea40ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.602 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eac2e1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40655d0 con 0x7f0ea40aa020 2020-11-20T12:21:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eac2e1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4065870 con 0x7f0ea40aa400 2020-11-20T12:21:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eac2e1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4065b10 con 0x7f0ea413ac60 2020-11-20T12:21:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0ea987c700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea40aa400 msgr2=0x7f0ea40ad140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0ea987c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea40aa400 0x7f0ea40ad140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ea40aa020 0x7f0ea4133140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ea40aa020 0x7f0ea4133140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43440/0 (socket says 172.21.15.44:43440) 2020-11-20T12:21:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 -- 172.21.15.44:0/2173262612 learned_addr learned my addr 172.21.15.44:0/2173262612 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa87e700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea413ac60 0x7f0ea40ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 -- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea413ac60 msgr2=0x7f0ea40ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea413ac60 0x7f0ea40ad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 -- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea40aa400 msgr2=0x7f0ea40ad140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea40aa400 0x7f0ea40ad140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.593+0000 7f0eaa07d700 1 -- 172.21.15.44:0/2173262612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4065db0 con 0x7f0ea40aa020 2020-11-20T12:21:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.594+0000 7f0eaa87e700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea413ac60 0x7f0ea40ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.594+0000 7f0eaa07d700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ea40aa020 0x7f0ea4133140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0ea40a7180 tx=0x7f0e9c00d670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.594+0000 7f0e927fc700 1 -- 172.21.15.44:0/2173262612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00b520 con 0x7f0ea40aa020 2020-11-20T12:21:14.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.594+0000 7f0e927fc700 1 -- 172.21.15.44:0/2173262612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00b680 con 0x7f0ea40aa020 2020-11-20T12:21:14.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.594+0000 7f0e927fc700 1 -- 172.21.15.44:0/2173262612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00b830 con 0x7f0ea40aa020 2020-11-20T12:21:14.606 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.595+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea40679f0 con 0x7f0ea40aa020 2020-11-20T12:21:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.595+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea4067ec0 con 0x7f0ea40aa020 2020-11-20T12:21:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.596+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea40681a0 con 0x7f0ea40aa020 2020-11-20T12:21:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.596+0000 7f0e927fc700 1 -- 172.21.15.44:0/2173262612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f0e9c029030 con 0x7f0ea40aa020 2020-11-20T12:21:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.596+0000 7f0e927fc700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e7c056d20 0x7f0e7c0591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.607 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.597+0000 7f0e927fc700 1 -- 172.21.15.44:0/2173262612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0e9c074dc0 con 0x7f0ea40aa020 2020-11-20T12:21:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.597+0000 7f0ea987c700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e7c056d20 0x7f0e7c0591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.597+0000 7f0ea987c700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e7c056d20 0x7f0e7c0591e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f0ea4062aa0 tx=0x7f0ea0006eb0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:14.602+0000 7f0e927fc700 1 -- 172.21.15.44:0/2173262612 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0e9c04a140 con 0x7f0ea40aa020 2020-11-20T12:21:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.000+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0ea40a9770 con 0x7f0ea40aa020 2020-11-20T12:21:15.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.000+0000 7f0e927fc700 1 -- 172.21.15.44:0/2173262612 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f0ea40a9770 con 0x7f0ea40aa020 2020-11-20T12:21:15.121 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:21:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e7c056d20 msgr2=0x7f0e7c0591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e7c056d20 0x7f0e7c0591e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f0ea4062aa0 tx=0x7f0ea0006eb0).stop 2020-11-20T12:21:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ea40aa020 msgr2=0x7f0ea4133140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ea40aa020 0x7f0ea4133140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0ea40a7180 tx=0x7f0e9c00d670).stop 2020-11-20T12:21:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 shutdown_connections 2020-11-20T12:21:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0e7c056d20 0x7f0e7c0591e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f0ea413ac60 0x7f0ea40ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ea40aa020 0x7f0ea4133140 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 --2- 172.21.15.44:0/2173262612 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ea40aa400 0x7f0ea40ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 >> 172.21.15.44:0/2173262612 conn(0x7f0ea4062690 msgr2=0x7f0ea4134890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 shutdown_connections 2020-11-20T12:21:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:15.003+0000 7f0eac2e1700 1 -- 172.21.15.44:0/2173262612 wait complete. 2020-11-20T12:21:17.838 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 3% (311/10047) Checking out files: 4% (402/10047) Checking out files: 5% (503/10047) Checking out files: 5% (589/10047) Checking out files: 6% (603/10047) Checking out files: 7% (704/10047) Checking out files: 8% (804/10047) Checking out files: 8% (821/10047) Checking out files: 9% (905/10047) Checking out files: 10% (1005/10047) Checking out files: 10% (1064/10047) Checking out files: 11% (1106/10047) Checking out files: 12% (1206/10047) Checking out files: 12% (1284/10047) Checking out files: 13% (1307/10047) Checking out files: 14% (1407/10047) Checking out files: 15% (1508/10047) Checking out files: 15% (1558/10047) Checking out files: 16% (1608/10047) Checking out files: 16% (1704/10047) Checking out files: 17% (1708/10047) Checking out files: 18% (1809/10047) Checking out files: 19% (1909/10047) Checking out files: 19% (1930/10047) Checking out files: 20% (2010/10047) Checking out files: 21% (2110/10047) Checking out files: 21% (2118/10047) Checking out files: 22% (2211/10047) Checking out files: 22% (2307/10047) Checking out files: 23% (2311/10047) Checking out files: 24% (2412/10047) Checking out files: 25% (2512/10047) Checking out files: 25% (2560/10047) Checking out files: 26% (2613/10047) Checking out files: 27% (2713/10047) Checking out files: 27% (2813/10047) Checking out files: 28% (2814/10047) Checking out files: 29% (2914/10047) Checking out files: 29% (2989/10047) Checking out files: 30% (3015/10047) Checking out files: 31% (3115/10047) Checking out files: 32% (3216/10047) Checking out files: 32% (3228/10047) Checking out files: 33% (3316/10047) Checking out files: 33% (3406/10047) Checking out files: 34% (3416/10047) Checking out files: 35% (3517/10047) Checking out files: 36% (3617/10047) Checking out files: 36% (3678/10047) Checking out files: 37% (3718/10047) Checking out files: 38% (3818/10047) Checking out files: 39% (3919/10047) Checking out files: 39% (3941/10047) Checking out files: 40% (4019/10047) Checking out files: 40% (4101/10047) Checking out files: 41% (4120/10047) Checking out files: 42% (4220/10047) Checking out files: 42% (4287/10047) Checking out files: 43% (4321/10047) Checking out files: 44% (4421/10047) Checking out files: 44% (4476/10047) Checking out files: 45% (4522/10047) Checking out files: 46% (4622/10047) Checking out files: 47% (4723/10047) Checking out files: 47% (4724/10047) Checking out files: 48% (4823/10047) Checking out files: 49% (4924/10047) Checking out files: 49% (4945/10047) Checking out files: 50% (5024/10047) Checking out files: 51% (5124/10047) Checking out files: 51% (5129/10047) Checking out files: 52% (5225/10047) Checking out files: 53% (5325/10047) Checking out files: 53% (5372/10047) Checking out files: 54% (5426/10047) Checking out files: 55% (5526/10047) Checking out files: 55% (5538/10047) Checking out files: 56% (5627/10047) Checking out files: 57% (5727/10047) Checking out files: 57% (5782/10047) Checking out files: 58% (5828/10047) Checking out files: 59% (5928/10047) Checking out files: 60% (6029/10047) Checking out files: 60% (6046/10047) Checking out files: 60% (6099/10047) Checking out files: 61% (6129/10047) Checking out files: 62% (6230/10047) Checking out files: 63% (6330/10047) Checking out files: 63% (6336/10047) Checking out files: 64% (6431/10047) Checking out files: 64% (6491/10047) Checking out files: 65% (6531/10047) Checking out files: 66% (6632/10047) Checking out files: 66% (6684/10047) Checking out files: 67% (6732/10047) Checking out files: 67% (6808/10047) Checking out files: 68% (6832/10047) Checking out files: 69% (6933/10047) Checking out files: 69% (6956/10047) Checking out files: 70% (7033/10047) Checking out files: 70% (7094/10047) Checking out files: 71% (7134/10047) Checking out files: 71% (7188/10047) Checking out files: 72% (7234/10047) Checking out files: 73% (7335/10047) Checking out files: 73% (7362/10047) Checking out files: 74% (7435/10047) Checking out files: 74% (7530/10047) Checking out files: 75% (7536/10047) Checking out files: 76% (7636/10047) Checking out files: 76% (7677/10047) Checking out files: 77% (7737/10047) Checking out files: 78% (7837/10047) Checking out files: 78% (7847/10047) Checking out files: 78% (7892/10047) Checking out files: 79% (7938/10047) Checking out files: 80% (8038/10047) Checking out files: 80% (8094/10047) Checking out files: 81% (8139/10047) Checking out files: 82% (8239/10047) Checking out files: 82% (8262/10047) Checking out files: 82% (8292/10047) Checking out files: 83% (8340/10047) Checking out files: 84% (8440/10047) Checking out files: 84% (8499/10047) Checking out files: 85% (8540/10047) Checking out files: 86% (8641/10047) Checking out files: 86% (8678/10047) Checking out files: 87% (8741/10047) Checking out files: 88% (8842/10047) Checking out files: 88% (8884/10047) Checking out files: 89% (8942/10047) Checking out files: 90% (9043/10047) Checking out files: 90% (9075/10047) Checking out files: 91% (9143/10047) Checking out files: 92% (9244/10047) Checking out files: 92% (9292/10047) Checking out files: 93% (9344/10047) Checking out files: 94% (9445/10047) Checking out files: 94% (9495/10047) Checking out files: 95% (9545/10047) Checking out files: 96% (9646/10047) Checking out files: 96% (9674/10047) Checking out files: 97% (9746/10047) Checking out files: 98% (9847/10047) Checking out files: 98% (9910/10047) Checking out files: 99% (9947/10047) Checking out files: 100% (10047/10047) Checking out files: 100% (10047/10047), done. 2020-11-20T12:21:17.883 INFO:tasks.workunit.client.0.smithi044.stderr:+ cd ceph 2020-11-20T12:21:17.885 INFO:tasks.workunit.client.0.smithi044.stderr:++ seq 1 21 2020-11-20T12:21:17.886 INFO:tasks.workunit.client.0.smithi044.stderr:+ versions='1 2020-11-20T12:21:17.887 INFO:tasks.workunit.client.0.smithi044.stderr:2 2020-11-20T12:21:17.887 INFO:tasks.workunit.client.0.smithi044.stderr:3 2020-11-20T12:21:17.887 INFO:tasks.workunit.client.0.smithi044.stderr:4 2020-11-20T12:21:17.887 INFO:tasks.workunit.client.0.smithi044.stderr:5 2020-11-20T12:21:17.887 INFO:tasks.workunit.client.0.smithi044.stderr:6 2020-11-20T12:21:17.888 INFO:tasks.workunit.client.0.smithi044.stderr:7 2020-11-20T12:21:17.888 INFO:tasks.workunit.client.0.smithi044.stderr:8 2020-11-20T12:21:17.888 INFO:tasks.workunit.client.0.smithi044.stderr:9 2020-11-20T12:21:17.888 INFO:tasks.workunit.client.0.smithi044.stderr:10 2020-11-20T12:21:17.889 INFO:tasks.workunit.client.0.smithi044.stderr:11 2020-11-20T12:21:17.889 INFO:tasks.workunit.client.0.smithi044.stderr:12 2020-11-20T12:21:17.889 INFO:tasks.workunit.client.0.smithi044.stderr:13 2020-11-20T12:21:17.890 INFO:tasks.workunit.client.0.smithi044.stderr:14 2020-11-20T12:21:17.890 INFO:tasks.workunit.client.0.smithi044.stderr:15 2020-11-20T12:21:17.890 INFO:tasks.workunit.client.0.smithi044.stderr:16 2020-11-20T12:21:17.890 INFO:tasks.workunit.client.0.smithi044.stderr:17 2020-11-20T12:21:17.890 INFO:tasks.workunit.client.0.smithi044.stderr:18 2020-11-20T12:21:17.891 INFO:tasks.workunit.client.0.smithi044.stderr:19 2020-11-20T12:21:17.891 INFO:tasks.workunit.client.0.smithi044.stderr:20 2020-11-20T12:21:17.891 INFO:tasks.workunit.client.0.smithi044.stderr:21' 2020-11-20T12:21:17.891 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:21:17.891 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.1 2020-11-20T12:21:17.892 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.1 2020-11-20T12:21:17.892 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.1 2020-11-20T12:21:17.892 INFO:tasks.workunit.client.0.smithi044.stdout:v0.1 2020-11-20T12:21:18.126 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:21:18.273 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.270+0000 7fa82939f700 1 Processor -- start 2020-11-20T12:21:18.274 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.270+0000 7fa82939f700 1 -- start start 2020-11-20T12:21:18.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa82939f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa82413e0c0 0x7fa824146470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:18.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa82939f700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8240aa6a0 con 0x7fa8240b1b50 2020-11-20T12:21:18.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa82939f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8240ab0f0 con 0x7fa8240b1770 2020-11-20T12:21:18.276 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa82939f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8240ab390 con 0x7fa82413e0c0 2020-11-20T12:21:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa822ffd700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fa8240b1770 legacy=0x7fa8240a9170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:21:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa8227fc700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa8240b1b50 0x7fa824137ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47196/0 (socket says 172.21.15.44:47196) 2020-11-20T12:21:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa8237fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa82413e0c0 0x7fa824146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa8227fc700 1 -- 172.21.15.44:0/1475420440 learned_addr learned my addr 172.21.15.44:0/1475420440 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:21:18.277 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa8237fe700 1 -- 172.21.15.44:0/1475420440 >> v1:172.21.15.79:6789/0 conn(0x7fa8240b1770 legacy=0x7fa8240a9170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa8237fe700 1 -- 172.21.15.44:0/1475420440 >> v1:172.21.15.44:6789/0 conn(0x7fa8240b1b50 legacy=0x7fa824137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.271+0000 7fa8237fe700 1 -- 172.21.15.44:0/1475420440 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8240a8cd0 con 0x7fa82413e0c0 2020-11-20T12:21:18.278 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa8237fe700 1 --2- 172.21.15.44:0/1475420440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82413e0c0 0x7fa824146470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fa81c009c90 tx=0x7fa81c0091d0).ready entity=mon.0 client_cookie=abdb99cb8d3054cc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa821ffb700 1 -- 172.21.15.44:0/1475420440 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00bb80 con 0x7fa82413e0c0 2020-11-20T12:21:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa821ffb700 1 -- 172.21.15.44:0/1475420440 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c00bce0 con 0x7fa82413e0c0 2020-11-20T12:21:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa821ffb700 1 -- 172.21.15.44:0/1475420440 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c011610 con 0x7fa82413e0c0 2020-11-20T12:21:18.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa82939f700 1 -- 172.21.15.44:0/1475420440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82413e0c0 msgr2=0x7fa824146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa82939f700 1 --2- 172.21.15.44:0/1475420440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82413e0c0 0x7fa824146470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fa81c009c90 tx=0x7fa81c0091d0).stop 2020-11-20T12:21:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa82939f700 1 -- 172.21.15.44:0/1475420440 shutdown_connections 2020-11-20T12:21:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa82939f700 1 --2- 172.21.15.44:0/1475420440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82413e0c0 0x7fa824146470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.272+0000 7fa82939f700 1 -- 172.21.15.44:0/1475420440 >> 172.21.15.44:0/1475420440 conn(0x7fa824060c40 msgr2=0x7fa8240b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:18.280 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.273+0000 7fa82939f700 1 -- 172.21.15.44:0/1475420440 shutdown_connections 2020-11-20T12:21:18.281 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.273+0000 7fa82939f700 1 -- 172.21.15.44:0/1475420440 wait complete. 2020-11-20T12:21:18.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 Processor -- start 2020-11-20T12:21:18.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 -- start start 2020-11-20T12:21:18.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8240b1770 0x7fa824131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:18.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8240b1b50 0x7fa8241355c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:18.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa824130810 0x7fa824130c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:18.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82413eb10 con 0x7fa8240b1b50 2020-11-20T12:21:18.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa824131180 con 0x7fa824130810 2020-11-20T12:21:18.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa82939f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa824133460 con 0x7fa8240b1770 2020-11-20T12:21:18.285 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.274+0000 7fa8237fe700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa824130810 msgr2=0x7fa824130c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:18.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8240b1b50 0x7fa8241355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:18.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8237fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa824130810 0x7fa824130c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:18.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8240b1b50 0x7fa8241355c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43454/0 (socket says 172.21.15.44:43454) 2020-11-20T12:21:18.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 -- 172.21.15.44:0/3495414362 learned_addr learned my addr 172.21.15.44:0/3495414362 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:18.286 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa822ffd700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8240b1770 0x7fa824131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:18.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 -- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8240b1770 msgr2=0x7fa824131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:18.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8240b1770 0x7fa824131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 -- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa824130810 msgr2=0x7fa824130c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:18.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa824130810 0x7fa824130c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:18.287 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 -- 172.21.15.44:0/3495414362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8240a6d30 con 0x7fa8240b1b50 2020-11-20T12:21:18.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa8227fc700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8240b1b50 0x7fa8241355c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa81800c710 tx=0x7fa818007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:18.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa80affd700 1 -- 172.21.15.44:0/3495414362 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818007320 con 0x7fa8240b1b50 2020-11-20T12:21:18.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa80affd700 1 -- 172.21.15.44:0/3495414362 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa818007480 con 0x7fa8240b1b50 2020-11-20T12:21:18.288 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8240a6fd0 con 0x7fa8240b1b50 2020-11-20T12:21:18.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa80affd700 1 -- 172.21.15.44:0/3495414362 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818005800 con 0x7fa8240b1b50 2020-11-20T12:21:18.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.275+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa824135ab0 con 0x7fa8240b1b50 2020-11-20T12:21:18.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.277+0000 7fa80affd700 1 -- 172.21.15.44:0/3495414362 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa818010030 con 0x7fa8240b1b50 2020-11-20T12:21:18.289 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.277+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa824057d80 con 0x7fa8240b1b50 2020-11-20T12:21:18.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.277+0000 7fa80affd700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa800056cd0 0x7fa800059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:18.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.277+0000 7fa80affd700 1 -- 172.21.15.44:0/3495414362 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa81806fd20 con 0x7fa8240b1b50 2020-11-20T12:21:18.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.277+0000 7fa822ffd700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa800056cd0 0x7fa800059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:18.290 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.278+0000 7fa822ffd700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa800056cd0 0x7fa800059190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fa814006fd0 tx=0x7fa814006dd0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:18.291 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.281+0000 7fa80affd700 1 -- 172.21.15.44:0/3495414362 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa818044f90 con 0x7fa8240b1b50 2020-11-20T12:21:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.545+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa824062de0 con 0x7fa8240b1b50 2020-11-20T12:21:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.546+0000 7fa80affd700 1 -- 172.21.15.44:0/3495414362 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fa8180485b0 con 0x7fa8240b1b50 2020-11-20T12:21:18.549 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:21:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa800056cd0 msgr2=0x7fa800059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa800056cd0 0x7fa800059190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fa814006fd0 tx=0x7fa814006dd0).stop 2020-11-20T12:21:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8240b1b50 msgr2=0x7fa8241355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8240b1b50 0x7fa8241355c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa81800c710 tx=0x7fa818007c00).stop 2020-11-20T12:21:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 shutdown_connections 2020-11-20T12:21:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa800056cd0 0x7fa800059190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa824130810 0x7fa824130c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa8240b1770 0x7fa824131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 --2- 172.21.15.44:0/3495414362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8240b1b50 0x7fa8241355c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 >> 172.21.15.44:0/3495414362 conn(0x7fa824060c40 msgr2=0x7fa8241372c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:18.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.547+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 shutdown_connections 2020-11-20T12:21:18.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:18.548+0000 7fa82939f700 1 -- 172.21.15.44:0/3495414362 wait complete. 2020-11-20T12:21:21.566 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:21:21.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.704+0000 7f203acda700 1 Processor -- start 2020-11-20T12:21:21.708 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.704+0000 7f203acda700 1 -- start start 2020-11-20T12:21:21.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f203acda700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f203acda700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20340aa9d0 con 0x7f2034067a60 2020-11-20T12:21:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f203acda700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2034067ee0 con 0x7f20340a9290 2020-11-20T12:21:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f203acda700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340a9740 con 0x7f2034061340 2020-11-20T12:21:21.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f2039277700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f20340a9290 legacy=0x7f20340a8780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:21:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f2033fff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2034067a60 0x7f20340a8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47212/0 (socket says 172.21.15.44:47212) 2020-11-20T12:21:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 learned_addr learned my addr 172.21.15.44:0/2811380604 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:21:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.705+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.706+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:21:21.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.706+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:21:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.706+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:21:21.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.706+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:21.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.906+0000 7f2039277700 1 -- 172.21.15.44:0/2811380604 >> v1:172.21.15.79:6789/0 conn(0x7f20340a9290 legacy=0x7f20340a8780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:21:21.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.909+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.909+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:21:21.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.909+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:21:21.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.909+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:21:21.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:21.909+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:21:22.310 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:22.307+0000 7f2039277700 1 -- 172.21.15.44:0/2811380604 >> v1:172.21.15.79:6789/0 conn(0x7f20340a9290 legacy=0x7f20340a8780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:21:22.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:22.312+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:22.315 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:22.312+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:21:22.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:22.312+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:21:22.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:22.312+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:21:22.316 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:22.312+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:21:23.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:23.109+0000 7f2039277700 1 -- 172.21.15.44:0/2811380604 >> v1:172.21.15.79:6789/0 conn(0x7f20340a9290 legacy=0x7f20340a8780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:21:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:23.113+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:23.113+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:21:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:23.113+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:21:23.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:23.113+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:21:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:23.113+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:21:24.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 msgr2=0x7f20340674a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:24.709 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2034061340 0x7f20340674a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 -- 172.21.15.44:0/2811380604 >> v1:172.21.15.79:6789/0 conn(0x7f20340a9290 legacy=0x7f20340a8780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 -- 172.21.15.44:0/2811380604 >> v1:172.21.15.44:6789/0 conn(0x7f2034067a60 legacy=0x7f20340a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 0x7f201c005840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.710 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 -- 172.21.15.44:0/2811380604 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20340a9740 con 0x7f201c005ed0 2020-11-20T12:21:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.705+0000 7f20327fc700 1 -- 172.21.15.44:0/2811380604 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201c0089f0 con 0x7f201c0033c0 2020-11-20T12:21:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.706+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 msgr2=0x7f201c005840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.706+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 0x7f201c005840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:24.711 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.706+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.706+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 msgr2=0x7f201c008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:21:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.706+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 msgr2=0x7f201c008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:21:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.706+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:21:24.712 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.706+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:24.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.907+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 msgr2=0x7f201c005840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:24.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.907+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.907+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 0x7f201c005840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:21:24.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.907+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 msgr2=0x7f201c008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:21:24.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.907+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 msgr2=0x7f201c008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:21:24.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.907+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:21:24.912 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:24.907+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:21:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:25.308+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 msgr2=0x7f201c005840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:25.308+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:25.308+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 0x7f201c005840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:21:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:25.308+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 msgr2=0x7f201c008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:21:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:25.308+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 msgr2=0x7f201c008390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:21:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:25.308+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:21:25.313 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:25.308+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:21:26.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.109+0000 7f2038a76700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 msgr2=0x7f201c005840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.109+0000 7f2038a76700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 0x7f201c005840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:21:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.109+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.44:3300/0 conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.109+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 msgr2=0x7f201c005840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:26.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.109+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> v2:172.21.15.79:3300/0 conn(0x7f201c0033c0 0x7f201c005840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.109+0000 7f2033fff700 1 -- 172.21.15.44:0/2811380604 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340a8e90 con 0x7f201c005ed0 2020-11-20T12:21:26.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.110+0000 7f2033fff700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f201c005ed0 0x7f201c008390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f201c003330 tx=0x7f2020009040).ready entity=mon.0 client_cookie=395dbdeb8d5328 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.110+0000 7f20337fe700 1 -- 172.21.15.44:0/2811380604 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020014550 con 0x7f201c005ed0 2020-11-20T12:21:26.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.110+0000 7f20337fe700 1 -- 172.21.15.44:0/2811380604 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20200146b0 con 0x7f201c005ed0 2020-11-20T12:21:26.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.110+0000 7f20337fe700 1 -- 172.21.15.44:0/2811380604 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020014a30 con 0x7f201c005ed0 2020-11-20T12:21:26.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f203acda700 1 -- 172.21.15.44:0/2811380604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f201c005ed0 msgr2=0x7f201c008390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f203acda700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f201c005ed0 0x7f201c008390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f201c003330 tx=0x7f2020009040).stop 2020-11-20T12:21:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f2039277700 1 -- 172.21.15.44:0/2811380604 reap_dead start 2020-11-20T12:21:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f203acda700 1 -- 172.21.15.44:0/2811380604 shutdown_connections 2020-11-20T12:21:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f203acda700 1 --2- 172.21.15.44:0/2811380604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f201c005ed0 0x7f201c008390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f203acda700 1 -- 172.21.15.44:0/2811380604 >> 172.21.15.44:0/2811380604 conn(0x7f20340ae1a0 msgr2=0x7f20340aa170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f203acda700 1 -- 172.21.15.44:0/2811380604 shutdown_connections 2020-11-20T12:21:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.111+0000 7f203acda700 1 -- 172.21.15.44:0/2811380604 wait complete. 2020-11-20T12:21:26.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.112+0000 7f203acda700 1 Processor -- start 2020-11-20T12:21:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.112+0000 7f203acda700 1 -- start start 2020-11-20T12:21:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f203acda700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f201c005ed0 0x7f203413b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f203acda700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20340a3b00 0x7f203413e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f203acda700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f203413e640 0x7f203414d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f203acda700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203413eac0 con 0x7f20340a3b00 2020-11-20T12:21:26.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f203acda700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20340a1c00 con 0x7f203413e640 2020-11-20T12:21:26.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f203acda700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340a1ea0 con 0x7f201c005ed0 2020-11-20T12:21:26.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f2033fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20340a3b00 0x7f203413e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f2039277700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f203413e640 msgr2=0x7f203414d700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:26.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f2039277700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f203413e640 0x7f203414d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f2033fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20340a3b00 0x7f203413e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43498/0 (socket says 172.21.15.44:43498) 2020-11-20T12:21:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.113+0000 7f2033fff700 1 -- 172.21.15.44:0/3311922083 learned_addr learned my addr 172.21.15.44:0/3311922083 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2038a76700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f201c005ed0 0x7f203413b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2033fff700 1 -- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f201c005ed0 msgr2=0x7f203413b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2033fff700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f201c005ed0 0x7f203413b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2033fff700 1 -- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f203413e640 msgr2=0x7f203414d700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2033fff700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f203413e640 0x7f203414d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2033fff700 1 -- 172.21.15.44:0/3311922083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340a2140 con 0x7f20340a3b00 2020-11-20T12:21:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2033fff700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20340a3b00 0x7f203413e150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f2034066da0 tx=0x7f2020009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2030ff9700 1 -- 172.21.15.44:0/3311922083 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020005240 con 0x7f20340a3b00 2020-11-20T12:21:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2030ff9700 1 -- 172.21.15.44:0/3311922083 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20200053a0 con 0x7f20340a3b00 2020-11-20T12:21:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20340a1880 con 0x7f20340a3b00 2020-11-20T12:21:26.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f2030ff9700 1 -- 172.21.15.44:0/3311922083 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020016af0 con 0x7f20340a3b00 2020-11-20T12:21:26.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.114+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203413d550 con 0x7f20340a3b00 2020-11-20T12:21:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.116+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20340658f0 con 0x7f20340a3b00 2020-11-20T12:21:26.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.116+0000 7f2030ff9700 1 -- 172.21.15.44:0/3311922083 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2020013040 con 0x7f20340a3b00 2020-11-20T12:21:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.116+0000 7f2030ff9700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f200c056d00 0x7f200c0591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.116+0000 7f2030ff9700 1 -- 172.21.15.44:0/3311922083 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f20200721c0 con 0x7f20340a3b00 2020-11-20T12:21:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.117+0000 7f2038a76700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f200c056d00 0x7f200c0591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.117+0000 7f2038a76700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f200c056d00 0x7f200c0591c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f2028006c00 tx=0x7f2028008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.120+0000 7f2030ff9700 1 -- 172.21.15.44:0/3311922083 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2020047a80 con 0x7f20340a3b00 2020-11-20T12:21:26.373 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.370+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f20340658f0 con 0x7f20340a3b00 2020-11-20T12:21:26.374 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.370+0000 7f2030ff9700 1 -- 172.21.15.44:0/3311922083 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7f202004aab0 con 0x7f20340a3b00 2020-11-20T12:21:26.374 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":180,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:21:26.375 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f200c056d00 msgr2=0x7f200c0591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f200c056d00 0x7f200c0591c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f2028006c00 tx=0x7f2028008040).stop 2020-11-20T12:21:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20340a3b00 msgr2=0x7f203413e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20340a3b00 0x7f203413e150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f2034066da0 tx=0x7f2020009040).stop 2020-11-20T12:21:26.376 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 shutdown_connections 2020-11-20T12:21:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f200c056d00 0x7f200c0591c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f203413e640 0x7f203414d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f201c005ed0 0x7f203413b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 --2- 172.21.15.44:0/3311922083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20340a3b00 0x7f203413e150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.377 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 >> 172.21.15.44:0/3311922083 conn(0x7f20340ae1a0 msgr2=0x7f20340acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 shutdown_connections 2020-11-20T12:21:26.378 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.372+0000 7f203acda700 1 -- 172.21.15.44:0/3311922083 wait complete. 2020-11-20T12:21:26.386 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:21:26.387 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:21:26.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 Processor -- start 2020-11-20T12:21:26.514 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 -- start start 2020-11-20T12:21:26.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03a8060330 0x7f03a80a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a80a4c30 0x7f03a813a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f03a813ae80 0x7f03a8136f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a80a52e0 con 0x7f03a8060330 2020-11-20T12:21:26.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a80a6260 con 0x7f03a813ae80 2020-11-20T12:21:26.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.511+0000 7f03ad615700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80a6490 con 0x7f03a80a4c30 2020-11-20T12:21:26.523 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03a8060330 0x7f03a80a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a77fe700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f03a813ae80 msgr2=0x7f03a8136f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f03a8060330 0x7f03a80a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43504/0 (socket says 172.21.15.44:43504) 2020-11-20T12:21:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 -- 172.21.15.44:0/3106589858 learned_addr learned my addr 172.21.15.44:0/3106589858 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a77fe700 1 --2- 172.21.15.44:0/3106589858 >> v2:172.21.15.79:3300/0 conn(0x7f03a813ae80 0x7f03a8136f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:26.524 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a67fc700 1 --2- 172.21.15.44:0/3106589858 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a80a4c30 0x7f03a813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 -- 172.21.15.44:0/3106589858 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a80a4c30 msgr2=0x7f03a813a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:26.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 --2- 172.21.15.44:0/3106589858 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a80a4c30 0x7f03a813a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.525 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 -- 172.21.15.44:0/3106589858 >> v2:172.21.15.79:3300/0 conn(0x7f03a813ae80 msgr2=0x7f03a8136f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:26.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 --2- 172.21.15.44:0/3106589858 >> v2:172.21.15.79:3300/0 conn(0x7f03a813ae80 0x7f03a8136f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:26.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 -- 172.21.15.44:0/3106589858 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a80a6730 con 0x7f03a8060330 2020-11-20T12:21:26.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a67fc700 1 --2- 172.21.15.44:0/3106589858 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a80a4c30 0x7f03a813a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:26.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a6ffd700 1 --2- 172.21.15.44:0/3106589858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80a4740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f03a8061220 tx=0x7f0398006eb0).ready entity=mon.0 client_cookie=26ed2c269e27639 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a5ffb700 1 -- 172.21.15.44:0/3106589858 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980098d0 con 0x7f03a8060330 2020-11-20T12:21:26.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a5ffb700 1 -- 172.21.15.44:0/3106589858 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0398009a30 con 0x7f03a8060330 2020-11-20T12:21:26.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03a5ffb700 1 -- 172.21.15.44:0/3106589858 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398009db0 con 0x7f03a8060330 2020-11-20T12:21:26.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03ad615700 1 -- 172.21.15.44:0/3106589858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 msgr2=0x7f03a80a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.512+0000 7f03ad615700 1 --2- 172.21.15.44:0/3106589858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80a4740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f03a8061220 tx=0x7f0398006eb0).stop 2020-11-20T12:21:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.513+0000 7f03ad615700 1 -- 172.21.15.44:0/3106589858 shutdown_connections 2020-11-20T12:21:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.513+0000 7f03ad615700 1 --2- 172.21.15.44:0/3106589858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80a4740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.513+0000 7f03ad615700 1 --2- 172.21.15.44:0/3106589858 >> v2:172.21.15.79:3300/0 conn(0x7f03a813ae80 0x7f03a8136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.513+0000 7f03ad615700 1 --2- 172.21.15.44:0/3106589858 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a80a4c30 0x7f03a813a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.513+0000 7f03ad615700 1 -- 172.21.15.44:0/3106589858 >> 172.21.15.44:0/3106589858 conn(0x7f03a8062690 msgr2=0x7f03a80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.513+0000 7f03ad615700 1 -- 172.21.15.44:0/3106589858 shutdown_connections 2020-11-20T12:21:26.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.513+0000 7f03ad615700 1 -- 172.21.15.44:0/3106589858 wait complete. 2020-11-20T12:21:26.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 Processor -- start 2020-11-20T12:21:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 -- start start 2020-11-20T12:21:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a80a4c30 0x7f03a80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a813ae80 0x7f03a812c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80add30 con 0x7f03a8060330 2020-11-20T12:21:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8130120 con 0x7f03a80a4c30 2020-11-20T12:21:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03ad615700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80acde0 con 0x7f03a813ae80 2020-11-20T12:21:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03a6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03a67fc700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a80a4c30 msgr2=0x7f03a80ad840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:26.532 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03a6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43510/0 (socket says 172.21.15.44:43510) 2020-11-20T12:21:26.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03a6ffd700 1 -- 172.21.15.44:0/337552578 learned_addr learned my addr 172.21.15.44:0/337552578 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:26.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.514+0000 7f03a67fc700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a80a4c30 0x7f03a80ad840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:26.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03a77fe700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a813ae80 0x7f03a812c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03a6ffd700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a813ae80 msgr2=0x7f03a812c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03a6ffd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f03a813ae80 0x7f03a812c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.537 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03a6ffd700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a80a4c30 msgr2=0x7f03a80ad840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03a6ffd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03a80a4c30 0x7f03a80ad840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03a6ffd700 1 -- 172.21.15.44:0/337552578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a80ad080 con 0x7f03a8060330 2020-11-20T12:21:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03a6ffd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80ab500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f0398006ef0 tx=0x7f0398015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039801e460 con 0x7f03a8060330 2020-11-20T12:21:26.538 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039801e5c0 con 0x7f03a8060330 2020-11-20T12:21:26.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a80ad520 con 0x7f03a8060330 2020-11-20T12:21:26.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039801e980 con 0x7f03a8060330 2020-11-20T12:21:26.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.515+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a812c7d0 con 0x7f03a8060330 2020-11-20T12:21:26.539 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.516+0000 7f03ad615700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812fba0 0x7f03a8129f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.516+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a812ff80 con 0x7f03a812fba0 2020-11-20T12:21:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.516+0000 7f03a67fc700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812fba0 0x7f03a8129f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.516+0000 7f03a67fc700 1 -- 172.21.15.44:0/337552578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a8061650 con 0x7f03a812fba0 2020-11-20T12:21:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.516+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f039801ed70 con 0x7f03a8060330 2020-11-20T12:21:26.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.516+0000 7f03a67fc700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812fba0 0x7f03a8129f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f03a80a4000 tx=0x7f0390008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0384056cd0 0x7f0384059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f0398071d00 con 0x7f03a8060330 2020-11-20T12:21:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000ce10 con 0x7f03a812fba0 2020-11-20T12:21:26.541 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f03900077a0 con 0x7f03a812fba0 2020-11-20T12:21:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812fba0 msgr2=0x7f03a8129f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812fba0 0x7f03a8129f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f03a80a4000 tx=0x7f0390008040).stop 2020-11-20T12:21:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812fba0 msgr2=0x7f03a8129f50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f039effd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812fba0 0x7f03a8129f50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.542 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.517+0000 7f03a77fe700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0384056cd0 0x7f0384059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.543 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.518+0000 7f03a77fe700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0384056cd0 0x7f0384059190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f03a0004fd0 tx=0x7f03a0006f00).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.525+0000 7f03ad615700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812a440 0x7f03a814b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.525+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80446d0 con 0x7f03a812a440 2020-11-20T12:21:26.545 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.525+0000 7f03a67fc700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812a440 0x7f03a814b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.525+0000 7f03a67fc700 1 -- 172.21.15.44:0/337552578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03a80680f0 con 0x7f03a812a440 2020-11-20T12:21:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03a67fc700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812a440 0x7f03a814b230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0390006fd0 tx=0x7f0390008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039001ad80 con 0x7f03a812a440 2020-11-20T12:21:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4767 (secure 0 0 0) 0x7f0390007590 con 0x7f03a812a440 2020-11-20T12:21:26.546 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812a440 msgr2=0x7f03a814b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f039effd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812a440 0x7f03a814b230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0390006fd0 tx=0x7f0390008040).stop 2020-11-20T12:21:26.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f039effd700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812a440 msgr2=0x7f03a814b230 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:26.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f039effd700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a812a440 0x7f03a814b230 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0x7f0390006fd0 tx=0x7f0390008040).stop 2020-11-20T12:21:26.547 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0384056cd0 msgr2=0x7f0384059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f0384056cd0 0x7f0384059190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f03a0004fd0 tx=0x7f03a0006f00).stop 2020-11-20T12:21:26.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 msgr2=0x7f03a80ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 --2- 172.21.15.44:0/337552578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03a8060330 0x7f03a80ab500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f0398006ef0 tx=0x7f0398015000).stop 2020-11-20T12:21:26.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03a77fe700 1 -- 172.21.15.44:0/337552578 reap_dead start 2020-11-20T12:21:26.548 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03a77fe700 1 -- 172.21.15.44:0/337552578 reap_dead start 2020-11-20T12:21:26.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 shutdown_connections 2020-11-20T12:21:26.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 >> 172.21.15.44:0/337552578 conn(0x7f03a8062690 msgr2=0x7f03a81394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 shutdown_connections 2020-11-20T12:21:26.549 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.526+0000 7f03ad615700 1 -- 172.21.15.44:0/337552578 wait complete. 2020-11-20T12:21:26.550 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:21:26.550 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:21:26.550 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:21:26.550 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:21:26.550 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 180, 2020-11-20T12:21:26.551 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:21:26.551 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:21:26.551 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:21:26.551 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.551 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:21:26.552 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:26.552 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:21:26.552 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:21:26.552 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:26.552 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:26.553 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:26.553 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:26.553 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:26.553 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:26.554 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:26.554 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:26.554 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.554 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:21:26.554 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:21:26.555 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:26.555 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:26.555 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:26.555 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:26.555 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:26.556 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:26.556 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:26.556 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:26.556 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.557 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:21:26.557 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:21:26.557 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:21:26.557 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:21:26.558 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:21:26.558 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:21:26.558 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:26.558 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:26.599 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:21:26.599 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:21:26.600 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:21:26.600 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:21:26.600 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:21:26.600 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:26.601 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:21:26.601 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:26.601 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:26.601 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:26.601 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:26.602 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:26.602 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:26.602 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:26.602 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:26.603 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.603 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:21:26.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.603 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:21:26.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.604 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:21:26.604 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:21:26.604 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:26.604 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:26.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:26.605 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:21:26.605 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:26.606 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:21:26.606 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.606 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.607 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.607 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:21:26.607 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:21:26.607 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:26.608 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:26.608 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:26.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.612 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:21:26.612 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:21:26.612 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:26.612 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:26.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:21:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:21:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.615 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:21:26.615 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:21:26.615 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:26.616 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:26.616 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:26.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.616 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:21:26.617 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:21:26.617 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:26.617 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:26.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:26.618 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:21:26.618 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.618 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:26.619 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:21:26.619 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.619 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.620 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:21:26.620 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:21:26.620 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:26.620 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:26.620 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:26.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.621 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.621 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:21:26.621 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:21:26.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.622 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.622 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.622 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:26.622 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.623 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.623 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:21:26.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.623 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.623 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.624 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:26.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.624 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.624 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:21:26.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.625 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.625 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.625 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:21:26.625 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.625 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.626 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:21:26.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.626 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.626 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.626 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:21:26.627 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.627 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.627 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:21:26.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.627 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.628 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.628 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:21:26.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.628 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.629 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:21:26.629 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:21:26.629 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:21:26.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.755+0000 7fa849f0f700 1 Processor -- start 2020-11-20T12:21:26.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.755+0000 7fa849f0f700 1 -- start start 2020-11-20T12:21:26.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa849f0f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa8440aa400 0x7fa8440a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa849f0f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa8440a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.761 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa849f0f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8440a9770 con 0x7fa8440aa020 2020-11-20T12:21:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa849f0f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8440a56c0 con 0x7fa8440aa400 2020-11-20T12:21:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa849f0f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440a6da0 con 0x7fa84413ac60 2020-11-20T12:21:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa8440aa400 0x7fa8440a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa8437fe700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fa8440aa020 legacy=0x7fa8440a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:21:26.762 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa8440aa400 0x7fa8440a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43524/0 (socket says 172.21.15.44:43524) 2020-11-20T12:21:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 -- 172.21.15.44:0/327564562 learned_addr learned my addr 172.21.15.44:0/327564562 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa843fff700 1 --2- 172.21.15.44:0/327564562 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa8440a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 -- 172.21.15.44:0/327564562 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 msgr2=0x7fa8440a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 --2- 172.21.15.44:0/327564562 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa8440a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.763 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 -- 172.21.15.44:0/327564562 >> v1:172.21.15.79:6789/0 conn(0x7fa8440aa020 legacy=0x7fa8440a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:21:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 -- 172.21.15.44:0/327564562 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8440a7040 con 0x7fa8440aa400 2020-11-20T12:21:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa843fff700 1 --2- 172.21.15.44:0/327564562 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa8440a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa842ffd700 1 --2- 172.21.15.44:0/327564562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa8440a9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa838006fd0 tx=0x7fa838006eb0).ready entity=mon.0 client_cookie=9cf32bf8cdcd8d83 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.756+0000 7fa8427fc700 1 -- 172.21.15.44:0/327564562 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838009890 con 0x7fa8440aa400 2020-11-20T12:21:26.764 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa8427fc700 1 -- 172.21.15.44:0/327564562 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8380099f0 con 0x7fa8440aa400 2020-11-20T12:21:26.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa8427fc700 1 -- 172.21.15.44:0/327564562 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838009d70 con 0x7fa8440aa400 2020-11-20T12:21:26.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 -- 172.21.15.44:0/327564562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 msgr2=0x7fa8440a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 --2- 172.21.15.44:0/327564562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa8440a9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa838006fd0 tx=0x7fa838006eb0).stop 2020-11-20T12:21:26.765 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 -- 172.21.15.44:0/327564562 shutdown_connections 2020-11-20T12:21:26.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 --2- 172.21.15.44:0/327564562 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa8440a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 --2- 172.21.15.44:0/327564562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa8440a9280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 -- 172.21.15.44:0/327564562 >> 172.21.15.44:0/327564562 conn(0x7fa844062690 msgr2=0x7fa8440a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 -- 172.21.15.44:0/327564562 shutdown_connections 2020-11-20T12:21:26.766 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.757+0000 7fa849f0f700 1 -- 172.21.15.44:0/327564562 wait complete. 2020-11-20T12:21:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.758+0000 7fa849f0f700 1 Processor -- start 2020-11-20T12:21:26.767 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.758+0000 7fa849f0f700 1 -- start start 2020-11-20T12:21:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa849f0f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8440aa020 0x7fa8440abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa849f0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa84414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa849f0f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa84414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa849f0f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440ab300 con 0x7fa8440aa400 2020-11-20T12:21:26.770 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa849f0f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84414a280 con 0x7fa8440aa020 2020-11-20T12:21:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa849f0f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84414a520 con 0x7fa84413ac60 2020-11-20T12:21:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa84414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa8437fe700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8440aa020 msgr2=0x7fa8440abfe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:21:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa84414d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43530/0 (socket says 172.21.15.44:43530) 2020-11-20T12:21:26.771 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa8437fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8440aa020 0x7fa8440abfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:21:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 -- 172.21.15.44:0/3407660875 learned_addr learned my addr 172.21.15.44:0/3407660875 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa843fff700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa84414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 msgr2=0x7fa84414d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.772 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413ac60 0x7fa84414d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8440aa020 msgr2=0x7fa8440abfe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8440aa020 0x7fa8440abfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 -- 172.21.15.44:0/3407660875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84414a7c0 con 0x7fa8440aa400 2020-11-20T12:21:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa842ffd700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa84414d3b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa844061280 tx=0x7fa838015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.773 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838011e50 con 0x7fa8440aa400 2020-11-20T12:21:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.759+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83801e460 con 0x7fa8440aa400 2020-11-20T12:21:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.760+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84414ac60 con 0x7fa8440aa400 2020-11-20T12:21:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.760+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83801e7e0 con 0x7fa8440aa400 2020-11-20T12:21:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.760+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8440a31b0 con 0x7fa8440aa400 2020-11-20T12:21:26.774 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.760+0000 7fa849f0f700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa844149de0 0x7fa84414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.760+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84414e280 con 0x7fa844149de0 2020-11-20T12:21:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8437fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa844149de0 0x7fa84414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8437fe700 1 -- 172.21.15.44:0/3407660875 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa844061650 con 0x7fa844149de0 2020-11-20T12:21:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa838025030 con 0x7fa8440aa400 2020-11-20T12:21:26.775 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8437fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa844149de0 0x7fa84414dd90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa834009640 tx=0x7fa834006e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8237fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa81c056d00 0x7fa81c0591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa8380717b0 con 0x7fa8440aa400 2020-11-20T12:21:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa843fff700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa81c056d00 0x7fa81c0591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340050d0 con 0x7fa844149de0 2020-11-20T12:21:26.776 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.761+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fa834005230 con 0x7fa844149de0 2020-11-20T12:21:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.762+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa844149de0 msgr2=0x7fa84414dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.762+0000 7fa8237fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa844149de0 0x7fa84414dd90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa834009640 tx=0x7fa834006e60).stop 2020-11-20T12:21:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.762+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa844149de0 msgr2=0x7fa84414dd90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:26.777 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.762+0000 7fa8237fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa844149de0 0x7fa84414dd90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.778 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.762+0000 7fa843fff700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa81c056d00 0x7fa81c0591c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fa83c006f50 tx=0x7fa83c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.769+0000 7fa849f0f700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413bce0 0x7fa84413c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.779 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.769+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440a9770 con 0x7fa84413bce0 2020-11-20T12:21:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8437fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413bce0 0x7fa84413c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8437fe700 1 -- 172.21.15.44:0/3407660875 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa8440446d0 con 0x7fa84413bce0 2020-11-20T12:21:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8437fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413bce0 0x7fa84413c0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa834009640 tx=0x7fa834013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834015c60 con 0x7fa84413bce0 2020-11-20T12:21:26.780 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4595 (secure 0 0 0) 0x7fa834003720 con 0x7fa84413bce0 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413bce0 msgr2=0x7fa84413c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8237fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413bce0 0x7fa84413c0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fa834009640 tx=0x7fa834013000).stop 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8237fe700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413bce0 msgr2=0x7fa84413c0f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.770+0000 7fa8237fe700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa84413bce0 0x7fa84413c0f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa81c056d00 msgr2=0x7fa81c0591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa81c056d00 0x7fa81c0591c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fa83c006f50 tx=0x7fa83c008040).stop 2020-11-20T12:21:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 msgr2=0x7fa84414d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 --2- 172.21.15.44:0/3407660875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8440aa400 0x7fa84414d3b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa844061280 tx=0x7fa838015000).stop 2020-11-20T12:21:26.782 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa843fff700 1 -- 172.21.15.44:0/3407660875 reap_dead start 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa843fff700 1 -- 172.21.15.44:0/3407660875 reap_dead start 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 shutdown_connections 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 >> 172.21.15.44:0/3407660875 conn(0x7fa844062690 msgr2=0x7fa844137810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 shutdown_connections 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:26.771+0000 7fa849f0f700 1 -- 172.21.15.44:0/3407660875 wait complete. 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:21:26.785 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 180, 2020-11-20T12:21:26.785 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:21:26.785 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:21:26.785 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:21:26.785 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.786 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:21:26.786 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:26.786 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:21:26.786 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:21:26.786 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:26.787 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:26.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:26.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:26.787 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:26.787 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:21:26.789 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:21:26.789 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:26.789 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:26.789 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:21:26.795 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:21:26.795 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:21:26.795 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:21:26.795 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:21:26.795 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:21:26.796 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.796 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:21:26.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.796 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:21:26.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:21:26.797 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:21:26.797 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:26.797 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:26.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:26.798 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:21:26.798 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:26.799 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:21:26.799 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.799 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.799 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.800 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:21:26.800 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:21:26.800 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:26.800 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:26.801 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:26.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:21:26.801 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:21:26.802 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:26.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:26.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:26.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:21:26.803 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:26.803 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:21:26.804 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.804 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.804 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.804 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:21:26.805 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:21:26.805 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:26.805 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:26.805 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:26.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:21:26.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:21:26.806 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:21:26.806 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:21:26.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.807 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:21:26.807 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:21:26.807 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:21:26.808 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:21:26.808 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:21:26.808 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.809 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.809 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:21:26.809 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:21:26.809 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:21:26.810 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:21:26.810 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:21:26.810 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.810 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.811 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:21:26.811 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:21:26.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.811 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.811 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.812 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:26.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.812 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.812 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:21:26.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.813 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.813 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.813 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:26.813 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.813 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.814 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:21:26.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.814 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.814 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.814 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:21:26.815 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.815 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:21:26.815 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:21:26.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:21:26.815 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:21:26.816 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:21:26.816 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:21:26.816 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:21:26.816 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:21:26.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:21:26.817 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:21:26.817 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:21:26.818 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:21:39.673 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:21:39.677 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:21:46.819 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:21:46.819 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:21:46.820 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:21:46.820 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:21:46.823 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:21:46.824 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:21:46.824 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:21:47.017 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.014+0000 7f6f01c36700 1 Processor -- start 2020-11-20T12:21:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.014+0000 7f6f01c36700 1 -- start start 2020-11-20T12:21:47.018 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.016+0000 7f6f01c36700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6efc0a4c80 0x7f6efc0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.016+0000 7f6f01c36700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc066ef0 con 0x7f6efc13ea40 2020-11-20T12:21:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.016+0000 7f6f01c36700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc0669b0 con 0x7f6efc0aae40 2020-11-20T12:21:47.019 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.016+0000 7f6f01c36700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc066c50 con 0x7f6efc0a4c80 2020-11-20T12:21:47.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.016+0000 7f6efb7fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6efc0a4c80 0x7f6efc0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:47.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.016+0000 7f6efbfff700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6efc13ea40 0x7f6efc134010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47292/0 (socket says 172.21.15.44:47292) 2020-11-20T12:21:47.020 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.016+0000 7f6efbfff700 1 -- 172.21.15.44:0/1686138576 learned_addr learned my addr 172.21.15.44:0/1686138576 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:21:47.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.017+0000 7f6efa7fc700 1 -- 172.21.15.44:0/1686138576 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181352068 0 0) 0x7f6efc066ef0 con 0x7f6efc13ea40 2020-11-20T12:21:47.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.017+0000 7f6efa7fc700 1 -- 172.21.15.44:0/1686138576 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee8003240 con 0x7f6efc13ea40 2020-11-20T12:21:47.021 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.017+0000 7f6efa7fc700 1 -- 172.21.15.44:0/1686138576 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3867544666 0 0) 0x7f6ee8003240 con 0x7f6efc13ea40 2020-11-20T12:21:47.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.017+0000 7f6efb7fe700 1 -- 172.21.15.44:0/1686138576 >> v1:172.21.15.79:6789/0 conn(0x7f6efc0aae40 legacy=0x7f6efc13e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:47.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.017+0000 7f6efb7fe700 1 -- 172.21.15.44:0/1686138576 >> v1:172.21.15.44:6789/0 conn(0x7f6efc13ea40 legacy=0x7f6efc134010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:47.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.017+0000 7f6efb7fe700 1 -- 172.21.15.44:0/1686138576 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc0a5060 con 0x7f6efc0a4c80 2020-11-20T12:21:47.022 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.017+0000 7f6efb7fe700 1 --2- 172.21.15.44:0/1686138576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc0aa950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f6ef4009c90 tx=0x7f6ef40091d0).ready entity=mon.0 client_cookie=777f8088c9a5545 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.020+0000 7f6efa7fc700 1 -- 172.21.15.44:0/1686138576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0014e0 con 0x7f6efc0a4c80 2020-11-20T12:21:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.020+0000 7f6efa7fc700 1 -- 172.21.15.44:0/1686138576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400b760 con 0x7f6efc0a4c80 2020-11-20T12:21:47.023 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.020+0000 7f6efa7fc700 1 -- 172.21.15.44:0/1686138576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400bae0 con 0x7f6efc0a4c80 2020-11-20T12:21:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.021+0000 7f6f01c36700 1 -- 172.21.15.44:0/1686138576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 msgr2=0x7f6efc0aa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:47.026 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.021+0000 7f6f01c36700 1 --2- 172.21.15.44:0/1686138576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc0aa950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f6ef4009c90 tx=0x7f6ef40091d0).stop 2020-11-20T12:21:47.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.024+0000 7f6f01c36700 1 -- 172.21.15.44:0/1686138576 shutdown_connections 2020-11-20T12:21:47.027 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.024+0000 7f6f01c36700 1 --2- 172.21.15.44:0/1686138576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc0aa950 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:47.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.024+0000 7f6f01c36700 1 -- 172.21.15.44:0/1686138576 >> 172.21.15.44:0/1686138576 conn(0x7f6efc062bc0 msgr2=0x7f6efc062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:47.028 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.025+0000 7f6f01c36700 1 -- 172.21.15.44:0/1686138576 shutdown_connections 2020-11-20T12:21:47.029 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.025+0000 7f6f01c36700 1 -- 172.21.15.44:0/1686138576 wait complete. 2020-11-20T12:21:47.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.027+0000 7f6f01c36700 1 Processor -- start 2020-11-20T12:21:47.030 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.027+0000 7f6f01c36700 1 -- start start 2020-11-20T12:21:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6f01c36700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc1314b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:47.031 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6f01c36700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6efc13ea40 0x7f6efc13a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:47.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6f01c36700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6efc1319a0 0x7f6efc13a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:47.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6f01c36700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc13ad00 con 0x7f6efc0a4c80 2020-11-20T12:21:47.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6f01c36700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0aec40 con 0x7f6efc1319a0 2020-11-20T12:21:47.032 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6f01c36700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0adc30 con 0x7f6efc13ea40 2020-11-20T12:21:47.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6efb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc1314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:47.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6efb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc1314b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43548/0 (socket says 172.21.15.44:43548) 2020-11-20T12:21:47.033 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.028+0000 7f6efb7fe700 1 -- 172.21.15.44:0/81819939 learned_addr learned my addr 172.21.15.44:0/81819939 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:21:47.034 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6efb7fe700 1 -- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6efc13ea40 msgr2=0x7f6efc13a320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:47.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6efb7fe700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6efc13ea40 0x7f6efc13a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:47.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6efb7fe700 1 -- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6efc1319a0 msgr2=0x7f6efc13a810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:21:47.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6efb7fe700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6efc1319a0 0x7f6efc13a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:47.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6efb7fe700 1 -- 172.21.15.44:0/81819939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc139be0 con 0x7f6efc0a4c80 2020-11-20T12:21:47.035 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6efb7fe700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc1314b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f6ef400b670 tx=0x7f6ef4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6edb7fe700 1 -- 172.21.15.44:0/81819939 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4021070 con 0x7f6efc0a4c80 2020-11-20T12:21:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6edb7fe700 1 -- 172.21.15.44:0/81819939 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef401d480 con 0x7f6efc0a4c80 2020-11-20T12:21:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc139e80 con 0x7f6efc0a4c80 2020-11-20T12:21:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6edb7fe700 1 -- 172.21.15.44:0/81819939 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4011ea0 con 0x7f6efc0a4c80 2020-11-20T12:21:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.029+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc0acef0 con 0x7f6efc0a4c80 2020-11-20T12:21:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.031+0000 7f6edb7fe700 1 -- 172.21.15.44:0/81819939 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f6ef4017070 con 0x7f6efc0a4c80 2020-11-20T12:21:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.031+0000 7f6edb7fe700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ed4056d20 0x7f6ed40591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.032+0000 7f6efaffd700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ed4056d20 0x7f6ed40591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.032+0000 7f6edb7fe700 1 -- 172.21.15.44:0/81819939 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f6ef40720c0 con 0x7f6efc0a4c80 2020-11-20T12:21:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.032+0000 7f6efaffd700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ed4056d20 0x7f6ed40591e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6ef0006fd0 tx=0x7f6ef0006930).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.032+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc0ac290 con 0x7f6efc0a4c80 2020-11-20T12:21:47.052 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.050+0000 7f6edb7fe700 1 -- 172.21.15.44:0/81819939 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6ef406efd0 con 0x7f6efc0a4c80 2020-11-20T12:21:47.564 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":180,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:21:47.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.556+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6efc05be70 con 0x7f6efc0a4c80 2020-11-20T12:21:47.564 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.557+0000 7f6edb7fe700 1 -- 172.21.15.44:0/81819939 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7f6ef4015020 con 0x7f6efc0a4c80 2020-11-20T12:21:47.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ed4056d20 msgr2=0x7f6ed40591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:47.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ed4056d20 0x7f6ed40591e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6ef0006fd0 tx=0x7f6ef0006930).stop 2020-11-20T12:21:47.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 msgr2=0x7f6efc1314b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:47.565 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc1314b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f6ef400b670 tx=0x7f6ef4016000).stop 2020-11-20T12:21:47.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 shutdown_connections 2020-11-20T12:21:47.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ed4056d20 0x7f6ed40591e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:47.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6efc1319a0 0x7f6efc13a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:47.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc0a4c80 0x7f6efc1314b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:47.566 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 --2- 172.21.15.44:0/81819939 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6efc13ea40 0x7f6efc13a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:47.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 >> 172.21.15.44:0/81819939 conn(0x7f6efc062bc0 msgr2=0x7f6efc134950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:47.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 shutdown_connections 2020-11-20T12:21:47.567 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:21:47.559+0000 7f6f01c36700 1 -- 172.21.15.44:0/81819939 wait complete. 2020-11-20T12:21:50.955 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:03.104 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.101+0000 7f8d81cca700 1 Processor -- start 2020-11-20T12:22:03.105 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.101+0000 7f8d81cca700 1 -- start start 2020-11-20T12:22:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d81cca700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8d78006e70 0x7f8d78036d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d81cca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c8a70 0x7f8d780c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.108 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d81cca700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d780cc410 con 0x7f8d780c93e0 2020-11-20T12:22:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d81cca700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d78037a10 con 0x7f8d78006e70 2020-11-20T12:22:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d81cca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780cc6b0 con 0x7f8d780c8a70 2020-11-20T12:22:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d8372d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8d78006e70 0x7f8d78036d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.109 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d8372d700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8d78006e70 0x7f8d78036d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43554/0 (socket says 172.21.15.44:43554) 2020-11-20T12:22:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d80cc8700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c8a70 0x7f8d780c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d8372d700 1 -- 172.21.15.44:0/3207426501 learned_addr learned my addr 172.21.15.44:0/3207426501 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.102+0000 7f8d824cb700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8d780c93e0 0x7f8d78038460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:50182/0 (socket says 172.21.15.44:50182) 2020-11-20T12:22:03.110 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d6bfff700 1 -- 172.21.15.44:0/3207426501 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474195555 0 0) 0x7f8d780cc410 con 0x7f8d780c93e0 2020-11-20T12:22:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d8372d700 1 -- 172.21.15.44:0/3207426501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c8a70 msgr2=0x7f8d780c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d8372d700 1 --2- 172.21.15.44:0/3207426501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c8a70 0x7f8d780c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.111 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d8372d700 1 -- 172.21.15.44:0/3207426501 >> v1:172.21.15.79:6789/0 conn(0x7f8d780c93e0 legacy=0x7f8d78038460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d8372d700 1 -- 172.21.15.44:0/3207426501 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780cc950 con 0x7f8d78006e70 2020-11-20T12:22:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d80cc8700 1 --2- 172.21.15.44:0/3207426501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c8a70 0x7f8d780c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:03.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d8372d700 1 --2- 172.21.15.44:0/3207426501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d78006e70 0x7f8d78036d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f8d74009d90 tx=0x7f8d740092d0).ready entity=mon.0 client_cookie=fae5a13f07e7acb4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d6bfff700 1 -- 172.21.15.44:0/3207426501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400bbc0 con 0x7f8d78006e70 2020-11-20T12:22:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d6bfff700 1 -- 172.21.15.44:0/3207426501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7400bd20 con 0x7f8d78006e70 2020-11-20T12:22:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.103+0000 7f8d6bfff700 1 -- 172.21.15.44:0/3207426501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74011610 con 0x7f8d78006e70 2020-11-20T12:22:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 -- 172.21.15.44:0/3207426501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d78006e70 msgr2=0x7f8d78036d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 --2- 172.21.15.44:0/3207426501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d78006e70 0x7f8d78036d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f8d74009d90 tx=0x7f8d740092d0).stop 2020-11-20T12:22:03.114 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 -- 172.21.15.44:0/3207426501 shutdown_connections 2020-11-20T12:22:03.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 --2- 172.21.15.44:0/3207426501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d78006e70 0x7f8d78036d20 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0x7f8d74009d90 tx=0x7f8d740092d0).stop 2020-11-20T12:22:03.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 --2- 172.21.15.44:0/3207426501 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c8a70 0x7f8d780c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 -- 172.21.15.44:0/3207426501 >> 172.21.15.44:0/3207426501 conn(0x7f8d7803fe90 msgr2=0x7f8d7803beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:03.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 -- 172.21.15.44:0/3207426501 shutdown_connections 2020-11-20T12:22:03.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.104+0000 7f8d81cca700 1 -- 172.21.15.44:0/3207426501 wait complete. 2020-11-20T12:22:03.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.105+0000 7f8d81cca700 1 Processor -- start 2020-11-20T12:22:03.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.105+0000 7f8d81cca700 1 -- start start 2020-11-20T12:22:03.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d81cca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d780c8a70 0x7f8d78008eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d81cca700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c93e0 0x7f8d780312d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d81cca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d780dd4f0 0x7f8d780dd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d81cca700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7803e850 con 0x7f8d780c8a70 2020-11-20T12:22:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d81cca700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78007a00 con 0x7f8d780dd4f0 2020-11-20T12:22:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d81cca700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78007ca0 con 0x7f8d780c93e0 2020-11-20T12:22:03.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d780c8a70 0x7f8d78008eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d780c8a70 0x7f8d78008eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43560/0 (socket says 172.21.15.44:43560) 2020-11-20T12:22:03.122 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 -- 172.21.15.44:0/796530103 learned_addr learned my addr 172.21.15.44:0/796530103 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:03.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d80cc8700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c93e0 0x7f8d780312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 -- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c93e0 msgr2=0x7f8d780312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c93e0 0x7f8d780312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 -- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d780dd4f0 msgr2=0x7f8d780dd970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d780dd4f0 0x7f8d780dd970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 -- 172.21.15.44:0/796530103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d78007f40 con 0x7f8d780c8a70 2020-11-20T12:22:03.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d80cc8700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c93e0 0x7f8d780312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d8372d700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d780c8a70 0x7f8d78008eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f8d74011000 tx=0x7f8d74016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.106+0000 7f8d697fa700 1 -- 172.21.15.44:0/796530103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7401c680 con 0x7f8d780c8a70 2020-11-20T12:22:03.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.107+0000 7f8d697fa700 1 -- 172.21.15.44:0/796530103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7401c7e0 con 0x7f8d780c8a70 2020-11-20T12:22:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.107+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7803eb70 con 0x7f8d780c8a70 2020-11-20T12:22:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.107+0000 7f8d697fa700 1 -- 172.21.15.44:0/796530103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7401cb60 con 0x7f8d780c8a70 2020-11-20T12:22:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.107+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d780317c0 con 0x7f8d780c8a70 2020-11-20T12:22:03.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.108+0000 7f8d697fa700 1 -- 172.21.15.44:0/796530103 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8d74017070 con 0x7f8d780c8a70 2020-11-20T12:22:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.109+0000 7f8d697fa700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8d54056d20 0x7f8d540591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.109+0000 7f8d697fa700 1 -- 172.21.15.44:0/796530103 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8d74071f10 con 0x7f8d780c8a70 2020-11-20T12:22:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.109+0000 7f8d80cc8700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8d54056d20 0x7f8d540591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.109+0000 7f8d80cc8700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8d54056d20 0x7f8d540591e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8d6c006bd0 tx=0x7f8d6c008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.380+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d780c6b60 con 0x7f8d780c8a70 2020-11-20T12:22:03.390 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.386+0000 7f8d697fa700 1 -- 172.21.15.44:0/796530103 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8d74015020 con 0x7f8d780c8a70 2020-11-20T12:22:03.819 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.816+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8d780080d0 con 0x7f8d780c8a70 2020-11-20T12:22:03.820 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.816+0000 7f8d697fa700 1 -- 172.21.15.44:0/796530103 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f8d74047290 con 0x7f8d780c8a70 2020-11-20T12:22:03.821 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":182,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:03.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8d54056d20 msgr2=0x7f8d540591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8d54056d20 0x7f8d540591e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8d6c006bd0 tx=0x7f8d6c008040).stop 2020-11-20T12:22:03.824 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d780c8a70 msgr2=0x7f8d78008eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d780c8a70 0x7f8d78008eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f8d74011000 tx=0x7f8d74016000).stop 2020-11-20T12:22:03.825 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 shutdown_connections 2020-11-20T12:22:03.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8d54056d20 0x7f8d540591e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8d780dd4f0 0x7f8d780dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:03.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d780c8a70 0x7f8d78008eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.826 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 --2- 172.21.15.44:0/796530103 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8d780c93e0 0x7f8d780312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.821+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 >> 172.21.15.44:0/796530103 conn(0x7f8d7803fe90 msgr2=0x7f8d780046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:03.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.822+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 shutdown_connections 2020-11-20T12:22:03.827 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:03.822+0000 7f8d81cca700 1 -- 172.21.15.44:0/796530103 wait complete. 2020-11-20T12:22:04.020 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:22:04.021 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:22:05.112 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.109+0000 7f02e51ad700 1 Processor -- start 2020-11-20T12:22:05.113 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.109+0000 7f02e51ad700 1 -- start start 2020-11-20T12:22:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02e51ad700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f02e0060330 0x7f02e00a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02e51ad700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e00a4c30 0x7f02e013a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02e51ad700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00a6260 con 0x7f02e013ae80 2020-11-20T12:22:05.115 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02e51ad700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e00a52e0 con 0x7f02e0060330 2020-11-20T12:22:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02e51ad700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00aa730 con 0x7f02e00a4c30 2020-11-20T12:22:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02ded9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f02e0060330 0x7f02e00a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02de59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e00a4c30 0x7f02e013a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02df59e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f02e013ae80 0x7f02e00aa020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:50194/0 (socket says 172.21.15.44:50194) 2020-11-20T12:22:05.116 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02df59e700 1 -- 172.21.15.44:0/2408995270 learned_addr learned my addr 172.21.15.44:0/2408995270 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:22:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02de59c700 1 --2- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e00a4c30 0x7f02e013a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55006/0 (socket says 172.21.15.44:55006) 2020-11-20T12:22:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02ded9d700 1 -- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e00a4c30 msgr2=0x7f02e013a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02ded9d700 1 --2- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e00a4c30 0x7f02e013a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02ded9d700 1 -- 172.21.15.44:0/2408995270 >> v1:172.21.15.79:6789/0 conn(0x7f02e013ae80 legacy=0x7f02e00aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.117 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02ded9d700 1 -- 172.21.15.44:0/2408995270 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e00a6490 con 0x7f02e0060330 2020-11-20T12:22:05.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.110+0000 7f02ded9d700 1 --2- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e0060330 0x7f02e00a4740 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f02d000b9d0 tx=0x7f02d0006df0).ready entity=mon.1 client_cookie=31cb465069be3b25 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02ddd9b700 1 -- 172.21.15.44:0/2408995270 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00096b0 con 0x7f02e0060330 2020-11-20T12:22:05.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02ddd9b700 1 -- 172.21.15.44:0/2408995270 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0009810 con 0x7f02e0060330 2020-11-20T12:22:05.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02ddd9b700 1 -- 172.21.15.44:0/2408995270 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0009b90 con 0x7f02e0060330 2020-11-20T12:22:05.118 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 -- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e0060330 msgr2=0x7f02e00a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 --2- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e0060330 0x7f02e00a4740 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f02d000b9d0 tx=0x7f02d0006df0).stop 2020-11-20T12:22:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 -- 172.21.15.44:0/2408995270 shutdown_connections 2020-11-20T12:22:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 --2- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e0060330 0x7f02e00a4740 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 --2- 172.21.15.44:0/2408995270 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e00a4c30 0x7f02e013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.119 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 -- 172.21.15.44:0/2408995270 >> 172.21.15.44:0/2408995270 conn(0x7f02e0062690 msgr2=0x7f02e00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 -- 172.21.15.44:0/2408995270 shutdown_connections 2020-11-20T12:22:05.120 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.111+0000 7f02e51ad700 1 -- 172.21.15.44:0/2408995270 wait complete. 2020-11-20T12:22:05.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.112+0000 7f02e51ad700 1 Processor -- start 2020-11-20T12:22:05.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.112+0000 7f02e51ad700 1 -- start start 2020-11-20T12:22:05.123 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02e51ad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0060330 0x7f02e013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02e51ad700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e00a4c30 0x7f02e0067350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02e51ad700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e013ae80 0x7f02e0136830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02e51ad700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e014d460 con 0x7f02e0060330 2020-11-20T12:22:05.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02e51ad700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e013cf40 con 0x7f02e00a4c30 2020-11-20T12:22:05.124 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02e51ad700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e013cca0 con 0x7f02e013ae80 2020-11-20T12:22:05.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0060330 0x7f02e013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0060330 0x7f02e013c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43572/0 (socket says 172.21.15.44:43572) 2020-11-20T12:22:05.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 -- 172.21.15.44:0/1333649909 learned_addr learned my addr 172.21.15.44:0/1333649909 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:05.125 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02de59c700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e00a4c30 0x7f02e0067350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02df59e700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e013ae80 0x7f02e0136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e013ae80 msgr2=0x7f02e0136830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f02e013ae80 0x7f02e0136830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e00a4c30 msgr2=0x7f02e0067350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.126 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e00a4c30 0x7f02e0067350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02ded9d700 1 -- 172.21.15.44:0/1333649909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0136390 con 0x7f02e0060330 2020-11-20T12:22:05.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.113+0000 7f02de59c700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02e00a4c30 0x7f02e0067350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:05.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.114+0000 7f02ded9d700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0060330 0x7f02e013c6b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f02d0010000 tx=0x7f02d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.114+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00067a0 con 0x7f02e0060330 2020-11-20T12:22:05.127 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.114+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0011b40 con 0x7f02e0060330 2020-11-20T12:22:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.114+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e00a2d90 con 0x7f02e0060330 2020-11-20T12:22:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.114+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d001e460 con 0x7f02e0060330 2020-11-20T12:22:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.114+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e0137cb0 con 0x7f02e0060330 2020-11-20T12:22:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.115+0000 7f02e51ad700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0136d20 0x7f02e014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.128 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.115+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0136520 con 0x7f02e0136d20 2020-11-20T12:22:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.115+0000 7f02de59c700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0136d20 0x7f02e014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.115+0000 7f02de59c700 1 -- 172.21.15.44:0/1333649909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02e0061650 con 0x7f02e0136d20 2020-11-20T12:22:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.115+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f02d0025030 con 0x7f02e0060330 2020-11-20T12:22:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.115+0000 7f02de59c700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0136d20 0x7f02e014da00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f02e013c1b0 tx=0x7f02d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.129 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f02bc056d50 0x7f02bc059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f02d00715d0 con 0x7f02e0060330 2020-11-20T12:22:05.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02df59e700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f02bc056d50 0x7f02bc059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400fec0 con 0x7f02e0136d20 2020-11-20T12:22:05.130 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f02d40135b0 con 0x7f02e0136d20 2020-11-20T12:22:05.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0136d20 msgr2=0x7f02e014da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0136d20 0x7f02e014da00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f02e013c1b0 tx=0x7f02d4006df0).stop 2020-11-20T12:22:05.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0136d20 msgr2=0x7f02e014da00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:05.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02c6ffd700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0136d20 0x7f02e014da00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.131 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.116+0000 7f02df59e700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f02bc056d50 0x7f02bc059210 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f02d8007fd0 tx=0x7f02d8009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.124+0000 7f02e51ad700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e014def0 0x7f02e00a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.124+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00446d0 con 0x7f02e014def0 2020-11-20T12:22:05.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.124+0000 7f02de59c700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e014def0 0x7f02e00a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02de59c700 1 -- 172.21.15.44:0/1333649909 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f02e00680f0 con 0x7f02e014def0 2020-11-20T12:22:05.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02de59c700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e014def0 0x7f02e00a1880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f02d4018000 tx=0x7f02d401b000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4014e30 con 0x7f02e014def0 2020-11-20T12:22:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4779 (secure 0 0 0) 0x7f02d40133a0 con 0x7f02e014def0 2020-11-20T12:22:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e014def0 msgr2=0x7f02e00a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02c6ffd700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e014def0 0x7f02e00a1880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f02d4018000 tx=0x7f02d401b000).stop 2020-11-20T12:22:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02c6ffd700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e014def0 msgr2=0x7f02e00a1880 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:05.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.125+0000 7f02c6ffd700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e014def0 0x7f02e00a1880 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0x7f02d4018000 tx=0x7f02d401b000).stop 2020-11-20T12:22:05.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f02bc056d50 msgr2=0x7f02bc059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f02bc056d50 0x7f02bc059210 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f02d8007fd0 tx=0x7f02d8009040).stop 2020-11-20T12:22:05.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0060330 msgr2=0x7f02e013c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.136 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 --2- 172.21.15.44:0/1333649909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e0060330 0x7f02e013c6b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f02d0010000 tx=0x7f02d0015000).stop 2020-11-20T12:22:05.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02df59e700 1 -- 172.21.15.44:0/1333649909 reap_dead start 2020-11-20T12:22:05.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02df59e700 1 -- 172.21.15.44:0/1333649909 reap_dead start 2020-11-20T12:22:05.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 shutdown_connections 2020-11-20T12:22:05.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 >> 172.21.15.44:0/1333649909 conn(0x7f02e0062690 msgr2=0x7f02e00a9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 shutdown_connections 2020-11-20T12:22:05.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.126+0000 7f02e51ad700 1 -- 172.21.15.44:0/1333649909 wait complete. 2020-11-20T12:22:05.138 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:05.138 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:05.138 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:05.139 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:22:05.139 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 182, 2020-11-20T12:22:05.139 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:05.139 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:05.139 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:22:05.140 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:05.140 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.140 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 13, 2020-11-20T12:22:05.140 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:05.140 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:05.141 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:05.141 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.141 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.141 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.142 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.143 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.143 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.143 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.144 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.144 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:05.144 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:05.144 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.144 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.145 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.145 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.145 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.145 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.146 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.146 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:05.146 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:05.147 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:05.147 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:05.147 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:05.147 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:05.148 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:05.148 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:05.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:05.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:05.148 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:05.149 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:05.149 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:05.149 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:05.149 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:05.149 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.150 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.150 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.150 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.150 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.150 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.151 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.151 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.151 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:05.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.152 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:05.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.152 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:05.152 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:05.152 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:05.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:05.154 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.155 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:05.155 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:05.156 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.156 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.156 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.157 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:05.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:05.157 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:05.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:05.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.159 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:05.160 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:05.160 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.161 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.161 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.161 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:05.162 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:05.162 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:05.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:05.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:05.165 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:05.165 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.165 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.165 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.166 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.166 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:05.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:05.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.167 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.167 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.167 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:05.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.168 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.168 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:22:05.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.168 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.168 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.169 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:05.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.169 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.169 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:22:05.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.170 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.170 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.170 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:22:05.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.170 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:05.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.171 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.171 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.171 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:22:05.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.172 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.172 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:22:05.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.172 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.173 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.173 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:22:05.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.173 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.174 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:05.174 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:05.175 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:22:05.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 Processor -- start 2020-11-20T12:22:05.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 -- start start 2020-11-20T12:22:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f39c8060330 0x7f39c80a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f39c80a4c30 0x7f39c813a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c813ae80 0x7f39c8136f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c80a52e0 con 0x7f39c80a4c30 2020-11-20T12:22:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c80a6260 con 0x7f39c8060330 2020-11-20T12:22:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.323+0000 7f39cd64f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80a6490 con 0x7f39c813ae80 2020-11-20T12:22:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f39c80a4c30 0x7f39c813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f39c80a4c30 0x7f39c813a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43584/0 (socket says 172.21.15.44:43584) 2020-11-20T12:22:05.330 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 -- 172.21.15.44:0/1358020657 learned_addr learned my addr 172.21.15.44:0/1358020657 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c6ffd700 1 --2- 172.21.15.44:0/1358020657 >> v2:172.21.15.79:3300/0 conn(0x7f39c8060330 0x7f39c80a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c77fe700 1 --2- 172.21.15.44:0/1358020657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c813ae80 0x7f39c8136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 -- 172.21.15.44:0/1358020657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c813ae80 msgr2=0x7f39c8136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 --2- 172.21.15.44:0/1358020657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c813ae80 0x7f39c8136f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 -- 172.21.15.44:0/1358020657 >> v2:172.21.15.79:3300/0 conn(0x7f39c8060330 msgr2=0x7f39c80a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 --2- 172.21.15.44:0/1358020657 >> v2:172.21.15.79:3300/0 conn(0x7f39c8060330 0x7f39c80a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 -- 172.21.15.44:0/1358020657 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80a6730 con 0x7f39c80a4c30 2020-11-20T12:22:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c77fe700 1 --2- 172.21.15.44:0/1358020657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c813ae80 0x7f39c8136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c67fc700 1 --2- 172.21.15.44:0/1358020657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 0x7f39c813a990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f39c80adec0 tx=0x7f39b0006eb0).ready entity=mon.0 client_cookie=c2ed64db533e90bd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c5ffb700 1 -- 172.21.15.44:0/1358020657 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b00097d0 con 0x7f39c80a4c30 2020-11-20T12:22:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c5ffb700 1 -- 172.21.15.44:0/1358020657 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b0009930 con 0x7f39c80a4c30 2020-11-20T12:22:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.324+0000 7f39c5ffb700 1 -- 172.21.15.44:0/1358020657 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0009cb0 con 0x7f39c80a4c30 2020-11-20T12:22:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 -- 172.21.15.44:0/1358020657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 msgr2=0x7f39c813a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 --2- 172.21.15.44:0/1358020657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 0x7f39c813a990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f39c80adec0 tx=0x7f39b0006eb0).stop 2020-11-20T12:22:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 -- 172.21.15.44:0/1358020657 shutdown_connections 2020-11-20T12:22:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 --2- 172.21.15.44:0/1358020657 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c813ae80 0x7f39c8136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 --2- 172.21.15.44:0/1358020657 >> v2:172.21.15.79:3300/0 conn(0x7f39c8060330 0x7f39c80a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 --2- 172.21.15.44:0/1358020657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 0x7f39c813a990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 -- 172.21.15.44:0/1358020657 >> 172.21.15.44:0/1358020657 conn(0x7f39c8062690 msgr2=0x7f39c80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 -- 172.21.15.44:0/1358020657 shutdown_connections 2020-11-20T12:22:05.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.325+0000 7f39cd64f700 1 -- 172.21.15.44:0/1358020657 wait complete. 2020-11-20T12:22:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.327+0000 7f39cd64f700 1 Processor -- start 2020-11-20T12:22:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.327+0000 7f39cd64f700 1 -- start start 2020-11-20T12:22:05.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39cd64f700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c8060330 0x7f39c80ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39cd64f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 0x7f39c8130460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39cd64f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c813ae80 0x7f39c8137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39cd64f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80ab8e0 con 0x7f39c80a4c30 2020-11-20T12:22:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39cd64f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8134d00 con 0x7f39c813ae80 2020-11-20T12:22:05.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39cd64f700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80aac60 con 0x7f39c8060330 2020-11-20T12:22:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 0x7f39c8130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c67fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 0x7f39c8130460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43590/0 (socket says 172.21.15.44:43590) 2020-11-20T12:22:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c813ae80 0x7f39c8137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c67fc700 1 -- 172.21.15.44:0/2747807996 learned_addr learned my addr 172.21.15.44:0/2747807996 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c77fe700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c8060330 msgr2=0x7f39c80ac680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c77fe700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f39c8060330 0x7f39c80ac680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c77fe700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 msgr2=0x7f39c8130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c77fe700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c80a4c30 0x7f39c8130460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c77fe700 1 -- 172.21.15.44:0/2747807996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80aaf00 con 0x7f39c813ae80 2020-11-20T12:22:05.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.328+0000 7f39c77fe700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c813ae80 0x7f39c8137280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f39b800b7b0 tx=0x7f39b8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.329+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80125f0 con 0x7f39c813ae80 2020-11-20T12:22:05.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.329+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8012750 con 0x7f39c813ae80 2020-11-20T12:22:05.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.329+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8012af0 con 0x7f39c813ae80 2020-11-20T12:22:05.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.329+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c80ab1a0 con 0x7f39c813ae80 2020-11-20T12:22:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.329+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c80abc50 con 0x7f39c813ae80 2020-11-20T12:22:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.330+0000 7f39cd64f700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c8137770 0x7f39c8146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.330+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8061650 con 0x7f39c8137770 2020-11-20T12:22:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.330+0000 7f39c6ffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c8137770 0x7f39c8146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.330+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f39b8013480 con 0x7f39c813ae80 2020-11-20T12:22:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.330+0000 7f39c6ffd700 1 -- 172.21.15.44:0/2747807996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39c80446d0 con 0x7f39c8137770 2020-11-20T12:22:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.330+0000 7f39c6ffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c8137770 0x7f39c8146be0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f39c000ebd0 tx=0x7f39c00091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39a405b160 0x7f39a405d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f39b80702b0 con 0x7f39c813ae80 2020-11-20T12:22:05.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000caf0 con 0x7f39c8137770 2020-11-20T12:22:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39c67fc700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39a405b160 0x7f39a405d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f39c000cc50 con 0x7f39c8137770 2020-11-20T12:22:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c8137770 msgr2=0x7f39c8146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c8137770 0x7f39c8146be0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f39c000ebd0 tx=0x7f39c00091d0).stop 2020-11-20T12:22:05.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c8137770 msgr2=0x7f39c8146be0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39beffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c8137770 0x7f39c8146be0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.331+0000 7f39c67fc700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39a405b160 0x7f39a405d620 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f39c80ad650 tx=0x7f39b0015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.344+0000 7f39cd64f700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c81470d0 0x7f39c814a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.344+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c81474e0 con 0x7f39c81470d0 2020-11-20T12:22:05.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.345+0000 7f39c6ffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c81470d0 0x7f39c814a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.346+0000 7f39c6ffd700 1 -- 172.21.15.44:0/2747807996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f39c80680f0 con 0x7f39c81470d0 2020-11-20T12:22:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.346+0000 7f39c6ffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c81470d0 0x7f39c814a130 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f39c000ebd0 tx=0x7f39c0018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.346+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0020950 con 0x7f39c81470d0 2020-11-20T12:22:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.346+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7f39c000c8e0 con 0x7f39c81470d0 2020-11-20T12:22:05.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.347+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c81470d0 msgr2=0x7f39c814a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.347+0000 7f39beffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c81470d0 0x7f39c814a130 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f39c000ebd0 tx=0x7f39c0018000).stop 2020-11-20T12:22:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.347+0000 7f39beffd700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c81470d0 msgr2=0x7f39c814a130 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:05.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.347+0000 7f39beffd700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c81470d0 0x7f39c814a130 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0x7f39c000ebd0 tx=0x7f39c0018000).stop 2020-11-20T12:22:05.353 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:05.353 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:05.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:05.354 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:22:05.354 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 182, 2020-11-20T12:22:05.354 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:05.354 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:05.354 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:22:05.355 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:05.355 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.355 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:22:05.355 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:05.355 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:05.356 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:05.356 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.356 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.356 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.356 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.357 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.357 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.357 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.357 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.357 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.358 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:05.358 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:05.358 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.358 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.358 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.359 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.359 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.359 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.359 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.359 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.360 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.360 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:05.360 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:05.360 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:05.361 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:05.361 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:05.361 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:05.361 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:05.362 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:05.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:05.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:05.362 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:05.362 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:05.363 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:05.363 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:05.363 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:05.363 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.363 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.363 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.364 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.364 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.364 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.364 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.364 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.365 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.365 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:05.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.365 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:05.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:05.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:05.366 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.366 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.367 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:05.367 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.368 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:05.368 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.368 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.369 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:05.369 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:05.369 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.369 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.370 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:05.370 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:05.371 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.371 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.371 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:05.372 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.372 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:05.373 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.373 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.373 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:05.374 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:05.374 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.374 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.374 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:05.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:05.375 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.375 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.376 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:05.376 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.377 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.377 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:05.377 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.377 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.378 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:05.378 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:05.378 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.379 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.379 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.379 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.380 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:05.380 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:05.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.380 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.380 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.381 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:05.381 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.381 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.381 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:05.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.382 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.382 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.382 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:22:05.382 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.382 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.383 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:05.383 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:05.383 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.348+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39a405b160 msgr2=0x7f39a405d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.348+0000 7f39cd64f700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f39a405b160 0x7f39a405d620 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f39c80ad650 tx=0x7f39b0015000).stop 2020-11-20T12:22:05.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.348+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c813ae80 msgr2=0x7f39c8137280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.384 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.348+0000 7f39cd64f700 1 --2- 172.21.15.44:0/2747807996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f39c813ae80 0x7f39c8137280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f39b800b7b0 tx=0x7f39b8008040).stop 2020-11-20T12:22:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.348+0000 7f39c77fe700 1 -- 172.21.15.44:0/2747807996 reap_dead start 2020-11-20T12:22:05.385 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.348+0000 7f39c77fe700 1 -- 172.21.15.44:0/2747807996 reap_dead start 2020-11-20T12:22:05.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.350+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 shutdown_connections 2020-11-20T12:22:05.386 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.350+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 >> 172.21.15.44:0/2747807996 conn(0x7f39c8062690 msgr2=0x7f39c8067860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.354+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 shutdown_connections 2020-11-20T12:22:05.387 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.355+0000 7f39cd64f700 1 -- 172.21.15.44:0/2747807996 wait complete. 2020-11-20T12:22:05.387 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:22:05.634 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.631+0000 7fcfef280700 1 Processor -- start 2020-11-20T12:22:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.631+0000 7fcfef280700 1 -- start start 2020-11-20T12:22:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.633+0000 7fcfef280700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80aa020 0x7fcfe8060330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.633+0000 7fcfef280700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcfe80a8af0 0x7fcfe80a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.633+0000 7fcfef280700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe8137440 con 0x7fcfe80a5f80 2020-11-20T12:22:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.633+0000 7fcfef280700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe80a6700 con 0x7fcfe80a8af0 2020-11-20T12:22:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.633+0000 7fcfef280700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80a69e0 con 0x7fcfe80aa020 2020-11-20T12:22:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.633+0000 7fcfec81b700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcfe80a5f80 0x7fcfe8136d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47348/0 (socket says 172.21.15.44:47348) 2020-11-20T12:22:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.633+0000 7fcfec81b700 1 -- 172.21.15.44:0/2883868171 learned_addr learned my addr 172.21.15.44:0/2883868171 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:22:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.634+0000 7fcfed81d700 1 --2- 172.21.15.44:0/2883868171 >> v2:172.21.15.79:3300/0 conn(0x7fcfe80a8af0 0x7fcfe80a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.634+0000 7fcfd7fff700 1 -- 172.21.15.44:0/2883868171 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007785142 0 0) 0x7fcfe8137440 con 0x7fcfe80a5f80 2020-11-20T12:22:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.634+0000 7fcfd7fff700 1 -- 172.21.15.44:0/2883868171 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd0003240 con 0x7fcfe80a5f80 2020-11-20T12:22:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.634+0000 7fcfed01c700 1 --2- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80aa020 0x7fcfe8060330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfed81d700 1 -- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80aa020 msgr2=0x7fcfe8060330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:05.641 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfed81d700 1 --2- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80aa020 0x7fcfe8060330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfed81d700 1 -- 172.21.15.44:0/2883868171 >> v1:172.21.15.44:6789/0 conn(0x7fcfe80a5f80 legacy=0x7fcfe8136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfed81d700 1 -- 172.21.15.44:0/2883868171 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe80a6c80 con 0x7fcfe80a8af0 2020-11-20T12:22:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfed81d700 1 --2- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfe80a8af0 0x7fcfe80a8fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fcfe0014940 tx=0x7fcfe000fa10).ready entity=mon.1 client_cookie=194f502a66f34766 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfd7fff700 1 -- 172.21.15.44:0/2883868171 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0005e50 con 0x7fcfe80a8af0 2020-11-20T12:22:05.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfd7fff700 1 -- 172.21.15.44:0/2883868171 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe000f0c0 con 0x7fcfe80a8af0 2020-11-20T12:22:05.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.635+0000 7fcfd7fff700 1 -- 172.21.15.44:0/2883868171 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0012840 con 0x7fcfe80a8af0 2020-11-20T12:22:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.636+0000 7fcfef280700 1 -- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfe80a8af0 msgr2=0x7fcfe80a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.636+0000 7fcfef280700 1 --2- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfe80a8af0 0x7fcfe80a8fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fcfe0014940 tx=0x7fcfe000fa10).stop 2020-11-20T12:22:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.637+0000 7fcfef280700 1 -- 172.21.15.44:0/2883868171 shutdown_connections 2020-11-20T12:22:05.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.638+0000 7fcfef280700 1 --2- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfe80a8af0 0x7fcfe80a8fa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.638+0000 7fcfef280700 1 --2- 172.21.15.44:0/2883868171 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80aa020 0x7fcfe8060330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.638+0000 7fcfef280700 1 -- 172.21.15.44:0/2883868171 >> 172.21.15.44:0/2883868171 conn(0x7fcfe8062600 msgr2=0x7fcfe80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.638+0000 7fcfef280700 1 -- 172.21.15.44:0/2883868171 shutdown_connections 2020-11-20T12:22:05.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.638+0000 7fcfef280700 1 -- 172.21.15.44:0/2883868171 wait complete. 2020-11-20T12:22:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.639+0000 7fcfef280700 1 Processor -- start 2020-11-20T12:22:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.639+0000 7fcfef280700 1 -- start start 2020-11-20T12:22:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfef280700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5f80 0x7fcfe813c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfef280700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80a8af0 0x7fcfe8146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfef280700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfe80aa020 0x7fcfe8133480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfef280700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8145b10 con 0x7fcfe80a5f80 2020-11-20T12:22:05.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfef280700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe814a020 con 0x7fcfe80aa020 2020-11-20T12:22:05.647 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfef280700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8145d00 con 0x7fcfe80a8af0 2020-11-20T12:22:05.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5f80 0x7fcfe813c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5f80 0x7fcfe813c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43606/0 (socket says 172.21.15.44:43606) 2020-11-20T12:22:05.648 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 -- 172.21.15.44:0/2158748351 learned_addr learned my addr 172.21.15.44:0/2158748351 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:05.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80a8af0 msgr2=0x7fcfe8146400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:05.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe80a8af0 0x7fcfe8146400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:05.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfe80aa020 msgr2=0x7fcfe8133480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:05.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcfe80aa020 0x7fcfe8133480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.641+0000 7fcfed01c700 1 -- 172.21.15.44:0/2158748351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8145f60 con 0x7fcfe80a5f80 2020-11-20T12:22:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.642+0000 7fcfed01c700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5f80 0x7fcfe813c4f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fcfe400d8e0 tx=0x7fcfe4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.645+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4009790 con 0x7fcfe80a5f80 2020-11-20T12:22:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.645+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe40098f0 con 0x7fcfe80a5f80 2020-11-20T12:22:05.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.645+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4009c70 con 0x7fcfe80a5f80 2020-11-20T12:22:05.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.645+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe813a410 con 0x7fcfe80a5f80 2020-11-20T12:22:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.645+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe80664b0 con 0x7fcfe80a5f80 2020-11-20T12:22:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.647+0000 7fcfef280700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8069560 0x7fcfe81495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.647+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8061650 con 0x7fcfe8069560 2020-11-20T12:22:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.647+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcfe4013480 con 0x7fcfe80a5f80 2020-11-20T12:22:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.647+0000 7fcfec81b700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8069560 0x7fcfe81495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfec81b700 1 -- 172.21.15.44:0/2158748351 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfe813c110 con 0x7fcfe8069560 2020-11-20T12:22:05.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfec81b700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8069560 0x7fcfe81495f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fcfe813e0a0 tx=0x7fcfd80079e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcfc0056d70 0x7fcfc0059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fcfe40703d0 con 0x7fcfe80a5f80 2020-11-20T12:22:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80072e0 con 0x7fcfe8069560 2020-11-20T12:22:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fcfd8007440 con 0x7fcfe8069560 2020-11-20T12:22:05.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8069560 msgr2=0x7fcfe81495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8069560 0x7fcfe81495f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fcfe813e0a0 tx=0x7fcfd80079e0).stop 2020-11-20T12:22:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8069560 msgr2=0x7fcfe81495f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.648+0000 7fcfd57fa700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8069560 0x7fcfe81495f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.649+0000 7fcfed81d700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcfc0056d70 0x7fcfc0059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.649+0000 7fcfed81d700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcfc0056d70 0x7fcfc0059230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fcfe813dea0 tx=0x7fcfe000f000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.664+0000 7fcfef280700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8149ae0 0x7fcfe813aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.664+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80a9910 con 0x7fcfe8149ae0 2020-11-20T12:22:05.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.664+0000 7fcfec81b700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8149ae0 0x7fcfe813aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.665+0000 7fcfec81b700 1 -- 172.21.15.44:0/2158748351 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcfe80446d0 con 0x7fcfe8149ae0 2020-11-20T12:22:05.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.666+0000 7fcfec81b700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8149ae0 0x7fcfe813aed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fcfe813e0a0 tx=0x7fcfd8013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.666+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8015910 con 0x7fcfe8149ae0 2020-11-20T12:22:05.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.666+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7fcfd80070d0 con 0x7fcfe8149ae0 2020-11-20T12:22:05.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.666+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8149ae0 msgr2=0x7fcfe813aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.666+0000 7fcfd57fa700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8149ae0 0x7fcfe813aed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fcfe813e0a0 tx=0x7fcfd8013000).stop 2020-11-20T12:22:05.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.666+0000 7fcfd57fa700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8149ae0 msgr2=0x7fcfe813aed0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:05.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.666+0000 7fcfd57fa700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcfe8149ae0 0x7fcfe813aed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.671 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:05.671 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:05.671 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:05.671 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:22:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 182, 2020-11-20T12:22:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:22:05.672 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 14, 2020-11-20T12:22:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:05.673 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.675 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.676 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.677 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:05.679 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:05.680 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:05.681 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:05.682 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.683 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:05.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.684 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:05.684 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:05.684 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.684 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.685 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:05.685 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.686 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:05.686 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.687 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.687 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:05.687 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:05.687 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.688 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.688 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.688 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:05.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:05.689 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.689 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:05.690 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:05.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.691 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.692 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:05.692 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:05.692 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.692 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.693 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.693 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:05.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:05.694 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:05.694 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:05.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:05.694 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:05.695 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:05.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:05.696 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:05.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.696 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.696 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:05.697 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:05.697 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:05.697 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:05.697 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:05.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.698 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.698 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:05.698 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:05.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.699 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.699 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.699 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:05.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.699 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.700 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:22:05.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.700 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.700 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.700 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:05.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.701 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.701 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:22:05.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.701 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.702 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.702 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:05.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.702 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:05.702 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:05.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:05.703 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:05.703 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:05.703 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:22:05.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:05.704 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:05.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:05.704 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:05.704 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:05.705 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.671+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcfc0056d70 msgr2=0x7fcfc0059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.671+0000 7fcfef280700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcfc0056d70 0x7fcfc0059230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fcfe813dea0 tx=0x7fcfe000f000).stop 2020-11-20T12:22:05.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.671+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5f80 msgr2=0x7fcfe813c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.671+0000 7fcfef280700 1 --2- 172.21.15.44:0/2158748351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5f80 0x7fcfe813c4f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fcfe400d8e0 tx=0x7fcfe4008040).stop 2020-11-20T12:22:05.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.672+0000 7fcfed81d700 1 -- 172.21.15.44:0/2158748351 reap_dead start 2020-11-20T12:22:05.706 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.672+0000 7fcfed81d700 1 -- 172.21.15.44:0/2158748351 reap_dead start 2020-11-20T12:22:05.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.677+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 shutdown_connections 2020-11-20T12:22:05.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.677+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 >> 172.21.15.44:0/2158748351 conn(0x7fcfe8062600 msgr2=0x7fcfe8136ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.682+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 shutdown_connections 2020-11-20T12:22:05.707 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.682+0000 7fcfef280700 1 -- 172.21.15.44:0/2158748351 wait complete. 2020-11-20T12:22:05.708 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:22:05.709 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:22:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.921+0000 7f6ef2482700 1 Processor -- start 2020-11-20T12:22:05.924 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.921+0000 7f6ef2482700 1 -- start start 2020-11-20T12:22:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.921+0000 7f6ef2482700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6eec068f60 0x7f6eec0a9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.921+0000 7f6ef2482700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec13c520 con 0x7f6eec135720 2020-11-20T12:22:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.921+0000 7f6ef2482700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec0a80f0 con 0x7f6eec13c0a0 2020-11-20T12:22:05.926 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.921+0000 7f6ef2482700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec0a9710 con 0x7f6eec068f60 2020-11-20T12:22:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6ef0a1f700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6eec135720 0x7f6eec135bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47362/0 (socket says 172.21.15.44:47362) 2020-11-20T12:22:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6ef0a1f700 1 -- 172.21.15.44:0/1582925203 learned_addr learned my addr 172.21.15.44:0/1582925203 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:22:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eebfff700 1 --2- 172.21.15.44:0/1582925203 >> v2:172.21.15.79:3300/0 conn(0x7f6eec068f60 0x7f6eec0a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eeb7fe700 1 -- 172.21.15.44:0/1582925203 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261746621 0 0) 0x7f6ee4001450 con 0x7f6eec135720 2020-11-20T12:22:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eeb7fe700 1 -- 172.21.15.44:0/1582925203 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ecc003240 con 0x7f6eec135720 2020-11-20T12:22:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eebfff700 1 -- 172.21.15.44:0/1582925203 >> v1:172.21.15.79:6789/0 conn(0x7f6eec13c0a0 legacy=0x7f6eec0a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eeb7fe700 1 -- 172.21.15.44:0/1582925203 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279136880 0 0) 0x7f6eec0a80f0 con 0x7f6eec13c0a0 2020-11-20T12:22:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eebfff700 1 -- 172.21.15.44:0/1582925203 >> v1:172.21.15.44:6789/0 conn(0x7f6eec135720 legacy=0x7f6eec135bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eebfff700 1 -- 172.21.15.44:0/1582925203 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0a5970 con 0x7f6eec068f60 2020-11-20T12:22:05.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eebfff700 1 --2- 172.21.15.44:0/1582925203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec068f60 0x7f6eec0a9000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f6edc00b9a0 tx=0x7f6edc006df0).ready entity=mon.1 client_cookie=37fbceaf898e49f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.922+0000 7f6eeb7fe700 1 -- 172.21.15.44:0/1582925203 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc009840 con 0x7f6eec068f60 2020-11-20T12:22:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6eeb7fe700 1 -- 172.21.15.44:0/1582925203 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc0099a0 con 0x7f6eec068f60 2020-11-20T12:22:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6eeb7fe700 1 -- 172.21.15.44:0/1582925203 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc009d20 con 0x7f6eec068f60 2020-11-20T12:22:05.932 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6ef2482700 1 -- 172.21.15.44:0/1582925203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec068f60 msgr2=0x7f6eec0a9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6ef2482700 1 --2- 172.21.15.44:0/1582925203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec068f60 0x7f6eec0a9000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f6edc00b9a0 tx=0x7f6edc006df0).stop 2020-11-20T12:22:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6ef2482700 1 -- 172.21.15.44:0/1582925203 shutdown_connections 2020-11-20T12:22:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6ef2482700 1 --2- 172.21.15.44:0/1582925203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec068f60 0x7f6eec0a9000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.933 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6ef2482700 1 -- 172.21.15.44:0/1582925203 >> 172.21.15.44:0/1582925203 conn(0x7f6eec0ae1b0 msgr2=0x7f6eec0aa180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6ef2482700 1 -- 172.21.15.44:0/1582925203 shutdown_connections 2020-11-20T12:22:05.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.923+0000 7f6ef2482700 1 -- 172.21.15.44:0/1582925203 wait complete. 2020-11-20T12:22:05.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.924+0000 7f6ef2482700 1 Processor -- start 2020-11-20T12:22:05.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.924+0000 7f6ef2482700 1 -- start start 2020-11-20T12:22:05.934 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef2482700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eec068f60 0x7f6eec13b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef2482700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec135720 0x7f6eec130a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef2482700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eec13c0a0 0x7f6eec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef2482700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0a73e0 con 0x7f6eec068f60 2020-11-20T12:22:05.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef2482700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0a7680 con 0x7f6eec135720 2020-11-20T12:22:05.935 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef2482700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0a7920 con 0x7f6eec13c0a0 2020-11-20T12:22:05.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eec068f60 0x7f6eec13b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eec068f60 0x7f6eec13b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43620/0 (socket says 172.21.15.44:43620) 2020-11-20T12:22:05.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 -- 172.21.15.44:0/2722682976 learned_addr learned my addr 172.21.15.44:0/2722682976 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:05.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ee3fff700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec135720 0x7f6eec130a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef0a1f700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eec13c0a0 0x7f6eec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 -- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eec13c0a0 msgr2=0x7f6eec0a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eec13c0a0 0x7f6eec0a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 -- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec135720 msgr2=0x7f6eec130a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec135720 0x7f6eec130a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 -- 172.21.15.44:0/2722682976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0a7060 con 0x7f6eec068f60 2020-11-20T12:22:05.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6eebfff700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eec068f60 0x7f6eec13b8d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f6edc00be60 tx=0x7f6edc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ee8ff9700 1 -- 172.21.15.44:0/2722682976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc011dd0 con 0x7f6eec068f60 2020-11-20T12:22:05.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ee8ff9700 1 -- 172.21.15.44:0/2722682976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc01e460 con 0x7f6eec068f60 2020-11-20T12:22:05.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.925+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec131170 con 0x7f6eec068f60 2020-11-20T12:22:05.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.926+0000 7f6ee8ff9700 1 -- 172.21.15.44:0/2722682976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc01e7e0 con 0x7f6eec068f60 2020-11-20T12:22:05.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.926+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec0a6910 con 0x7f6eec068f60 2020-11-20T12:22:05.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.927+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec13cf80 con 0x7f6eec068f60 2020-11-20T12:22:05.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.927+0000 7f6ee8ff9700 1 -- 172.21.15.44:0/2722682976 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f6edc025030 con 0x7f6eec068f60 2020-11-20T12:22:05.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.927+0000 7f6ee8ff9700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ec8056d20 0x7f6ec80591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.927+0000 7f6ee8ff9700 1 -- 172.21.15.44:0/2722682976 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f6edc0717f0 con 0x7f6eec068f60 2020-11-20T12:22:05.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.928+0000 7f6ee3fff700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ec8056d20 0x7f6ec80591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.928+0000 7f6ee3fff700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ec8056d20 0x7f6ec80591e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f6eec13bdc0 tx=0x7f6ed4003040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:05.933+0000 7f6ee8ff9700 1 -- 172.21.15.44:0/2722682976 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6edc046b70 con 0x7f6eec068f60 2020-11-20T12:22:06.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.276+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f6eec0a71f0 con 0x7f6eec068f60 2020-11-20T12:22:06.279 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.276+0000 7f6ee8ff9700 1 -- 172.21.15.44:0/2722682976 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f6edc023020 con 0x7f6eec068f60 2020-11-20T12:22:06.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.278+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ec8056d20 msgr2=0x7f6ec80591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:06.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.278+0000 7f6ef2482700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ec8056d20 0x7f6ec80591e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f6eec13bdc0 tx=0x7f6ed4003040).stop 2020-11-20T12:22:06.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.278+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eec068f60 msgr2=0x7f6eec13b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:06.282 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.278+0000 7f6ef2482700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eec068f60 0x7f6eec13b8d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f6edc00be60 tx=0x7f6edc015000).stop 2020-11-20T12:22:06.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.278+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 shutdown_connections 2020-11-20T12:22:06.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.279+0000 7f6ef2482700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f6ec8056d20 0x7f6ec80591e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.279+0000 7f6ef2482700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f6eec13c0a0 0x7f6eec0a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.279+0000 7f6ef2482700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6eec068f60 0x7f6eec13b8d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.283 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.279+0000 7f6ef2482700 1 --2- 172.21.15.44:0/2722682976 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6eec135720 0x7f6eec130a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.279+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 >> 172.21.15.44:0/2722682976 conn(0x7f6eec0ae1b0 msgr2=0x7f6eec135bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.279+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 shutdown_connections 2020-11-20T12:22:06.284 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:06.279+0000 7f6ef2482700 1 -- 172.21.15.44:0/2722682976 wait complete. 2020-11-20T12:22:06.296 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:22:09.748 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:09.754 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:16.297 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:22:16.300 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:17.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.058+0000 7fe22a9dd700 1 Processor -- start 2020-11-20T12:22:17.063 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.059+0000 7fe22a9dd700 1 -- start start 2020-11-20T12:22:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe22a9dd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe22a9dd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe2240ab0e0 0x7fe22413ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe22a9dd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2241385f0 con 0x7fe2240a86d0 2020-11-20T12:22:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe22a9dd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22413e250 con 0x7fe2240ab0e0 2020-11-20T12:22:17.067 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe22a9dd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22413e4f0 con 0x7fe2240aad00 2020-11-20T12:22:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe2237fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe2240ab0e0 0x7fe22413ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe2237fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe2240ab0e0 0x7fe22413ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35678/0 (socket says 172.21.15.44:35678) 2020-11-20T12:22:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe2237fe700 1 -- 172.21.15.44:0/3109708879 learned_addr learned my addr 172.21.15.44:0/3109708879 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.062+0000 7fe223fff700 1 --2- 172.21.15.44:0/3109708879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.068 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.063+0000 7fe223fff700 1 -- 172.21.15.44:0/3109708879 >> v2:172.21.15.79:3300/0 conn(0x7fe2240ab0e0 msgr2=0x7fe22413ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.063+0000 7fe223fff700 1 --2- 172.21.15.44:0/3109708879 >> v2:172.21.15.79:3300/0 conn(0x7fe2240ab0e0 0x7fe22413ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.063+0000 7fe223fff700 1 -- 172.21.15.44:0/3109708879 >> v1:172.21.15.79:6789/0 conn(0x7fe2240a86d0 legacy=0x7fe2240abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.063+0000 7fe223fff700 1 -- 172.21.15.44:0/3109708879 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2240ac340 con 0x7fe2240aad00 2020-11-20T12:22:17.069 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.063+0000 7fe223fff700 1 --2- 172.21.15.44:0/3109708879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240a94d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe21400b9a0 tx=0x7fe214006df0).ready entity=mon.2 client_cookie=842b93b1812ed8ed server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.070 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.065+0000 7fe222ffd700 1 -- 172.21.15.44:0/3109708879 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214009800 con 0x7fe2240aad00 2020-11-20T12:22:17.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.065+0000 7fe222ffd700 1 -- 172.21.15.44:0/3109708879 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214009960 con 0x7fe2240aad00 2020-11-20T12:22:17.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.066+0000 7fe222ffd700 1 -- 172.21.15.44:0/3109708879 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214009ce0 con 0x7fe2240aad00 2020-11-20T12:22:17.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.067+0000 7fe22a9dd700 1 -- 172.21.15.44:0/3109708879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 msgr2=0x7fe2240a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.071 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.067+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/3109708879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240a94d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe21400b9a0 tx=0x7fe214006df0).stop 2020-11-20T12:22:17.074 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.071+0000 7fe22a9dd700 1 -- 172.21.15.44:0/3109708879 shutdown_connections 2020-11-20T12:22:17.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.071+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/3109708879 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240a94d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.071+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/3109708879 >> v2:172.21.15.79:3300/0 conn(0x7fe2240ab0e0 0x7fe22413ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.071+0000 7fe22a9dd700 1 -- 172.21.15.44:0/3109708879 >> 172.21.15.44:0/3109708879 conn(0x7fe224060170 msgr2=0x7fe224060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:17.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.071+0000 7fe22a9dd700 1 -- 172.21.15.44:0/3109708879 shutdown_connections 2020-11-20T12:22:17.075 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.071+0000 7fe22a9dd700 1 -- 172.21.15.44:0/3109708879 wait complete. 2020-11-20T12:22:17.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.073+0000 7fe22a9dd700 1 Processor -- start 2020-11-20T12:22:17.076 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.073+0000 7fe22a9dd700 1 -- start start 2020-11-20T12:22:17.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe22a9dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2240a86d0 0x7fe224067820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.078 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe22a9dd700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe22a9dd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2240ab0e0 0x7fe224065780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe22a9dd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe224065290 con 0x7fe2240ab0e0 2020-11-20T12:22:17.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe22a9dd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe224064ac0 con 0x7fe2240a86d0 2020-11-20T12:22:17.079 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe22a9dd700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe224064d50 con 0x7fe2240aad00 2020-11-20T12:22:17.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55072/0 (socket says 172.21.15.44:55072) 2020-11-20T12:22:17.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 -- 172.21.15.44:0/296120936 learned_addr learned my addr 172.21.15.44:0/296120936 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:17.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe228f7a700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2240ab0e0 0x7fe224065780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.080 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 -- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2240a86d0 msgr2=0x7fe224067820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2240a86d0 0x7fe224067820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 -- 172.21.15.44:0/296120936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2240ab0e0 msgr2=0x7fe224065780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2240ab0e0 0x7fe224065780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.074+0000 7fe2237fe700 1 -- 172.21.15.44:0/296120936 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe224064ff0 con 0x7fe2240aad00 2020-11-20T12:22:17.081 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.075+0000 7fe2237fe700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240645d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe218015fa0 tx=0x7fe21800f850).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.075+0000 7fe20bfff700 1 -- 172.21.15.44:0/296120936 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218013c90 con 0x7fe2240aad00 2020-11-20T12:22:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.075+0000 7fe20bfff700 1 -- 172.21.15.44:0/296120936 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe218017040 con 0x7fe2240aad00 2020-11-20T12:22:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.075+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2240a7b60 con 0x7fe2240aad00 2020-11-20T12:22:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.075+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe224065e30 con 0x7fe2240aad00 2020-11-20T12:22:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.075+0000 7fe20bfff700 1 -- 172.21.15.44:0/296120936 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21801b3f0 con 0x7fe2240aad00 2020-11-20T12:22:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.076+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe224066130 con 0x7fe2240aad00 2020-11-20T12:22:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.077+0000 7fe20bfff700 1 -- 172.21.15.44:0/296120936 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fe21801b760 con 0x7fe2240aad00 2020-11-20T12:22:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.077+0000 7fe20bfff700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe200056d20 0x7fe2000591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.077+0000 7fe20bfff700 1 -- 172.21.15.44:0/296120936 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fe218078300 con 0x7fe2240aad00 2020-11-20T12:22:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.077+0000 7fe223fff700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe200056d20 0x7fe2000591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.078+0000 7fe223fff700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe200056d20 0x7fe2000591e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fe214006e30 tx=0x7fe214015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.082+0000 7fe20bfff700 1 -- 172.21.15.44:0/296120936 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe21804d570 con 0x7fe2240aad00 2020-11-20T12:22:17.510 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.507+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe224069680 con 0x7fe2240aad00 2020-11-20T12:22:17.515 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.511+0000 7fe20bfff700 1 -- 172.21.15.44:0/296120936 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7fe218050b90 con 0x7fe2240aad00 2020-11-20T12:22:17.518 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":182,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":26,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe200056d20 msgr2=0x7fe2000591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe200056d20 0x7fe2000591e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fe214006e30 tx=0x7fe214015000).stop 2020-11-20T12:22:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 msgr2=0x7fe2240645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.520 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240645d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe218015fa0 tx=0x7fe21800f850).stop 2020-11-20T12:22:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 shutdown_connections 2020-11-20T12:22:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fe200056d20 0x7fe2000591e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2240ab0e0 0x7fe224065780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2240a86d0 0x7fe224067820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:17.521 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 --2- 172.21.15.44:0/296120936 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fe2240aad00 0x7fe2240645d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 >> 172.21.15.44:0/296120936 conn(0x7fe224060170 msgr2=0x7fe2240ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:17.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 shutdown_connections 2020-11-20T12:22:17.522 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.517+0000 7fe22a9dd700 1 -- 172.21.15.44:0/296120936 wait complete. 2020-11-20T12:22:17.545 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:22:17.546 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2020-11-20T12:22:17.546 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:22:17.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.787+0000 7fcc36f26700 1 Processor -- start 2020-11-20T12:22:17.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.787+0000 7fcc36f26700 1 -- start start 2020-11-20T12:22:17.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.788+0000 7fcc36f26700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcc3013f930 0x7fcc3013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.788+0000 7fcc36f26700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc300a57b0 con 0x7fcc301479e0 2020-11-20T12:22:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.788+0000 7fcc36f26700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc300a80c0 con 0x7fcc300aa1e0 2020-11-20T12:22:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.788+0000 7fcc36f26700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc300a4b80 con 0x7fcc3013f930 2020-11-20T12:22:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.788+0000 7fcc2ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcc3013f930 0x7fcc3013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.789+0000 7fcc2ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcc3013f930 0x7fcc3013fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35692/0 (socket says 172.21.15.44:35692) 2020-11-20T12:22:17.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.789+0000 7fcc2ffff700 1 -- 172.21.15.44:0/2425343759 learned_addr learned my addr 172.21.15.44:0/2425343759 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:17.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.790+0000 7fcc2ffff700 1 -- 172.21.15.44:0/2425343759 >> v1:172.21.15.79:6789/0 conn(0x7fcc300aa1e0 legacy=0x7fcc300a5b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2020-11-20T12:22:17.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.790+0000 7fcc2ffff700 1 -- 172.21.15.44:0/2425343759 >> v1:172.21.15.44:6789/0 conn(0x7fcc301479e0 legacy=0x7fcc30149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:17.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.790+0000 7fcc2ffff700 1 -- 172.21.15.44:0/2425343759 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc300a82f0 con 0x7fcc3013f930 2020-11-20T12:22:17.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.793+0000 7fcc2ffff700 1 --2- 172.21.15.44:0/2425343759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc3013fd90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcc20006fd0 tx=0x7fcc20006eb0).ready entity=mon.1 client_cookie=41592fe2874a089a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.794+0000 7fcc2f7fe700 1 -- 172.21.15.44:0/2425343759 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20009850 con 0x7fcc3013f930 2020-11-20T12:22:17.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.794+0000 7fcc2f7fe700 1 -- 172.21.15.44:0/2425343759 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc200099b0 con 0x7fcc3013f930 2020-11-20T12:22:17.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.794+0000 7fcc2f7fe700 1 -- 172.21.15.44:0/2425343759 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20009d30 con 0x7fcc3013f930 2020-11-20T12:22:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.796+0000 7fcc36f26700 1 -- 172.21.15.44:0/2425343759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 msgr2=0x7fcc3013fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.796+0000 7fcc36f26700 1 --2- 172.21.15.44:0/2425343759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc3013fd90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcc20006fd0 tx=0x7fcc20006eb0).stop 2020-11-20T12:22:17.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.798+0000 7fcc36f26700 1 -- 172.21.15.44:0/2425343759 shutdown_connections 2020-11-20T12:22:17.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.798+0000 7fcc36f26700 1 --2- 172.21.15.44:0/2425343759 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc3013fd90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.798+0000 7fcc36f26700 1 -- 172.21.15.44:0/2425343759 >> 172.21.15.44:0/2425343759 conn(0x7fcc30062600 msgr2=0x7fcc300a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:17.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.798+0000 7fcc36f26700 1 -- 172.21.15.44:0/2425343759 shutdown_connections 2020-11-20T12:22:17.803 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.798+0000 7fcc36f26700 1 -- 172.21.15.44:0/2425343759 wait complete. 2020-11-20T12:22:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.802+0000 7fcc36f26700 1 Processor -- start 2020-11-20T12:22:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.802+0000 7fcc36f26700 1 -- start start 2020-11-20T12:22:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.803+0000 7fcc36f26700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc300aa1e0 0x7fcc3013d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.803+0000 7fcc36f26700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc30144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.803+0000 7fcc36f26700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcc301479e0 0x7fcc30144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.803+0000 7fcc36f26700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30145470 con 0x7fcc300aa1e0 2020-11-20T12:22:17.807 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.803+0000 7fcc36f26700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30145f00 con 0x7fcc3013f930 2020-11-20T12:22:17.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.803+0000 7fcc36f26700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc301461a0 con 0x7fcc301479e0 2020-11-20T12:22:17.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc30144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc30144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35696/0 (socket says 172.21.15.44:35696) 2020-11-20T12:22:17.808 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc34cc2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc300aa1e0 0x7fcc3013d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.809 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 -- 172.21.15.44:0/4169763369 learned_addr learned my addr 172.21.15.44:0/4169763369 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcc301479e0 msgr2=0x7fcc30144f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcc301479e0 0x7fcc30144f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc300aa1e0 msgr2=0x7fcc3013d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc300aa1e0 0x7fcc3013d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 -- 172.21.15.44:0/4169763369 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc30146440 con 0x7fcc3013f930 2020-11-20T12:22:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2ffff700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc30144a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcc200119c0 tx=0x7fcc20015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.804+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2001e460 con 0x7fcc3013f930 2020-11-20T12:22:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.805+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2001e5c0 con 0x7fcc3013f930 2020-11-20T12:22:17.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.805+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2001e960 con 0x7fcc3013f930 2020-11-20T12:22:17.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.805+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3013e140 con 0x7fcc3013f930 2020-11-20T12:22:17.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.805+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3009da10 con 0x7fcc3013f930 2020-11-20T12:22:17.813 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.806+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fcc20025030 con 0x7fcc3013f930 2020-11-20T12:22:17.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.807+0000 7fcc2cff9700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcc08056d70 0x7fcc08059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.807+0000 7fcc34cc2700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcc08056d70 0x7fcc08059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.807+0000 7fcc36f26700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30069560 0x7fcc3013c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.814 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.807+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3013e330 con 0x7fcc30069560 2020-11-20T12:22:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.808+0000 7fcc354c3700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30069560 0x7fcc3013c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.815 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.808+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fcc20071cf0 con 0x7fcc3013f930 2020-11-20T12:22:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.808+0000 7fcc34cc2700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcc08056d70 0x7fcc08059230 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fcc24006fd0 tx=0x7fcc24008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.808+0000 7fcc354c3700 1 -- 172.21.15.44:0/4169763369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc301441f0 con 0x7fcc30069560 2020-11-20T12:22:17.816 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.809+0000 7fcc354c3700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30069560 0x7fcc3013c120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fcc2800b960 tx=0x7fcc28006f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.809+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2800c620 con 0x7fcc30069560 2020-11-20T12:22:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.809+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fcc2800c780 con 0x7fcc30069560 2020-11-20T12:22:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.809+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30069560 msgr2=0x7fcc3013c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.817 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.809+0000 7fcc2cff9700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30069560 0x7fcc3013c120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fcc2800b960 tx=0x7fcc28006f10).stop 2020-11-20T12:22:17.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.809+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30069560 msgr2=0x7fcc3013c120 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:17.818 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.809+0000 7fcc2cff9700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc30069560 0x7fcc3013c120 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0x7fcc2800b960 tx=0x7fcc28006f10).stop 2020-11-20T12:22:17.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.827+0000 7fcc36f26700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc3013c610 0x7fcc3015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.831 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.827+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300a6ec0 con 0x7fcc3013c610 2020-11-20T12:22:17.834 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.831+0000 7fcc354c3700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc3013c610 0x7fcc3015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.835 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.832+0000 7fcc354c3700 1 -- 172.21.15.44:0/4169763369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc30145d10 con 0x7fcc3013c610 2020-11-20T12:22:17.838 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.835+0000 7fcc354c3700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc3013c610 0x7fcc3015b2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fcc28013000 tx=0x7fcc28014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.839 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:17.839 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:17.840 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:17.840 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:22:17.840 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 182, 2020-11-20T12:22:17.840 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:17.841 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:17.841 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:22:17.841 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:17.841 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:17.841 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 26, 2020-11-20T12:22:17.842 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:17.842 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:17.842 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:17.842 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:17.842 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:17.843 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:17.843 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:17.843 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:17.843 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:17.843 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:17.844 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:17.844 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:17.844 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:17.844 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:17.844 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:17.845 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:17.845 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:17.845 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:17.845 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:17.846 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:17.846 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:17.846 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:17.846 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:17.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.847 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:17.847 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:17.847 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:17.848 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:17.848 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:17.848 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:17.848 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:17.848 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:17.849 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:17.849 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:17.849 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:17.849 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:17.850 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:17.850 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:17.850 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:17.850 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:17.851 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:17.851 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:17.851 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:17.851 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:17.851 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:17.852 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:17.852 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:17.852 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:17.852 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:17.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.853 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:17.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.853 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:17.853 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:17.853 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:17.854 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:17.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:17.854 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:17.854 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:17.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:17.855 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:17.856 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:17.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.856 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:17.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.856 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:17.857 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:17.857 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:17.857 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:17.857 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:17.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.858 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:17.858 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:17.858 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:17.858 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:17.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.859 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:17.859 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:17.859 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:17.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:17.860 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:17.860 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:17.860 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.861 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:17.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.861 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:17.861 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:17.861 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:17.862 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:17.862 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:17.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.862 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:17.864 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:17.864 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:17.865 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:17.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:17.866 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:17.866 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:17.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.867 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:17.868 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:17.868 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:17.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.868 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:17.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.869 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:17.869 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:17.869 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:17.869 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:17.870 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:17.870 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.870 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:17.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.871 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:17.871 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:17.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.872 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:17.872 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:17.872 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:17.872 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.873 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:17.873 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:22:17.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.873 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:17.873 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:17.874 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:17.874 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.874 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:17.874 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:22:17.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.875 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:17.875 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:17.875 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:22:17.875 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.876 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:17.876 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:17.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.876 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:17.876 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:17.877 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4 2020-11-20T12:22:17.877 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.877 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:17.877 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:22:17.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:17.878 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:17.878 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:17.878 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:22:17.878 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:17.879 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.836+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc28021400 con 0x7fcc3013c610 2020-11-20T12:22:17.881 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.836+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4779 (secure 0 0 0) 0x7fcc28021560 con 0x7fcc3013c610 2020-11-20T12:22:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.836+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc3013c610 msgr2=0x7fcc3015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.836+0000 7fcc2cff9700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc3013c610 0x7fcc3015b2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fcc28013000 tx=0x7fcc28014000).stop 2020-11-20T12:22:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.836+0000 7fcc2cff9700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc3013c610 msgr2=0x7fcc3015b2f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.836+0000 7fcc2cff9700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc3013c610 0x7fcc3015b2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.882 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcc08056d70 msgr2=0x7fcc08059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc36f26700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fcc08056d70 0x7fcc08059230 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fcc24006fd0 tx=0x7fcc24008040).stop 2020-11-20T12:22:17.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc354c3700 1 -- 172.21.15.44:0/4169763369 reap_dead start 2020-11-20T12:22:17.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 msgr2=0x7fcc30144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.883 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc36f26700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc30144a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fcc200119c0 tx=0x7fcc20015000).stop 2020-11-20T12:22:17.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 shutdown_connections 2020-11-20T12:22:17.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc36f26700 1 --2- 172.21.15.44:0/4169763369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc3013f930 0x7fcc30144a90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.838+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 >> 172.21.15.44:0/4169763369 conn(0x7fcc30062600 msgr2=0x7fcc3005ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:17.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.842+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 shutdown_connections 2020-11-20T12:22:17.884 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:17.842+0000 7fcc36f26700 1 -- 172.21.15.44:0/4169763369 wait complete. 2020-11-20T12:22:17.885 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:22:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.127+0000 7ffb5c6a9700 1 Processor -- start 2020-11-20T12:22:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.127+0000 7ffb5c6a9700 1 -- start start 2020-11-20T12:22:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.128+0000 7ffb5c6a9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb540a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.128+0000 7ffb5c6a9700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb540a6730 con 0x7ffb540aa1e0 2020-11-20T12:22:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.128+0000 7ffb5c6a9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb540a7280 con 0x7ffb540a62a0 2020-11-20T12:22:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.128+0000 7ffb5c6a9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540a7090 con 0x7ffb540a4800 2020-11-20T12:22:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59c44700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb540a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59c44700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb540a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55094/0 (socket says 172.21.15.44:55094) 2020-11-20T12:22:18.137 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59c44700 1 -- 172.21.15.44:0/229821872 learned_addr learned my addr 172.21.15.44:0/229821872 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:18.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59c44700 1 -- 172.21.15.44:0/229821872 >> v1:172.21.15.79:6789/0 conn(0x7ffb540a62a0 legacy=0x7ffb541413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59c44700 1 -- 172.21.15.44:0/229821872 >> v1:172.21.15.44:6789/0 conn(0x7ffb540aa1e0 legacy=0x7ffb541372e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:18.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59c44700 1 -- 172.21.15.44:0/229821872 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540a84c0 con 0x7ffb540a4800 2020-11-20T12:22:18.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59c44700 1 --2- 172.21.15.44:0/229821872 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb540a5db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ffb4400ba20 tx=0x7ffb44006eb0).ready entity=mon.2 client_cookie=64a9b52570c23cd7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.138 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59443700 1 -- 172.21.15.44:0/229821872 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44009890 con 0x7ffb540a4800 2020-11-20T12:22:18.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59443700 1 -- 172.21.15.44:0/229821872 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb440099f0 con 0x7ffb540a4800 2020-11-20T12:22:18.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.129+0000 7ffb59443700 1 -- 172.21.15.44:0/229821872 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44009d70 con 0x7ffb540a4800 2020-11-20T12:22:18.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.130+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/229821872 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 msgr2=0x7ffb540a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.139 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.130+0000 7ffb5c6a9700 1 --2- 172.21.15.44:0/229821872 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb540a5db0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ffb4400ba20 tx=0x7ffb44006eb0).stop 2020-11-20T12:22:18.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.131+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/229821872 shutdown_connections 2020-11-20T12:22:18.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.131+0000 7ffb5c6a9700 1 --2- 172.21.15.44:0/229821872 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb540a5db0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.140 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.131+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/229821872 >> 172.21.15.44:0/229821872 conn(0x7ffb54062690 msgr2=0x7ffb540a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.142 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.138+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/229821872 shutdown_connections 2020-11-20T12:22:18.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.139+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/229821872 wait complete. 2020-11-20T12:22:18.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.141+0000 7ffb5c6a9700 1 Processor -- start 2020-11-20T12:22:18.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.141+0000 7ffb5c6a9700 1 -- start start 2020-11-20T12:22:18.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.142+0000 7ffb5c6a9700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb5414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.142+0000 7ffb5c6a9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb540a62a0 0x7ffb5413b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.142+0000 7ffb5c6a9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb5414a150 0x7ffb5414a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.142+0000 7ffb5c6a9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5413bb40 con 0x7ffb5414a150 2020-11-20T12:22:18.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.142+0000 7ffb5c6a9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5413dec0 con 0x7ffb540a62a0 2020-11-20T12:22:18.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.142+0000 7ffb5c6a9700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5413b8b0 con 0x7ffb540a4800 2020-11-20T12:22:18.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb5414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb5414af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55098/0 (socket says 172.21.15.44:55098) 2020-11-20T12:22:18.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 -- 172.21.15.44:0/2279663802 learned_addr learned my addr 172.21.15.44:0/2279663802 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:18.150 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb540a62a0 msgr2=0x7ffb5413b3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:18.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb540a62a0 0x7ffb5413b3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:18.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb5414a150 msgr2=0x7ffb5414a590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:18.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb5414a150 0x7ffb5414a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:18.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 -- 172.21.15.44:0/2279663802 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5413d780 con 0x7ffb540a4800 2020-11-20T12:22:18.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb5a445700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb5414af50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7ffb500096e0 tx=0x7ffb50006d80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000d560 con 0x7ffb540a4800 2020-11-20T12:22:18.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5000d6c0 con 0x7ffb540a4800 2020-11-20T12:22:18.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.143+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000da80 con 0x7ffb540a4800 2020-11-20T12:22:18.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.144+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/2279663802 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5413da20 con 0x7ffb540a4800 2020-11-20T12:22:18.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.144+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/2279663802 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb54148d00 con 0x7ffb540a4800 2020-11-20T12:22:18.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.146+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ffb5000f030 con 0x7ffb540a4800 2020-11-20T12:22:18.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.146+0000 7ffb4a7fc700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb34056d20 0x7ffb340591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.146+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ffb5006fe50 con 0x7ffb540a4800 2020-11-20T12:22:18.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.147+0000 7ffb59c44700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb34056d20 0x7ffb340591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.149+0000 7ffb5c6a9700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54148f80 0x7ffb54149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.149+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/2279663802 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5413dbb0 con 0x7ffb54148f80 2020-11-20T12:22:18.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.149+0000 7ffb5ac46700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54148f80 0x7ffb54149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.152+0000 7ffb59c44700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb34056d20 0x7ffb340591e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7ffb5414df10 tx=0x7ffb44015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.152+0000 7ffb5ac46700 1 -- 172.21.15.44:0/2279663802 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb54061650 con 0x7ffb54148f80 2020-11-20T12:22:18.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.153+0000 7ffb5ac46700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54148f80 0x7ffb54149360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ffb4c0012a0 tx=0x7ffb4c007e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.157+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00e560 con 0x7ffb54148f80 2020-11-20T12:22:18.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.157+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7ffb4c00e6c0 con 0x7ffb54148f80 2020-11-20T12:22:18.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.157+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54148f80 msgr2=0x7ffb54149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.157+0000 7ffb4a7fc700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54148f80 0x7ffb54149360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7ffb4c0012a0 tx=0x7ffb4c007e90).stop 2020-11-20T12:22:18.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.157+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54148f80 msgr2=0x7ffb54149360 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:18.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.157+0000 7ffb4a7fc700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54148f80 0x7ffb54149360 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0x7ffb4c0012a0 tx=0x7ffb4c007e90).stop 2020-11-20T12:22:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.179+0000 7ffb5c6a9700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54149850 0x7ffb54135c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.183 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.179+0000 7ffb5c6a9700 1 -- 172.21.15.44:0/2279663802 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540446d0 con 0x7ffb54149850 2020-11-20T12:22:18.186 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.183+0000 7ffb5ac46700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54149850 0x7ffb54135c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.186+0000 7ffb5ac46700 1 -- 172.21.15.44:0/2279663802 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffb540680f0 con 0x7ffb54149850 2020-11-20T12:22:18.192 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.186+0000 7ffb5ac46700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54149850 0x7ffb54135c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ffb4c015000 tx=0x7ffb4c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.197+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c01a380 con 0x7ffb54149850 2020-11-20T12:22:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.197+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4267 (secure 0 0 0) 0x7ffb4c01a4e0 con 0x7ffb54149850 2020-11-20T12:22:18.201 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.197+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54149850 msgr2=0x7ffb54135c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.197+0000 7ffb4a7fc700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54149850 0x7ffb54135c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ffb4c015000 tx=0x7ffb4c016000).stop 2020-11-20T12:22:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.197+0000 7ffb4a7fc700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54149850 msgr2=0x7ffb54135c20 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:18.202 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.197+0000 7ffb4a7fc700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ffb54149850 0x7ffb54135c20 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0x7ffb4c015000 tx=0x7ffb4c016000).stop 2020-11-20T12:22:18.205 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:18.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:18.205 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:18.206 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:22:18.206 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 182, 2020-11-20T12:22:18.206 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:18.206 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:18.206 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:22:18.207 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:18.207 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.207 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 26, 2020-11-20T12:22:18.207 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:18.208 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:18.208 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:18.208 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:18.208 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:18.208 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:18.209 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:18.210 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:18.210 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:18.210 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:18.210 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:18.210 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.211 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:18.211 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:18.212 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:18.212 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:18.212 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:18.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:18.213 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:18.213 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:18.213 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:18.213 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:18.213 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.214 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:18.215 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:18.215 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:18.215 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:18.215 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:18.216 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:18.216 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:18.216 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:18.216 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:18.216 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:18.217 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:18.217 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:18.217 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:18.217 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:18.217 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:18.218 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:18.218 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:18.218 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:18.218 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:18.218 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:18.219 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:18.219 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:18.219 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:18.219 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.219 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:18.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.220 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:18.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.220 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:18.220 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:18.221 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:18.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:18.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:18.222 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:18.222 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:18.223 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:18.223 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.223 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.224 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:18.224 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:18.224 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:18.224 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:18.224 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:18.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.225 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:18.225 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:18.225 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:18.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:18.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:18.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:18.226 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:18.227 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:18.227 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.228 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.228 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:18.228 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:18.229 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:18.229 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:18.229 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:18.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.230 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:18.230 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:18.230 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:18.230 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:18.231 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:18.234 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:18.235 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:18.236 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:18.237 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.238 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:18.239 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:18.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.202+0000 7ffb2ffff700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb34056d20 msgr2=0x7ffb340591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.240 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.202+0000 7ffb2ffff700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ffb34056d20 0x7ffb340591e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7ffb5414df10 tx=0x7ffb44015000).stop 2020-11-20T12:22:18.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.202+0000 7ffb2ffff700 1 -- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 msgr2=0x7ffb5414af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.202+0000 7ffb2ffff700 1 --2- 172.21.15.44:0/2279663802 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ffb540a4800 0x7ffb5414af50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7ffb500096e0 tx=0x7ffb50006d80).stop 2020-11-20T12:22:18.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.204+0000 7ffb5ac46700 1 -- 172.21.15.44:0/2279663802 reap_dead start 2020-11-20T12:22:18.241 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.204+0000 7ffb2ffff700 1 -- 172.21.15.44:0/2279663802 shutdown_connections 2020-11-20T12:22:18.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.204+0000 7ffb2ffff700 1 -- 172.21.15.44:0/2279663802 >> 172.21.15.44:0/2279663802 conn(0x7ffb54062690 msgr2=0x7ffb540aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.206+0000 7ffb2ffff700 1 -- 172.21.15.44:0/2279663802 shutdown_connections 2020-11-20T12:22:18.242 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.209+0000 7ffb2ffff700 1 -- 172.21.15.44:0/2279663802 wait complete. 2020-11-20T12:22:18.252 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:22:18.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.481+0000 7f22e51a1700 1 Processor -- start 2020-11-20T12:22:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.481+0000 7f22e51a1700 1 -- start start 2020-11-20T12:22:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.482+0000 7f22e51a1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f22e013f930 0x7f22e013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.482+0000 7f22e51a1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f22e01479e0 0x7f22e0149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.482+0000 7f22e51a1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00a57b0 con 0x7f22e00aa1e0 2020-11-20T12:22:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.482+0000 7f22e51a1700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00a4b80 con 0x7f22e01479e0 2020-11-20T12:22:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.482+0000 7f22e51a1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00a82f0 con 0x7f22e013f930 2020-11-20T12:22:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.483+0000 7f22ded9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f22e00aa1e0 0x7f22e00a5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.44:50292/0 (socket says 172.21.15.44:50292) 2020-11-20T12:22:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.483+0000 7f22ded9d700 1 -- 172.21.15.44:0/3438325497 learned_addr learned my addr 172.21.15.44:0/3438325497 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:22:18.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.484+0000 7f22de59c700 1 --2- 172.21.15.44:0/3438325497 >> v2:172.21.15.79:3300/0 conn(0x7f22e013f930 0x7f22e013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.485+0000 7f22df59e700 1 --2- 172.21.15.44:0/3438325497 >> v2:172.21.15.44:3300/0 conn(0x7f22e01479e0 0x7f22e0149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.486+0000 7f22ddd9b700 1 -- 172.21.15.44:0/3438325497 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695112917 0 0) 0x7f22e00a57b0 con 0x7f22e00aa1e0 2020-11-20T12:22:18.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.487+0000 7f22ddd9b700 1 -- 172.21.15.44:0/3438325497 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22cc003240 con 0x7f22e00aa1e0 2020-11-20T12:22:18.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.487+0000 7f22df59e700 1 -- 172.21.15.44:0/3438325497 >> v2:172.21.15.79:3300/0 conn(0x7f22e013f930 msgr2=0x7f22e013fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:18.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.487+0000 7f22df59e700 1 --2- 172.21.15.44:0/3438325497 >> v2:172.21.15.79:3300/0 conn(0x7f22e013f930 0x7f22e013fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.488+0000 7f22df59e700 1 -- 172.21.15.44:0/3438325497 >> v1:172.21.15.79:6789/0 conn(0x7f22e00aa1e0 legacy=0x7f22e00a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.488+0000 7f22df59e700 1 -- 172.21.15.44:0/3438325497 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00a80c0 con 0x7f22e01479e0 2020-11-20T12:22:18.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.488+0000 7f22df59e700 1 --2- 172.21.15.44:0/3438325497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e01479e0 0x7f22e0149dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f22e0064120 tx=0x7f22d4006df0).ready entity=mon.0 client_cookie=eba4eac601d2f315 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.493+0000 7f22ddd9b700 1 -- 172.21.15.44:0/3438325497 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4009730 con 0x7f22e01479e0 2020-11-20T12:22:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.493+0000 7f22ddd9b700 1 -- 172.21.15.44:0/3438325497 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4009890 con 0x7f22e01479e0 2020-11-20T12:22:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.493+0000 7f22ddd9b700 1 -- 172.21.15.44:0/3438325497 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4009c10 con 0x7f22e01479e0 2020-11-20T12:22:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.494+0000 7f22e51a1700 1 -- 172.21.15.44:0/3438325497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e01479e0 msgr2=0x7f22e0149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.494+0000 7f22e51a1700 1 --2- 172.21.15.44:0/3438325497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e01479e0 0x7f22e0149dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f22e0064120 tx=0x7f22d4006df0).stop 2020-11-20T12:22:18.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.494+0000 7f22e51a1700 1 -- 172.21.15.44:0/3438325497 shutdown_connections 2020-11-20T12:22:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.494+0000 7f22e51a1700 1 --2- 172.21.15.44:0/3438325497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e01479e0 0x7f22e0149dd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.494+0000 7f22e51a1700 1 --2- 172.21.15.44:0/3438325497 >> v2:172.21.15.79:3300/0 conn(0x7f22e013f930 0x7f22e013fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.494+0000 7f22e51a1700 1 -- 172.21.15.44:0/3438325497 >> 172.21.15.44:0/3438325497 conn(0x7f22e0062600 msgr2=0x7f22e00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.503 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.500+0000 7f22e51a1700 1 -- 172.21.15.44:0/3438325497 shutdown_connections 2020-11-20T12:22:18.513 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.502+0000 7f22e51a1700 1 -- 172.21.15.44:0/3438325497 wait complete. 2020-11-20T12:22:18.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.520+0000 7f22e51a1700 1 Processor -- start 2020-11-20T12:22:18.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.520+0000 7f22e51a1700 1 -- start start 2020-11-20T12:22:18.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22e51a1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e00aa1e0 0x7f22e0146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22e51a1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e013f930 0x7f22e0143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22e51a1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e01479e0 0x7f22e01446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22e51a1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e009f620 con 0x7f22e013f930 2020-11-20T12:22:18.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22e51a1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e009d510 con 0x7f22e01479e0 2020-11-20T12:22:18.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22e51a1700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e009d7b0 con 0x7f22e00aa1e0 2020-11-20T12:22:18.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22ded9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e00aa1e0 0x7f22e0146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22ded9d700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e00aa1e0 0x7f22e0146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55112/0 (socket says 172.21.15.44:55112) 2020-11-20T12:22:18.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.521+0000 7f22ded9d700 1 -- 172.21.15.44:0/1370100836 learned_addr learned my addr 172.21.15.44:0/1370100836 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:18.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22de59c700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e013f930 0x7f22e0143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22df59e700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e01479e0 0x7f22e01446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22df59e700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e00aa1e0 msgr2=0x7f22e0146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22df59e700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e00aa1e0 0x7f22e0146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.529 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22df59e700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e013f930 msgr2=0x7f22e0143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22df59e700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e013f930 0x7f22e0143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22df59e700 1 -- 172.21.15.44:0/1370100836 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e009dc50 con 0x7f22e01479e0 2020-11-20T12:22:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.522+0000 7f22de59c700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22e013f930 0x7f22e0143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.523+0000 7f22df59e700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e01479e0 0x7f22e01446b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f22d4012000 tx=0x7f22d4013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.530 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.523+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4006b20 con 0x7f22e01479e0 2020-11-20T12:22:18.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.523+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e009f850 con 0x7f22e01479e0 2020-11-20T12:22:18.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.523+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4040470 con 0x7f22e01479e0 2020-11-20T12:22:18.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.523+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4040810 con 0x7f22e01479e0 2020-11-20T12:22:18.531 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.523+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e0147240 con 0x7f22e01479e0 2020-11-20T12:22:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.524+0000 7f22e51a1700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e0069560 0x7f22e0144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.524+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e009d970 con 0x7f22e0069560 2020-11-20T12:22:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.525+0000 7f22ded9d700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e0069560 0x7f22e0144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.526+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f22d401a040 con 0x7f22e01479e0 2020-11-20T12:22:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.527+0000 7f22c6ffd700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f22bc056d50 0x7f22bc059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.527+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f22d4093ff0 con 0x7f22e01479e0 2020-11-20T12:22:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.527+0000 7f22ded9d700 1 -- 172.21.15.44:0/1370100836 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22e0061650 con 0x7f22e0069560 2020-11-20T12:22:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.528+0000 7f22ded9d700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e0069560 0x7f22e0144ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f22d8001fb0 tx=0x7f22d8000f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.528+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8002090 con 0x7f22e0069560 2020-11-20T12:22:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.529+0000 7f22de59c700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f22bc056d50 0x7f22bc059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.530+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f22d800fcc0 con 0x7f22e0069560 2020-11-20T12:22:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.530+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e0069560 msgr2=0x7f22e0144ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.530+0000 7f22c6ffd700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e0069560 0x7f22e0144ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f22d8001fb0 tx=0x7f22d8000f00).stop 2020-11-20T12:22:18.535 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.530+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e0069560 msgr2=0x7f22e0144ba0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:18.536 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.530+0000 7f22c6ffd700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e0069560 0x7f22e0144ba0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0x7f22d8001fb0 tx=0x7f22d8000f00).stop 2020-11-20T12:22:18.540 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.537+0000 7f22de59c700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f22bc056d50 0x7f22bc059210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f22e00a9bc0 tx=0x7f22d0008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.547+0000 7f22e51a1700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e015a230 0x7f22e015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.548+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00a45d0 con 0x7f22e015a230 2020-11-20T12:22:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.548+0000 7f22ded9d700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e015a230 0x7f22e015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.553 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22ded9d700 1 -- 172.21.15.44:0/1370100836 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22e00446d0 con 0x7f22e015a230 2020-11-20T12:22:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22ded9d700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e015a230 0x7f22e015a610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f22d8015000 tx=0x7f22d8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8006800 con 0x7f22e015a230 2020-11-20T12:22:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7f22d800fab0 con 0x7f22e015a230 2020-11-20T12:22:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e015a230 msgr2=0x7f22e015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22c6ffd700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e015a230 0x7f22e015a610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f22d8015000 tx=0x7f22d8016000).stop 2020-11-20T12:22:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22c6ffd700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e015a230 msgr2=0x7f22e015a610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.550+0000 7f22c6ffd700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f22e015a230 0x7f22e015a610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.556 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:18.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:18.556 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:18.556 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:22:18.557 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 182, 2020-11-20T12:22:18.557 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:18.557 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:18.557 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:22:18.558 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:18.558 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.558 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 27, 2020-11-20T12:22:18.558 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:18.558 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:18.559 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:18.559 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:18.559 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:18.559 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:18.559 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:18.560 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:18.560 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:18.560 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:18.560 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:18.560 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.561 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:18.561 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:18.561 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:18.561 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:18.561 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:18.562 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:18.562 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:18.562 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:18.562 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:18.563 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:18.563 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.563 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:18.564 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:18.564 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:18.564 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:18.564 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:18.564 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:18.565 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:18.565 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:18.565 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:18.565 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:18.565 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:18.566 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:18.566 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:18.566 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:18.566 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:18.566 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:18.567 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:18.567 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:18.567 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:18.567 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:18.567 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:18.568 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:18.568 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:18.568 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.568 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:18.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.569 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:18.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.569 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:18.569 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:18.569 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:18.570 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:18.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:18.570 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:18.570 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.571 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:18.571 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:18.571 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.572 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.572 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:18.572 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:18.573 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:18.573 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:18.573 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:18.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.574 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:18.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:18.574 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:18.574 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:18.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:18.575 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:18.575 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.576 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:18.576 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:18.576 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.576 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.577 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:18.577 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:18.577 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:18.578 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:18.578 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:18.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.578 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:18.579 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:18.579 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:18.579 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:18.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:18.580 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:18.580 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.580 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:18.581 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:18.581 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:18.581 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.581 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.582 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:18.582 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:18.582 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:18.582 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:18.582 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:18.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.583 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:18.583 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:18.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.584 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:18.584 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:18.584 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:18.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.585 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.585 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:22:18.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.585 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:18.585 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:18.586 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:18.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.586 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.586 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:22:18.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.587 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:18.587 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:18.587 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:18.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.587 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:18.588 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:18.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:18.588 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:18.588 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:18.588 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:22:18.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:18.589 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:18.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:18.589 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:18.589 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:18.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.553+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f22bc056d50 msgr2=0x7f22bc059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.553+0000 7f22e51a1700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f22bc056d50 0x7f22bc059210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f22e00a9bc0 tx=0x7f22d0008040).stop 2020-11-20T12:22:18.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.553+0000 7f22df59e700 1 -- 172.21.15.44:0/1370100836 reap_dead start 2020-11-20T12:22:18.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.553+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e01479e0 msgr2=0x7f22e01446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.590 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.553+0000 7f22e51a1700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e01479e0 0x7f22e01446b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f22d4012000 tx=0x7f22d4013000).stop 2020-11-20T12:22:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.554+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 shutdown_connections 2020-11-20T12:22:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.554+0000 7f22e51a1700 1 --2- 172.21.15.44:0/1370100836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22e01479e0 0x7f22e01446b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.554+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 >> 172.21.15.44:0/1370100836 conn(0x7f22e0062600 msgr2=0x7f22e005ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.560+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 shutdown_connections 2020-11-20T12:22:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.561+0000 7f22e51a1700 1 -- 172.21.15.44:0/1370100836 wait complete. 2020-11-20T12:22:18.593 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2020-11-20T12:22:18.593 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2020-11-20T12:22:18.594 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2020-11-20T12:22:18.594 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:22:18.594 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2020-11-20T12:22:18.594 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:22:18.607 INFO:tasks.ceph.mon.b:Stopped 2020-11-20T12:22:18.608 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2020-11-20T12:22:18.608 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:18.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.885+0000 7f8167342700 1 Processor -- start 2020-11-20T12:22:18.889 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.886+0000 7f8167342700 1 -- start start 2020-11-20T12:22:18.892 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.887+0000 7f8167342700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f81600a80c0 0x7f8160137c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.887+0000 7f8167342700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8160138110 0x7f8160145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.887+0000 7f8167342700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81600a8520 con 0x7f81600aa1e0 2020-11-20T12:22:18.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.887+0000 7f8167342700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81600a94f0 con 0x7f81600a80c0 2020-11-20T12:22:18.893 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.887+0000 7f8167342700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81600a88c0 con 0x7f8160138110 2020-11-20T12:22:18.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.887+0000 7f81658df700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f8160138110 msgr2=0x7f8160145560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:18.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.887+0000 7f81658df700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8160138110 0x7f8160145560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:18.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f81600a80c0 0x7f8160137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f81600a80c0 0x7f8160137c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43688/0 (socket says 172.21.15.44:43688) 2020-11-20T12:22:18.894 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 -- 172.21.15.44:0/3965673163 learned_addr learned my addr 172.21.15.44:0/3965673163 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 -- 172.21.15.44:0/3965673163 >> v2:172.21.15.79:3300/0 conn(0x7f8160138110 msgr2=0x7f8160145560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 --2- 172.21.15.44:0/3965673163 >> v2:172.21.15.79:3300/0 conn(0x7f8160138110 0x7f8160145560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 -- 172.21.15.44:0/3965673163 >> v1:172.21.15.79:6789/0 conn(0x7f81600aa1e0 legacy=0x7f81600a30c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2020-11-20T12:22:18.895 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 -- 172.21.15.44:0/3965673163 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81600aa5c0 con 0x7f81600a80c0 2020-11-20T12:22:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.888+0000 7f81648dd700 1 --2- 172.21.15.44:0/3965673163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600a80c0 0x7f8160137c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f81600a28f0 tx=0x7f8150006eb0).ready entity=mon.0 client_cookie=2da80afa30345cf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.896 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.891+0000 7f814ffff700 1 -- 172.21.15.44:0/3965673163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150009890 con 0x7f81600a80c0 2020-11-20T12:22:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.891+0000 7f814ffff700 1 -- 172.21.15.44:0/3965673163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81500099f0 con 0x7f81600a80c0 2020-11-20T12:22:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.891+0000 7f814ffff700 1 -- 172.21.15.44:0/3965673163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150009d70 con 0x7f81600a80c0 2020-11-20T12:22:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.892+0000 7f8167342700 1 -- 172.21.15.44:0/3965673163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600a80c0 msgr2=0x7f8160137c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.897 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.892+0000 7f8167342700 1 --2- 172.21.15.44:0/3965673163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600a80c0 0x7f8160137c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f81600a28f0 tx=0x7f8150006eb0).stop 2020-11-20T12:22:18.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.895+0000 7f8167342700 1 -- 172.21.15.44:0/3965673163 shutdown_connections 2020-11-20T12:22:18.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.895+0000 7f8167342700 1 --2- 172.21.15.44:0/3965673163 >> v2:172.21.15.79:3300/0 conn(0x7f8160138110 0x7f8160145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:18.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.895+0000 7f8167342700 1 --2- 172.21.15.44:0/3965673163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600a80c0 0x7f8160137c20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.895+0000 7f8167342700 1 -- 172.21.15.44:0/3965673163 >> 172.21.15.44:0/3965673163 conn(0x7f8160062600 msgr2=0x7f81600a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.901+0000 7f8167342700 1 -- 172.21.15.44:0/3965673163 shutdown_connections 2020-11-20T12:22:18.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.904+0000 7f8167342700 1 -- 172.21.15.44:0/3965673163 wait complete. 2020-11-20T12:22:18.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.906+0000 7f8167342700 1 Processor -- start 2020-11-20T12:22:18.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.906+0000 7f8167342700 1 -- start start 2020-11-20T12:22:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f8167342700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81600a80c0 0x7f816013b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f8167342700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600aa1e0 0x7f81600a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f8167342700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8160138110 0x7f8160131c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f8167342700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8160147250 con 0x7f81600aa1e0 2020-11-20T12:22:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f8167342700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81600a6680 con 0x7f8160138110 2020-11-20T12:22:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f8167342700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81601397f0 con 0x7f81600a80c0 2020-11-20T12:22:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f81658df700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8160138110 msgr2=0x7f8160131c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f81658df700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8160138110 0x7f8160131c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f81648dd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600aa1e0 0x7f81600a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f81648dd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600aa1e0 0x7f81600a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43694/0 (socket says 172.21.15.44:43694) 2020-11-20T12:22:18.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f81648dd700 1 -- 172.21.15.44:0/4255341196 learned_addr learned my addr 172.21.15.44:0/4255341196 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:18.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.908+0000 7f81650de700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81600a80c0 0x7f816013b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f81648dd700 1 -- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81600a80c0 msgr2=0x7f816013b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f81648dd700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81600a80c0 0x7f816013b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f81648dd700 1 -- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8160138110 msgr2=0x7f8160131c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:18.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f81648dd700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8160138110 0x7f8160131c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:18.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f81648dd700 1 -- 172.21.15.44:0/4255341196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160139a90 con 0x7f81600aa1e0 2020-11-20T12:22:18.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f81648dd700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600aa1e0 0x7f81600a6ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f815000b1b0 tx=0x7f8150026000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f814d7fa700 1 -- 172.21.15.44:0/4255341196 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815000bec0 con 0x7f81600aa1e0 2020-11-20T12:22:18.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f814d7fa700 1 -- 172.21.15.44:0/4255341196 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815002e5d0 con 0x7f81600aa1e0 2020-11-20T12:22:18.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816013b9f0 con 0x7f81600aa1e0 2020-11-20T12:22:18.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f814d7fa700 1 -- 172.21.15.44:0/4255341196 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815002e970 con 0x7f81600aa1e0 2020-11-20T12:22:18.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.909+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81600a5ac0 con 0x7f81600aa1e0 2020-11-20T12:22:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.911+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81600a6810 con 0x7f81600aa1e0 2020-11-20T12:22:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.912+0000 7f814d7fa700 1 -- 172.21.15.44:0/4255341196 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8150036030 con 0x7f81600aa1e0 2020-11-20T12:22:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.912+0000 7f814d7fa700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8138056cd0 0x7f8138059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.912+0000 7f814d7fa700 1 -- 172.21.15.44:0/4255341196 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8150082a40 con 0x7f81600aa1e0 2020-11-20T12:22:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.913+0000 7f81650de700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8138056cd0 0x7f8138059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.915+0000 7f81650de700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8138056cd0 0x7f8138059190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8158006fd0 tx=0x7f8158008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.922 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:18.920+0000 7f814d7fa700 1 -- 172.21.15.44:0/4255341196 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f81600a6810 con 0x7f81600aa1e0 2020-11-20T12:22:19.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.415+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f81600a8520 con 0x7f81600aa1e0 2020-11-20T12:22:19.419 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":182,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":28,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:19.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.415+0000 7f814d7fa700 1 -- 172.21.15.44:0/4255341196 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f81600a8520 con 0x7f81600aa1e0 2020-11-20T12:22:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8138056cd0 msgr2=0x7f8138059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8138056cd0 0x7f8138059190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8158006fd0 tx=0x7f8158008040).stop 2020-11-20T12:22:19.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600aa1e0 msgr2=0x7f81600a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600aa1e0 0x7f81600a6ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f815000b1b0 tx=0x7f8150026000).stop 2020-11-20T12:22:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 shutdown_connections 2020-11-20T12:22:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f8138056cd0 0x7f8138059190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.422 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8160138110 0x7f8160131c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:19.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f81600a80c0 0x7f816013b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 --2- 172.21.15.44:0/4255341196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81600aa1e0 0x7f81600a6ab0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 >> 172.21.15.44:0/4255341196 conn(0x7f8160062600 msgr2=0x7f816005ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:19.423 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.418+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 shutdown_connections 2020-11-20T12:22:19.424 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:19.419+0000 7f8167342700 1 -- 172.21.15.44:0/4255341196 wait complete. 2020-11-20T12:22:22.448 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:22.783 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.780+0000 7f8517fff700 1 Processor -- start 2020-11-20T12:22:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.780+0000 7f8517fff700 1 -- start start 2020-11-20T12:22:22.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f8517fff700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f85180a79e0 0x7f85181440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.788 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f8517fff700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85180a7820 con 0x7f85180a6380 2020-11-20T12:22:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f8517fff700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85180a9df0 con 0x7f85180a6760 2020-11-20T12:22:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f8517fff700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8518136220 con 0x7f85180a79e0 2020-11-20T12:22:22.789 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f850ffff700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7f85180a6760 legacy=0x7f851813bd00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:22:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f85177fe700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f85180a79e0 0x7f85181440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f8516ffd700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f85180a6380 0x7f8518135b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47458/0 (socket says 172.21.15.44:47458) 2020-11-20T12:22:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f8516ffd700 1 -- 172.21.15.44:0/2453202892 learned_addr learned my addr 172.21.15.44:0/2453202892 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:22:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f85167fc700 1 -- 172.21.15.44:0/2453202892 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842100360 0 0) 0x7f85080010d0 con 0x7f85180a6380 2020-11-20T12:22:22.790 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f85167fc700 1 -- 172.21.15.44:0/2453202892 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f8003240 con 0x7f85180a6380 2020-11-20T12:22:22.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f85177fe700 1 -- 172.21.15.44:0/2453202892 >> v1:172.21.15.79:6789/0 conn(0x7f85180a6760 legacy=0x7f851813bd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:22.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f85177fe700 1 -- 172.21.15.44:0/2453202892 >> v1:172.21.15.44:6789/0 conn(0x7f85180a6380 legacy=0x7f8518135b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.781+0000 7f85177fe700 1 -- 172.21.15.44:0/2453202892 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180aa090 con 0x7f85180a79e0 2020-11-20T12:22:22.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f85177fe700 1 --2- 172.21.15.44:0/2453202892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a79e0 0x7f85181440b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8510009c90 tx=0x7f85100091d0).ready entity=mon.0 client_cookie=bd15e465dc3706a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.791 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f85167fc700 1 -- 172.21.15.44:0/2453202892 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000ba30 con 0x7f85180a79e0 2020-11-20T12:22:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f85167fc700 1 -- 172.21.15.44:0/2453202892 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851000bb90 con 0x7f85180a79e0 2020-11-20T12:22:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f85167fc700 1 -- 172.21.15.44:0/2453202892 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510011400 con 0x7f85180a79e0 2020-11-20T12:22:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f8517fff700 1 -- 172.21.15.44:0/2453202892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a79e0 msgr2=0x7f85181440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f8517fff700 1 --2- 172.21.15.44:0/2453202892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a79e0 0x7f85181440b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8510009c90 tx=0x7f85100091d0).stop 2020-11-20T12:22:22.792 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f8517fff700 1 -- 172.21.15.44:0/2453202892 shutdown_connections 2020-11-20T12:22:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f8517fff700 1 --2- 172.21.15.44:0/2453202892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a79e0 0x7f85181440b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f8517fff700 1 -- 172.21.15.44:0/2453202892 >> 172.21.15.44:0/2453202892 conn(0x7f8518064430 msgr2=0x7f85180619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f8517fff700 1 -- 172.21.15.44:0/2453202892 shutdown_connections 2020-11-20T12:22:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.782+0000 7f8517fff700 1 -- 172.21.15.44:0/2453202892 wait complete. 2020-11-20T12:22:22.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.783+0000 7f8517fff700 1 Processor -- start 2020-11-20T12:22:22.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.783+0000 7f8517fff700 1 -- start start 2020-11-20T12:22:22.794 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8517fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a6380 0x7f8518138510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.795 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8517fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85180a6760 0x7f85180abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8517fff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f85180a79e0 0x7f8518133520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8517fff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180ad150 con 0x7f85180a6380 2020-11-20T12:22:22.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8517fff700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851809e740 con 0x7f85180a6760 2020-11-20T12:22:22.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8517fff700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180ad940 con 0x7f85180a79e0 2020-11-20T12:22:22.796 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a6380 0x7f8518138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f850ffff700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85180a6760 msgr2=0x7f85180abf10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:22.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f850ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85180a6760 0x7f85180abf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:22.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a6380 0x7f8518138510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43714/0 (socket says 172.21.15.44:43714) 2020-11-20T12:22:22.797 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 -- 172.21.15.44:0/2760305621 learned_addr learned my addr 172.21.15.44:0/2760305621 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f85177fe700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f85180a79e0 0x7f8518133520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 -- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f85180a79e0 msgr2=0x7f8518133520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f85180a79e0 0x7f8518133520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 -- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85180a6760 msgr2=0x7f85180abf10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:22.798 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85180a6760 0x7f85180abf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 -- 172.21.15.44:0/2760305621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851809e930 con 0x7f85180a6380 2020-11-20T12:22:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.784+0000 7f8516ffd700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a6380 0x7f8518138510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f850800c6f0 tx=0x7f8508007940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.785+0000 7f850effd700 1 -- 172.21.15.44:0/2760305621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508007320 con 0x7f85180a6380 2020-11-20T12:22:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.785+0000 7f850effd700 1 -- 172.21.15.44:0/2760305621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508007480 con 0x7f85180a6380 2020-11-20T12:22:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.785+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180ac400 con 0x7f85180a6380 2020-11-20T12:22:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.785+0000 7f850effd700 1 -- 172.21.15.44:0/2760305621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508005800 con 0x7f85180a6380 2020-11-20T12:22:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.785+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518139110 con 0x7f85180a6380 2020-11-20T12:22:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.786+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518057d80 con 0x7f85180a6380 2020-11-20T12:22:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.786+0000 7f850effd700 1 -- 172.21.15.44:0/2760305621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8508010030 con 0x7f85180a6380 2020-11-20T12:22:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.787+0000 7f850effd700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.787+0000 7f850effd700 1 -- 172.21.15.44:0/2760305621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f850806fc70 con 0x7f85180a6380 2020-11-20T12:22:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.787+0000 7f850ffff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.787+0000 7f850ffff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f851809e240 tx=0x7f8500006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:22.791+0000 7f850effd700 1 -- 172.21.15.44:0/2760305621 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8508044ff0 con 0x7f85180a6380 2020-11-20T12:22:23.015 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 2% (270/10837) Checking out files: 3% (326/10837) Checking out files: 4% (434/10837) Checking out files: 5% (542/10837) Checking out files: 5% (571/10837) Checking out files: 6% (651/10837) Checking out files: 7% (759/10837) Checking out files: 7% (788/10837) Checking out files: 8% (867/10837) Checking out files: 8% (932/10837) Checking out files: 9% (976/10837) Checking out files: 9% (1052/10837) Checking out files: 9% (1054/10837) Checking out files: 10% (1084/10837) Checking out files: 10% (1110/10837) Checking out files: 11% (1193/10837) Checking out files: 12% (1301/10837) Checking out files: 12% (1305/10837) Checking out files: 13% (1409/10837) Checking out files: 13% (1490/10837) Checking out files: 14% (1518/10837) Checking out files: 15% (1626/10837) Checking out files: 15% (1651/10837) Checking out files: 16% (1734/10837) Checking out files: 16% (1799/10837) Checking out files: 17% (1843/10837) Checking out files: 18% (1951/10837) Checking out files: 18% (1963/10837) Checking out files: 19% (2060/10837) Checking out files: 19% (2119/10837) Checking out files: 20% (2168/10837) Checking out files: 21% (2276/10837) Checking out files: 21% (2304/10837) Checking out files: 22% (2385/10837) Checking out files: 23% (2493/10837) Checking out files: 23% (2520/10837) Checking out files: 24% (2601/10837) Checking out files: 24% (2708/10837) Checking out files: 25% (2710/10837) Checking out files: 26% (2818/10837) Checking out files: 26% (2873/10837) Checking out files: 27% (2926/10837) Checking out files: 28% (3035/10837) Checking out files: 28% (3039/10837) Checking out files: 29% (3143/10837) Checking out files: 29% (3204/10837) Checking out files: 30% (3252/10837) Checking out files: 31% (3360/10837) Checking out files: 31% (3369/10837) Checking out files: 32% (3468/10837) Checking out files: 33% (3577/10837) Checking out files: 33% (3613/10837) Checking out files: 34% (3685/10837) Checking out files: 35% (3793/10837) Checking out files: 35% (3856/10837) Checking out files: 36% (3902/10837) Checking out files: 37% (4010/10837) Checking out files: 37% (4041/10837) Checking out files: 38% (4119/10837) Checking out files: 38% (4150/10837) Checking out files: 39% (4227/10837) Checking out files: 39% (4267/10837) Checking out files: 40% (4335/10837) Checking out files: 41% (4444/10837) Checking out files: 41% (4498/10837) Checking out files: 42% (4552/10837) Checking out files: 43% (4660/10837) Checking out files: 44% (4769/10837) Checking out files: 44% (4787/10837) Checking out files: 45% (4877/10837) Checking out files: 46% (4986/10837) Checking out files: 46% (4993/10837) Checking out files: 47% (5094/10837) Checking out files: 47% (5186/10837) Checking out files: 48% (5202/10837) Checking out files: 49% (5311/10837) Checking out files: 49% (5341/10837) Checking out files: 50% (5419/10837) Checking out files: 51% (5527/10837) Checking out files: 52% (5636/10837) Checking out files: 53% (5744/10837) Checking out files: 53% (5781/10837) Checking out files: 54% (5852/10837) Checking out files: 55% (5961/10837) Checking out files: 55% (6035/10837) Checking out files: 56% (6069/10837) Checking out files: 57% (6178/10837) Checking out files: 57% (6280/10837) Checking out files: 58% (6286/10837) Checking out files: 59% (6394/10837) Checking out files: 59% (6435/10837) Checking out files: 60% (6503/10837) Checking out files: 60% (6564/10837) Checking out files: 61% (6611/10837) Checking out files: 61% (6667/10837) Checking out files: 62% (6719/10837) Checking out files: 62% (6789/10837) Checking out files: 63% (6828/10837) Checking out files: 63% (6923/10837) Checking out files: 64% (6936/10837) Checking out files: 65% (7045/10837) Checking out files: 65% (7069/10837) Checking out files: 66% (7153/10837) Checking out files: 66% (7244/10837) Checking out files: 67% (7261/10837) Checking out files: 68% (7370/10837) Checking out files: 68% (7427/10837) Checking out files: 69% (7478/10837) Checking out files: 70% (7586/10837) Checking out files: 70% (7646/10837) Checking out files: 71% (7695/10837) Checking out files: 71% (7795/10837) Checking out files: 72% (7803/10837) Checking out files: 73% (7912/10837) Checking out files: 73% (7987/10837) Checking out files: 74% (8020/10837) Checking out files: 75% (8128/10837) Checking out files: 76% (8237/10837) Checking out files: 76% (8257/10837) Checking out files: 77% (8345/10837) Checking out files: 77% (8386/10837) Checking out files: 78% (8453/10837) Checking out files: 79% (8562/10837) Checking out files: 79% (8607/10837) Checking out files: 80% (8670/10837) Checking out files: 80% (8747/10837) Checking out files: 81% (8778/10837) Checking out files: 82% (8887/10837) Checking out files: 82% (8919/10837) Checking out files: 83% (8995/10837) Checking out files: 83% (9073/10837) Checking out files: 84% (9104/10837) Checking out files: 85% (9212/10837) Checking out files: 85% (9290/10837) Checking out files: 86% (9320/10837) Checking out files: 87% (9429/10837) Checking out files: 87% (9525/10837) Checking out files: 88% (9537/10837) Checking out files: 89% (9645/10837) Checking out files: 90% (9754/10837) Checking out files: 90% (9758/10837) Checking out files: 91% (9862/10837) Checking out files: 91% (9918/10837) Checking out files: 92% (9971/10837) Checking out files: 92% (10056/10837) Checking out files: 93% (10079/10837) Checking out files: 93% (10177/10837) Checking out files: 94% (10187/10837) Checking out files: 95% (10296/10837) Checking out files: 95% (10373/10837) Checking out files: 96% (10404/10837) Checking out files: 97% (10512/10837) Checking out files: 97% (10583/10837) Checking out files: 98% (10621/10837) Checking out files: 98% (10716/10837) Checking out files: 99% (10729/10837) Checking out files: 100% (10837/10837) Checking out files: 100% (10837/10837), done. 2020-11-20T12:22:23.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.257+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f85180a7820 con 0x7f85180a6380 2020-11-20T12:22:23.260 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.257+0000 7f850effd700 1 -- 172.21.15.44:0/2760305621 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f85180a7820 con 0x7f85180a6380 2020-11-20T12:22:23.261 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":182,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":31,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:23.262 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 msgr2=0x7f84f4059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f851809e240 tx=0x7f8500006ae0).stop 2020-11-20T12:22:23.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a6380 msgr2=0x7f8518138510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a6380 0x7f8518138510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f850800c6f0 tx=0x7f8508007940).stop 2020-11-20T12:22:23.263 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 shutdown_connections 2020-11-20T12:22:23.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f84f4056cd0 0x7f84f4059190 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f85180a79e0 0x7f8518133520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85180a6380 0x7f8518138510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 --2- 172.21.15.44:0/2760305621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85180a6760 0x7f85180abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:23.264 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 >> 172.21.15.44:0/2760305621 conn(0x7f8518064430 msgr2=0x7f851805ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:23.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 shutdown_connections 2020-11-20T12:22:23.265 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:23.259+0000 7f8517fff700 1 -- 172.21.15.44:0/2760305621 wait complete. 2020-11-20T12:22:23.373 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 204c175eea msg: blobhash on entity_name_t causes strange badness... should look into that, but avoid for now 2020-11-20T12:22:23.377 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.1 2020-11-20T12:22:23.625 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:23.626 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.2 2020-11-20T12:22:23.626 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.2 2020-11-20T12:22:23.626 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.2 2020-11-20T12:22:23.626 INFO:tasks.workunit.client.0.smithi044.stdout:v0.2 2020-11-20T12:22:26.276 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.476+0000 7f64d7449700 1 Processor -- start 2020-11-20T12:22:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.476+0000 7f64d7449700 1 -- start start 2020-11-20T12:22:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d7449700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f64d00aa020 0x7f64d00a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d7449700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d00aa400 0x7f64d00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d7449700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f64d013ac60 0x7f64d00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d7449700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00a9770 con 0x7f64d00aa020 2020-11-20T12:22:26.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d7449700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00a6da0 con 0x7f64d013ac60 2020-11-20T12:22:26.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d7449700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00a7040 con 0x7f64d00aa400 2020-11-20T12:22:26.482 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d59e6700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7f64d013ac60 msgr2=0x7f64d00a3700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d59e6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f64d013ac60 0x7f64d00a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d51e5700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f64d00aa020 0x7f64d00a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d51e5700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f64d00aa020 0x7f64d00a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43728/0 (socket says 172.21.15.44:43728) 2020-11-20T12:22:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d51e5700 1 -- 172.21.15.44:0/53645394 learned_addr learned my addr 172.21.15.44:0/53645394 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:26.483 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.477+0000 7f64d49e4700 1 --2- 172.21.15.44:0/53645394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d00aa400 0x7f64d00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.486 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64d51e5700 1 -- 172.21.15.44:0/53645394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d00aa400 msgr2=0x7f64d00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:26.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64d51e5700 1 --2- 172.21.15.44:0/53645394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d00aa400 0x7f64d00a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64d51e5700 1 -- 172.21.15.44:0/53645394 >> v2:172.21.15.79:3300/0 conn(0x7f64d013ac60 msgr2=0x7f64d00a3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:26.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64d51e5700 1 --2- 172.21.15.44:0/53645394 >> v2:172.21.15.79:3300/0 conn(0x7f64d013ac60 0x7f64d00a3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:26.487 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64d51e5700 1 -- 172.21.15.44:0/53645394 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00a6a90 con 0x7f64d00aa020 2020-11-20T12:22:26.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64d51e5700 1 --2- 172.21.15.44:0/53645394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00a9280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f64d0063320 tx=0x7f64c8009290).ready entity=mon.0 client_cookie=f0c8f1beedce6755 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:26.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64c7fff700 1 -- 172.21.15.44:0/53645394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800ddd0 con 0x7f64d00aa020 2020-11-20T12:22:26.488 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64c7fff700 1 -- 172.21.15.44:0/53645394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8010470 con 0x7f64d00aa020 2020-11-20T12:22:26.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.478+0000 7f64c7fff700 1 -- 172.21.15.44:0/53645394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80107f0 con 0x7f64d00aa020 2020-11-20T12:22:26.489 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.479+0000 7f64d7449700 1 -- 172.21.15.44:0/53645394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 msgr2=0x7f64d00a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.479+0000 7f64d7449700 1 --2- 172.21.15.44:0/53645394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00a9280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f64d0063320 tx=0x7f64c8009290).stop 2020-11-20T12:22:26.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.480+0000 7f64d7449700 1 -- 172.21.15.44:0/53645394 shutdown_connections 2020-11-20T12:22:26.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.480+0000 7f64d7449700 1 --2- 172.21.15.44:0/53645394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00a9280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.480+0000 7f64d7449700 1 --2- 172.21.15.44:0/53645394 >> v2:172.21.15.79:3300/0 conn(0x7f64d013ac60 0x7f64d00a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:26.490 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.480+0000 7f64d7449700 1 --2- 172.21.15.44:0/53645394 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d00aa400 0x7f64d00a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.480+0000 7f64d7449700 1 -- 172.21.15.44:0/53645394 >> 172.21.15.44:0/53645394 conn(0x7f64d0062690 msgr2=0x7f64d00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:26.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.480+0000 7f64d7449700 1 -- 172.21.15.44:0/53645394 shutdown_connections 2020-11-20T12:22:26.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.480+0000 7f64d7449700 1 -- 172.21.15.44:0/53645394 wait complete. 2020-11-20T12:22:26.491 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.481+0000 7f64d7449700 1 Processor -- start 2020-11-20T12:22:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.481+0000 7f64d7449700 1 -- start start 2020-11-20T12:22:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d7449700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d7449700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64d00aa400 0x7f64d01326f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.492 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d7449700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d013ac60 0x7f64d00676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d7449700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d012d1d0 con 0x7f64d00aa020 2020-11-20T12:22:26.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d7449700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0132f90 con 0x7f64d00aa400 2020-11-20T12:22:26.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d7449700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0133230 con 0x7f64d013ac60 2020-11-20T12:22:26.493 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d49e4700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64d00aa400 msgr2=0x7f64d01326f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d51e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d49e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64d00aa400 0x7f64d01326f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d51e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00ac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43730/0 (socket says 172.21.15.44:43730) 2020-11-20T12:22:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.482+0000 7f64d51e5700 1 -- 172.21.15.44:0/3520101500 learned_addr learned my addr 172.21.15.44:0/3520101500 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d59e6700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d013ac60 0x7f64d00676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d51e5700 1 -- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d013ac60 msgr2=0x7f64d00676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d51e5700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d013ac60 0x7f64d00676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d51e5700 1 -- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64d00aa400 msgr2=0x7f64d01326f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:26.495 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d51e5700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64d00aa400 0x7f64d01326f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d51e5700 1 -- 172.21.15.44:0/3520101500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d01334d0 con 0x7f64d00aa020 2020-11-20T12:22:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d51e5700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00ac5c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f64c80090f0 tx=0x7f64c8018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64c57fa700 1 -- 172.21.15.44:0/3520101500 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800c070 con 0x7f64d00aa020 2020-11-20T12:22:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64c57fa700 1 -- 172.21.15.44:0/3520101500 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8010e00 con 0x7f64d00aa020 2020-11-20T12:22:26.496 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64c57fa700 1 -- 172.21.15.44:0/3520101500 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8019ac0 con 0x7f64d00aa020 2020-11-20T12:22:26.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00acab0 con 0x7f64d00aa020 2020-11-20T12:22:26.497 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.483+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d00ad290 con 0x7f64d00aa020 2020-11-20T12:22:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.485+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d00ad450 con 0x7f64d00aa020 2020-11-20T12:22:26.498 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.485+0000 7f64c57fa700 1 -- 172.21.15.44:0/3520101500 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f64c802b3f0 con 0x7f64d00aa020 2020-11-20T12:22:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.485+0000 7f64c57fa700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f64a8056d00 0x7f64a80591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.485+0000 7f64c57fa700 1 -- 172.21.15.44:0/3520101500 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f64c8071980 con 0x7f64d00aa020 2020-11-20T12:22:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.485+0000 7f64d49e4700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f64a8056d00 0x7f64a80591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.499 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.486+0000 7f64d49e4700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f64a8056d00 0x7f64a80591c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f64d00adde0 tx=0x7f64cc006df0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:26.500 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.489+0000 7f64c57fa700 1 -- 172.21.15.44:0/3520101500 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f64c8046bf0 con 0x7f64d00aa020 2020-11-20T12:22:26.766 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 54% (120/222) Checking out files: 55% (123/222) Checking out files: 56% (125/222) Checking out files: 57% (127/222) Checking out files: 58% (129/222) Checking out files: 59% (131/222) Checking out files: 60% (134/222) Checking out files: 61% (136/222) Checking out files: 62% (138/222) Checking out files: 63% (140/222) Checking out files: 64% (143/222) Checking out files: 65% (145/222) Checking out files: 66% (147/222) Checking out files: 67% (149/222) Checking out files: 68% (151/222) Checking out files: 69% (154/222) Checking out files: 70% (156/222) Checking out files: 71% (158/222) Checking out files: 72% (160/222) Checking out files: 73% (163/222) Checking out files: 74% (165/222) Checking out files: 75% (167/222) Checking out files: 76% (169/222) Checking out files: 77% (171/222) Checking out files: 78% (174/222) Checking out files: 79% (176/222) Checking out files: 80% (178/222) Checking out files: 81% (180/222) Checking out files: 82% (183/222) Checking out files: 83% (185/222) Checking out files: 84% (187/222) Checking out files: 85% (189/222) Checking out files: 86% (191/222) Checking out files: 87% (194/222) Checking out files: 88% (196/222) Checking out files: 89% (198/222) Checking out files: 90% (200/222) Checking out files: 91% (203/222) Checking out files: 92% (205/222) Checking out files: 93% (207/222) Checking out files: 94% (209/222) Checking out files: 95% (211/222) Checking out files: 96% (214/222) Checking out files: 97% (216/222) Checking out files: 98% (218/222) Checking out files: 99% (220/222) Checking out files: 100% (222/222) Checking out files: 100% (222/222), done. 2020-11-20T12:22:26.825 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 2b5f76f48e readme 2020-11-20T12:22:27.162 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.2 2020-11-20T12:22:27.162 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:27.162 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.3 2020-11-20T12:22:27.163 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.3 2020-11-20T12:22:27.163 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.3 2020-11-20T12:22:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.849+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f64d0133660 con 0x7f64d00aa020 2020-11-20T12:22:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.849+0000 7f64c57fa700 1 -- 172.21.15.44:0/3520101500 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1471 (secure 0 0 0) 0x7f64c8026070 con 0x7f64d00aa020 2020-11-20T12:22:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f64a8056d00 msgr2=0x7f64a80591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f64a8056d00 0x7f64a80591c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f64d00adde0 tx=0x7f64cc006df0).stop 2020-11-20T12:22:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 msgr2=0x7f64d00ac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00ac5c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f64c80090f0 tx=0x7f64c8018000).stop 2020-11-20T12:22:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 shutdown_connections 2020-11-20T12:22:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f64a8056d00 0x7f64a80591c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f64d013ac60 0x7f64d00676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64d00aa020 0x7f64d00ac5c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 --2- 172.21.15.44:0/3520101500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f64d00aa400 0x7f64d01326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 >> 172.21.15.44:0/3520101500 conn(0x7f64d0062690 msgr2=0x7f64d00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 shutdown_connections 2020-11-20T12:22:27.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:26.851+0000 7f64d7449700 1 -- 172.21.15.44:0/3520101500 wait complete. 2020-11-20T12:22:27.166 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":182,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":35,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:27.167 INFO:tasks.workunit.client.0.smithi044.stdout:v0.3 2020-11-20T12:22:29.937 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 37% (110/295) Checking out files: 38% (113/295) Checking out files: 39% (116/295) Checking out files: 40% (118/295) Checking out files: 41% (121/295) Checking out files: 42% (124/295) Checking out files: 43% (127/295) Checking out files: 44% (130/295) Checking out files: 45% (133/295) Checking out files: 46% (136/295) Checking out files: 47% (139/295) Checking out files: 48% (142/295) Checking out files: 49% (145/295) Checking out files: 50% (148/295) Checking out files: 51% (151/295) Checking out files: 52% (154/295) Checking out files: 53% (157/295) Checking out files: 54% (160/295) Checking out files: 55% (163/295) Checking out files: 56% (166/295) Checking out files: 57% (169/295) Checking out files: 58% (172/295) Checking out files: 59% (175/295) Checking out files: 60% (177/295) Checking out files: 61% (180/295) Checking out files: 62% (183/295) Checking out files: 63% (186/295) Checking out files: 64% (189/295) Checking out files: 65% (192/295) Checking out files: 66% (195/295) Checking out files: 67% (198/295) Checking out files: 68% (201/295) Checking out files: 69% (204/295) Checking out files: 70% (207/295) Checking out files: 71% (210/295) Checking out files: 72% (213/295) Checking out files: 73% (216/295) Checking out files: 74% (219/295) Checking out files: 75% (222/295) Checking out files: 76% (225/295) Checking out files: 77% (228/295) Checking out files: 78% (231/295) Checking out files: 79% (234/295) Checking out files: 80% (236/295) Checking out files: 81% (239/295) Checking out files: 82% (242/295) Checking out files: 82% (243/295) Checking out files: 83% (245/295) Checking out files: 84% (248/295) Checking out files: 85% (251/295) Checking out files: 86% (254/295) Checking out files: 87% (257/295) Checking out files: 88% (260/295) Checking out files: 89% (263/295) Checking out files: 90% (266/295) Checking out files: 91% (269/295) Checking out files: 92% (272/295) Checking out files: 93% (275/295) Checking out files: 94% (278/295) Checking out files: 95% (281/295) Checking out files: 96% (284/295) Checking out files: 97% (287/295) Checking out files: 98% (290/295) Checking out files: 99% (293/295) Checking out files: 100% (295/295) Checking out files: 100% (295/295), done. 2020-11-20T12:22:29.997 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 2e986fe9dd include ceph.spec in configure.ac 2020-11-20T12:22:30.001 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.3 2020-11-20T12:22:30.167 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:30.236 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:30.237 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.4 2020-11-20T12:22:30.237 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.4 2020-11-20T12:22:30.237 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.4 2020-11-20T12:22:30.238 INFO:tasks.workunit.client.0.smithi044.stdout:v0.4 2020-11-20T12:22:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.321+0000 7fc790107700 1 Processor -- start 2020-11-20T12:22:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.321+0000 7fc790107700 1 -- start start 2020-11-20T12:22:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc790107700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 0x7fc7880a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:30.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc790107700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:30.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc790107700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7880a6260 con 0x7fc7880a4c30 2020-11-20T12:22:30.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc790107700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7880a52e0 con 0x7fc78813b0a0 2020-11-20T12:22:30.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc790107700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7880a6490 con 0x7fc788060330 2020-11-20T12:22:30.326 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78dea3700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 msgr2=0x7fc7880a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:30.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78d6a2700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fc7880a4c30 legacy=0x7fc78813a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:22:30.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78e6a4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:30.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78dea3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 0x7fc7880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:30.327 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78e6a4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43742/0 (socket says 172.21.15.44:43742) 2020-11-20T12:22:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 learned_addr learned my addr 172.21.15.44:0/2399511036 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:30.328 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:22:30.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:22:30.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:22:30.329 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.322+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78d6a2700 1 -- 172.21.15.44:0/2399511036 >> v1:172.21.15.79:6789/0 conn(0x7fc7880a4c30 legacy=0x7fc78813a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:22:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:30.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:22:30.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78dea3700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 msgr2=0x7fc7880a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:30.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:22:30.527 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:22:30.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78dea3700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 0x7fc7880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:22:30.528 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.523+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:22:30.927 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78dea3700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 msgr2=0x7fc7880a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:30.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78dea3700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 0x7fc7880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:22:30.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:30.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78d6a2700 1 -- 172.21.15.44:0/2399511036 >> v1:172.21.15.79:6789/0 conn(0x7fc7880a4c30 legacy=0x7fc78813a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:22:30.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:22:30.928 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:22:30.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:22:30.929 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:30.924+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:22:31.728 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78dea3700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 msgr2=0x7fc7880a4740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:31.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78d6a2700 1 -- 172.21.15.44:0/2399511036 >> v1:172.21.15.79:6789/0 conn(0x7fc7880a4c30 legacy=0x7fc78813a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:22:31.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78dea3700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 0x7fc7880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:22:31.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.729 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:22:31.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:22:31.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:22:31.730 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:31.725+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:22:33.331 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 msgr2=0x7fc7880a4740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.79:3300/0 conn(0x7fc788060330 0x7fc7880a4740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 -- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 msgr2=0x7fc788141810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 --2- 172.21.15.44:0/2399511036 >> v2:172.21.15.44:3300/0 conn(0x7fc78813b0a0 0x7fc788141810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.332 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 -- 172.21.15.44:0/2399511036 >> v1:172.21.15.79:6789/0 conn(0x7fc7880a4c30 legacy=0x7fc78813a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc768006fb0 0x7fc768009470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 -- 172.21.15.44:0/2399511036 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7880a6730 con 0x7fc768003360 2020-11-20T12:22:33.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc7777fe700 1 -- 172.21.15.44:0/2399511036 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880a52e0 con 0x7fc768006fb0 2020-11-20T12:22:33.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc78d6a2700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc768006fb0 0x7fc768009470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135681197 0 0) 0x7fc7880a6730 con 0x7fc768003360 2020-11-20T12:22:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.322+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc770003240 con 0x7fc768003360 2020-11-20T12:22:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4153443019 0 0) 0x7fc770003240 con 0x7fc768003360 2020-11-20T12:22:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc768006fb0 msgr2=0x7fc768009470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:33.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc768006fb0 0x7fc768009470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880a6260 con 0x7fc768003360 2020-11-20T12:22:33.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2320012591 0 0) 0x7fc784003d80 con 0x7fc768003360 2020-11-20T12:22:33.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 >> v1:172.21.15.44:6789/0 conn(0x7fc768003360 legacy=0x7fc7680057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 reap_dead start 2020-11-20T12:22:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc770003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004040 0x7fc770004520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004bc0 0x7fc77000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880a52e0 con 0x7fc770004bc0 2020-11-20T12:22:33.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc770005260 con 0x7fc770004040 2020-11-20T12:22:33.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77000bc20 con 0x7fc770003730 2020-11-20T12:22:33.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78d6a2700 1 -- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004040 msgr2=0x7fc770004520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:33.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78d6a2700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004040 0x7fc770004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:33.337 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004bc0 0x7fc77000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78dea3700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc770003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 msgr2=0x7fc770003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc770003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004040 msgr2=0x7fc770004520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004040 0x7fc770004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 -- 172.21.15.44:0/2399511036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc770003240 con 0x7fc770004bc0 2020-11-20T12:22:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.323+0000 7fc78e6a4700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004bc0 0x7fc77000b730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc7880a6140 tx=0x7fc780008040).ready entity=mon.0 client_cookie=da401b9464a866c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780015d10 con 0x7fc770004bc0 2020-11-20T12:22:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780015e70 con 0x7fc770004bc0 2020-11-20T12:22:33.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc78cea1700 1 -- 172.21.15.44:0/2399511036 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78001e600 con 0x7fc770004bc0 2020-11-20T12:22:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc790107700 1 -- 172.21.15.44:0/2399511036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004bc0 msgr2=0x7fc77000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc790107700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004bc0 0x7fc77000b730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc7880a6140 tx=0x7fc780008040).stop 2020-11-20T12:22:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc790107700 1 -- 172.21.15.44:0/2399511036 shutdown_connections 2020-11-20T12:22:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc790107700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004bc0 0x7fc77000b730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc790107700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004040 0x7fc770004520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc790107700 1 --2- 172.21.15.44:0/2399511036 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc770003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.324+0000 7fc790107700 1 -- 172.21.15.44:0/2399511036 >> 172.21.15.44:0/2399511036 conn(0x7fc788062690 msgr2=0x7fc7880a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.325+0000 7fc790107700 1 -- 172.21.15.44:0/2399511036 shutdown_connections 2020-11-20T12:22:33.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.325+0000 7fc790107700 1 -- 172.21.15.44:0/2399511036 wait complete. 2020-11-20T12:22:33.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.326+0000 7fc790107700 1 Processor -- start 2020-11-20T12:22:33.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.326+0000 7fc790107700 1 -- start start 2020-11-20T12:22:33.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.326+0000 7fc790107700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc7880ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc790107700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004040 0x7fc788066ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc790107700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004bc0 0x7fc78814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc790107700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc788066fd0 con 0x7fc770004040 2020-11-20T12:22:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc790107700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78814dac0 con 0x7fc770004bc0 2020-11-20T12:22:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc790107700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78814dd60 con 0x7fc770003730 2020-11-20T12:22:33.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78e6a4700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004bc0 msgr2=0x7fc78814a170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78e6a4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004bc0 0x7fc78814a170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004040 0x7fc788066ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004040 0x7fc788066ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43776/0 (socket says 172.21.15.44:43776) 2020-11-20T12:22:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 -- 172.21.15.44:0/334276091 learned_addr learned my addr 172.21.15.44:0/334276091 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:33.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78dea3700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc7880ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 -- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 msgr2=0x7fc7880ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc7880ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 -- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004bc0 msgr2=0x7fc78814a170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004bc0 0x7fc78814a170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 -- 172.21.15.44:0/334276091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880a2d30 con 0x7fc770004040 2020-11-20T12:22:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc78d6a2700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004040 0x7fc788066ae0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fc778003820 tx=0x7fc778009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.327+0000 7fc775ffb700 1 -- 172.21.15.44:0/334276091 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778010650 con 0x7fc770004040 2020-11-20T12:22:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.328+0000 7fc775ffb700 1 -- 172.21.15.44:0/334276091 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7780107b0 con 0x7fc770004040 2020-11-20T12:22:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.328+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78814d740 con 0x7fc770004040 2020-11-20T12:22:33.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.328+0000 7fc775ffb700 1 -- 172.21.15.44:0/334276091 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778010b50 con 0x7fc770004040 2020-11-20T12:22:33.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.328+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78814a660 con 0x7fc770004040 2020-11-20T12:22:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.329+0000 7fc775ffb700 1 -- 172.21.15.44:0/334276091 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc77801a030 con 0x7fc770004040 2020-11-20T12:22:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.329+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc788061650 con 0x7fc770004040 2020-11-20T12:22:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.329+0000 7fc775ffb700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc760056cd0 0x7fc760059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.330+0000 7fc775ffb700 1 -- 172.21.15.44:0/334276091 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc77806fbf0 con 0x7fc770004040 2020-11-20T12:22:33.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.330+0000 7fc78dea3700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc760056cd0 0x7fc760059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.330+0000 7fc78dea3700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc760056cd0 0x7fc760059190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc7840075a0 tx=0x7fc784011040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.334+0000 7fc775ffb700 1 -- 172.21.15.44:0/334276091 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc778044f70 con 0x7fc770004040 2020-11-20T12:22:33.583 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.580+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc78814d8d0 con 0x7fc770004040 2020-11-20T12:22:33.584 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.581+0000 7fc775ffb700 1 -- 172.21.15.44:0/334276091 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1464 (secure 0 0 0) 0x7fc778018030 con 0x7fc770004040 2020-11-20T12:22:33.585 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":186,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc760056cd0 msgr2=0x7fc760059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc760056cd0 0x7fc760059190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc7840075a0 tx=0x7fc784011040).stop 2020-11-20T12:22:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004040 msgr2=0x7fc788066ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004040 0x7fc788066ae0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fc778003820 tx=0x7fc778009120).stop 2020-11-20T12:22:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 shutdown_connections 2020-11-20T12:22:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc760056cd0 0x7fc760059190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc770004bc0 0x7fc78814a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc770003730 0x7fc7880ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 --2- 172.21.15.44:0/334276091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc770004040 0x7fc788066ae0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 >> 172.21.15.44:0/334276091 conn(0x7fc788062690 msgr2=0x7fc78813a510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 shutdown_connections 2020-11-20T12:22:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.582+0000 7fc790107700 1 -- 172.21.15.44:0/334276091 wait complete. 2020-11-20T12:22:33.597 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2020-11-20T12:22:33.598 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:22:33.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.730+0000 7fedbf424700 1 Processor -- start 2020-11-20T12:22:33.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.730+0000 7fedbf424700 1 -- start start 2020-11-20T12:22:33.734 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.730+0000 7fedbf424700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fedb80aa020 0x7fedb80a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.730+0000 7fedbf424700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb813ac60 0x7fedb80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.730+0000 7fedbf424700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb80a9770 con 0x7fedb80aa400 2020-11-20T12:22:33.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.730+0000 7fedbf424700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb80a56c0 con 0x7fedb80aa020 2020-11-20T12:22:33.735 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.730+0000 7fedbf424700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80a6da0 con 0x7fedb813ac60 2020-11-20T12:22:33.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd1c0700 1 -- >> v2:172.21.15.79:3300/0 conn(0x7fedb80aa020 msgr2=0x7fedb80a9280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:33.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbc9bf700 1 -- >> v1:172.21.15.79:6789/0 conn(0x7fedb80aa400 legacy=0x7fedb80a4fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.79:6789/0 2020-11-20T12:22:33.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd1c0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fedb80aa020 0x7fedb80a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:33.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb813ac60 0x7fedb80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.736 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb813ac60 0x7fedb80a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55222/0 (socket says 172.21.15.44:55222) 2020-11-20T12:22:33.737 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 -- 172.21.15.44:0/3529222973 learned_addr learned my addr 172.21.15.44:0/3529222973 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:33.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 -- 172.21.15.44:0/3529222973 >> v2:172.21.15.79:3300/0 conn(0x7fedb80aa020 msgr2=0x7fedb80a9280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 --2- 172.21.15.44:0/3529222973 >> v2:172.21.15.79:3300/0 conn(0x7fedb80aa020 0x7fedb80a9280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 -- 172.21.15.44:0/3529222973 >> v1:172.21.15.79:6789/0 conn(0x7fedb80aa400 legacy=0x7fedb80a4fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.740 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 -- 172.21.15.44:0/3529222973 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80a7040 con 0x7fedb813ac60 2020-11-20T12:22:33.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.731+0000 7fedbd9c1700 1 --2- 172.21.15.44:0/3529222973 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb813ac60 0x7fedb80a3700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7feda8009d50 tx=0x7feda8009290).ready entity=mon.2 client_cookie=5adda0f1ad6dbfc2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedb7fff700 1 -- 172.21.15.44:0/3529222973 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800bc40 con 0x7fedb813ac60 2020-11-20T12:22:33.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedb7fff700 1 -- 172.21.15.44:0/3529222973 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda800bda0 con 0x7fedb813ac60 2020-11-20T12:22:33.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedb7fff700 1 -- 172.21.15.44:0/3529222973 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda8011610 con 0x7fedb813ac60 2020-11-20T12:22:33.741 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 -- 172.21.15.44:0/3529222973 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb813ac60 msgr2=0x7fedb80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 --2- 172.21.15.44:0/3529222973 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb813ac60 0x7fedb80a3700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7feda8009d50 tx=0x7feda8009290).stop 2020-11-20T12:22:33.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 -- 172.21.15.44:0/3529222973 shutdown_connections 2020-11-20T12:22:33.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 --2- 172.21.15.44:0/3529222973 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb813ac60 0x7fedb80a3700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 --2- 172.21.15.44:0/3529222973 >> v2:172.21.15.79:3300/0 conn(0x7fedb80aa020 0x7fedb80a9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.742 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 -- 172.21.15.44:0/3529222973 >> 172.21.15.44:0/3529222973 conn(0x7fedb8062690 msgr2=0x7fedb80a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 -- 172.21.15.44:0/3529222973 shutdown_connections 2020-11-20T12:22:33.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.732+0000 7fedbf424700 1 -- 172.21.15.44:0/3529222973 wait complete. 2020-11-20T12:22:33.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.733+0000 7fedbf424700 1 Processor -- start 2020-11-20T12:22:33.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.733+0000 7fedbf424700 1 -- start start 2020-11-20T12:22:33.743 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbf424700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb80aa020 0x7fedb80ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbf424700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedb80aa400 0x7fedb8134630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbf424700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb813ac60 0x7fedb80ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbf424700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80ad0b0 con 0x7fedb813ac60 2020-11-20T12:22:33.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbf424700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80664c0 con 0x7fedb80aa400 2020-11-20T12:22:33.744 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbf424700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80ace10 con 0x7fedb80aa020 2020-11-20T12:22:33.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbc9bf700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedb80aa400 msgr2=0x7fedb8134630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:33.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbc9bf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedb80aa400 0x7fedb8134630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:33.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb813ac60 0x7fedb80ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb813ac60 0x7fedb80ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43792/0 (socket says 172.21.15.44:43792) 2020-11-20T12:22:33.745 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 -- 172.21.15.44:0/2743836033 learned_addr learned my addr 172.21.15.44:0/2743836033 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:33.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd1c0700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb80aa020 0x7fedb80ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb80aa020 msgr2=0x7fedb80ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fedb80aa020 0x7fedb80ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedb80aa400 msgr2=0x7fedb8134630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:33.746 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedb80aa400 0x7fedb8134630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.734+0000 7fedbd9c1700 1 -- 172.21.15.44:0/2743836033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80ad390 con 0x7fedb813ac60 2020-11-20T12:22:33.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.735+0000 7fedbd9c1700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb813ac60 0x7fedb80ad830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7feda80092d0 tx=0x7feda8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.735+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda801e070 con 0x7fedb813ac60 2020-11-20T12:22:33.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.735+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda8009e90 con 0x7fedb813ac60 2020-11-20T12:22:33.747 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.735+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb80add20 con 0x7fedb813ac60 2020-11-20T12:22:33.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.735+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800bc40 con 0x7fedb813ac60 2020-11-20T12:22:33.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.735+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb8067170 con 0x7fedb813ac60 2020-11-20T12:22:33.748 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedbf424700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb80ab130 0x7fedb81320d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80ad520 con 0x7fedb80ab130 2020-11-20T12:22:33.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedbd1c0700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb80ab130 0x7fedb81320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedbd1c0700 1 -- 172.21.15.44:0/2743836033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedb8061650 con 0x7fedb80ab130 2020-11-20T12:22:33.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7feda800f070 con 0x7fedb813ac60 2020-11-20T12:22:33.751 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedbd1c0700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb80ab130 0x7fedb81320d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fedb0003650 tx=0x7fedb0006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedb57fa700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fed90056cd0 0x7fed90059190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.736+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7feda8073340 con 0x7fedb813ac60 2020-11-20T12:22:33.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0006530 con 0x7fedb80ab130 2020-11-20T12:22:33.752 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fedb000b900 con 0x7fedb80ab130 2020-11-20T12:22:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedbc9bf700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fed90056cd0 0x7fed90059190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb80ab130 msgr2=0x7fedb81320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedb57fa700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb80ab130 0x7fedb81320d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fedb0003650 tx=0x7fedb0006dd0).stop 2020-11-20T12:22:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb80ab130 msgr2=0x7fedb81320d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:33.753 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedb57fa700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb80ab130 0x7fedb81320d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.737+0000 7fedbc9bf700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fed90056cd0 0x7fed90059190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fedb8147a30 tx=0x7fedac006ae0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.744+0000 7fedbf424700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb81325c0 0x7fedb8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.744+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80a9770 con 0x7fedb81325c0 2020-11-20T12:22:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.744+0000 7fedbd1c0700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb81325c0 0x7fedb8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.754 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedbd1c0700 1 -- 172.21.15.44:0/2743836033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fedb80446d0 con 0x7fedb81325c0 2020-11-20T12:22:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedbd1c0700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb81325c0 0x7fedb8065150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fedb000efc0 tx=0x7fedb0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb001a350 con 0x7fedb81325c0 2020-11-20T12:22:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4767 (secure 0 0 0) 0x7fedb000b6f0 con 0x7fedb81325c0 2020-11-20T12:22:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb81325c0 msgr2=0x7fedb8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.755 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedb57fa700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb81325c0 0x7fedb8065150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fedb000efc0 tx=0x7fedb0016000).stop 2020-11-20T12:22:33.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedb57fa700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb81325c0 msgr2=0x7fedb8065150 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:33.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedb57fa700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb81325c0 0x7fedb8065150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fed90056cd0 msgr2=0x7fed90059190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedbf424700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fed90056cd0 0x7fed90059190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fedb8147a30 tx=0x7fedac006ae0).stop 2020-11-20T12:22:33.756 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.745+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb813ac60 msgr2=0x7fedb80ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.746+0000 7fedbd9c1700 1 -- 172.21.15.44:0/2743836033 reap_dead start 2020-11-20T12:22:33.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.746+0000 7fedbf424700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb813ac60 0x7fedb80ad830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7feda80092d0 tx=0x7feda8016000).stop 2020-11-20T12:22:33.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.746+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 shutdown_connections 2020-11-20T12:22:33.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.746+0000 7fedbf424700 1 --2- 172.21.15.44:0/2743836033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedb813ac60 0x7fedb80ad830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.757 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.746+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 >> 172.21.15.44:0/2743836033 conn(0x7fedb8062690 msgr2=0x7fedb8139380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.746+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 shutdown_connections 2020-11-20T12:22:33.758 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.746+0000 7fedbf424700 1 -- 172.21.15.44:0/2743836033 wait complete. 2020-11-20T12:22:33.758 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:33.758 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:33.758 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:22:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 186, 2020-11-20T12:22:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:33.759 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 0, 2020-11-20T12:22:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:33.760 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:33.761 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:33.762 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:33.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:33.764 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:33.765 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:33.766 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:33.767 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:33.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:33.769 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.770 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.771 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:33.772 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:33.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:33.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:33.775 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:33.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.776 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:33.776 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:33.776 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:33.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:33.778 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:33.778 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:33.778 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.779 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:33.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.779 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:33.779 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:33.780 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:33.780 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:33.780 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:33.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:33.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:33.781 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:33.781 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:33.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.782 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:33.782 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:33.782 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:33.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:33.783 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:33.783 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:33.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.784 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:33.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.784 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:33.784 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:33.785 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:33.785 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:33.785 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:33.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.785 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:33.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.786 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:33.786 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:33.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.786 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:33.787 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:33.787 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:33.787 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.787 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:33.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:22:33.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.788 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:33.788 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:33.788 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:33.788 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.789 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:33.789 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:22:33.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.789 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:33.790 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:33.790 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:22:33.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.790 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:33.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:33.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.791 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:33.791 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:33.791 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:22:33.791 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.792 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:33.792 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:22:33.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:33.792 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:33.793 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:33.793 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:22:33.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:33.793 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:33.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:33.794 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:33.794 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:33.795 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:22:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 Processor -- start 2020-11-20T12:22:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 -- start start 2020-11-20T12:22:33.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8a200aad00 0x7f8a200a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.976 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8a200ab0e0 0x7f8a2013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a86d0 0x7f8a200abc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a201385f0 con 0x7f8a200aad00 2020-11-20T12:22:33.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2013e250 con 0x7f8a200ab0e0 2020-11-20T12:22:33.977 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.970+0000 7f8a24829700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2013e4f0 con 0x7f8a200a86d0 2020-11-20T12:22:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8a200aad00 0x7f8a200a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8a200aad00 0x7f8a200a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43800/0 (socket says 172.21.15.44:43800) 2020-11-20T12:22:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 -- 172.21.15.44:0/427462880 learned_addr learned my addr 172.21.15.44:0/427462880 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:33.978 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a17fff700 1 -- 172.21.15.44:0/427462880 >> v2:172.21.15.79:3300/0 conn(0x7f8a200ab0e0 msgr2=0x7f8a2013ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:33.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a17fff700 1 --2- 172.21.15.44:0/427462880 >> v2:172.21.15.79:3300/0 conn(0x7f8a200ab0e0 0x7f8a2013ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:33.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 -- 172.21.15.44:0/427462880 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a86d0 msgr2=0x7f8a200abc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 --2- 172.21.15.44:0/427462880 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a86d0 0x7f8a200abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.979 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 -- 172.21.15.44:0/427462880 >> v2:172.21.15.79:3300/0 conn(0x7f8a200ab0e0 msgr2=0x7f8a2013ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:33.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 --2- 172.21.15.44:0/427462880 >> v2:172.21.15.79:3300/0 conn(0x7f8a200ab0e0 0x7f8a2013ec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.980 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 -- 172.21.15.44:0/427462880 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a200ac120 con 0x7f8a200aad00 2020-11-20T12:22:33.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1e59c700 1 --2- 172.21.15.44:0/427462880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200aad00 0x7f8a200a94d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f8a200698b0 tx=0x7f8a10006eb0).ready entity=mon.0 client_cookie=ea15352d845b69e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.981 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1dd9b700 1 -- 172.21.15.44:0/427462880 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10009960 con 0x7f8a200aad00 2020-11-20T12:22:33.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1dd9b700 1 -- 172.21.15.44:0/427462880 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a10009ac0 con 0x7f8a200aad00 2020-11-20T12:22:33.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.971+0000 7f8a1dd9b700 1 -- 172.21.15.44:0/427462880 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10009e40 con 0x7f8a200aad00 2020-11-20T12:22:33.982 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.972+0000 7f8a24829700 1 -- 172.21.15.44:0/427462880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200aad00 msgr2=0x7f8a200a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.972+0000 7f8a24829700 1 --2- 172.21.15.44:0/427462880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200aad00 0x7f8a200a94d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f8a200698b0 tx=0x7f8a10006eb0).stop 2020-11-20T12:22:33.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.973+0000 7f8a24829700 1 -- 172.21.15.44:0/427462880 shutdown_connections 2020-11-20T12:22:33.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.973+0000 7f8a24829700 1 --2- 172.21.15.44:0/427462880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200aad00 0x7f8a200a94d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.973+0000 7f8a24829700 1 --2- 172.21.15.44:0/427462880 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a86d0 0x7f8a200abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.983 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.973+0000 7f8a24829700 1 --2- 172.21.15.44:0/427462880 >> v2:172.21.15.79:3300/0 conn(0x7f8a200ab0e0 0x7f8a2013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.973+0000 7f8a24829700 1 -- 172.21.15.44:0/427462880 >> 172.21.15.44:0/427462880 conn(0x7f8a20060170 msgr2=0x7f8a20060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.973+0000 7f8a24829700 1 -- 172.21.15.44:0/427462880 shutdown_connections 2020-11-20T12:22:33.984 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.973+0000 7f8a24829700 1 -- 172.21.15.44:0/427462880 wait complete. 2020-11-20T12:22:33.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.974+0000 7f8a24829700 1 Processor -- start 2020-11-20T12:22:33.987 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.974+0000 7f8a24829700 1 -- start start 2020-11-20T12:22:33.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a24829700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a200a86d0 0x7f8a2013ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a24829700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200aad00 0x7f8a200a7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.988 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a24829700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200ab0e0 0x7f8a20136810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a24829700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a201388c0 con 0x7f8a200ab0e0 2020-11-20T12:22:33.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a24829700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a20137420 con 0x7f8a200a86d0 2020-11-20T12:22:33.989 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a24829700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a20137650 con 0x7f8a200aad00 2020-11-20T12:22:33.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a1e59c700 1 -- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a200a86d0 msgr2=0x7f8a2013ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:3300/0 2020-11-20T12:22:33.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a1e59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a200a86d0 0x7f8a2013ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:33.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a1ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200ab0e0 0x7f8a20136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a1ed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200ab0e0 0x7f8a20136810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43806/0 (socket says 172.21.15.44:43806) 2020-11-20T12:22:33.990 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a1ed9d700 1 -- 172.21.15.44:0/3463383475 learned_addr learned my addr 172.21.15.44:0/3463383475 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:33.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.975+0000 7f8a17fff700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200aad00 0x7f8a200a7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a1ed9d700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200aad00 msgr2=0x7f8a200a7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a1ed9d700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200aad00 0x7f8a200a7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.991 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a1ed9d700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a200a86d0 msgr2=0x7f8a2013ab80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:33.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a1ed9d700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a200a86d0 0x7f8a2013ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:33.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a1ed9d700 1 -- 172.21.15.44:0/3463383475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2009eef0 con 0x7f8a200ab0e0 2020-11-20T12:22:33.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a17fff700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200aad00 0x7f8a200a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:33.992 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a1ed9d700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200ab0e0 0x7f8a20136810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8a1800bab0 tx=0x7f8a180091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800c7d0 con 0x7f8a200ab0e0 2020-11-20T12:22:33.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1800c930 con 0x7f8a200ab0e0 2020-11-20T12:22:33.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a200a74b0 con 0x7f8a200ab0e0 2020-11-20T12:22:33.993 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800caa0 con 0x7f8a200ab0e0 2020-11-20T12:22:33.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.976+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2013b780 con 0x7f8a200ab0e0 2020-11-20T12:22:33.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.977+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f8a1801c070 con 0x7f8a200ab0e0 2020-11-20T12:22:33.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.978+0000 7f8a167fc700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f89fc056d00 0x7f89fc0591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.994 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.978+0000 7f8a24829700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a6680 0x7f8a20136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.978+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2005c5f0 con 0x7f8a200a6680 2020-11-20T12:22:33.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.978+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f8a1806fde0 con 0x7f8a200ab0e0 2020-11-20T12:22:33.995 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.978+0000 7f8a1e59c700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f89fc056d00 0x7f89fc0591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.978+0000 7f8a17fff700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a6680 0x7f8a20136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a17fff700 1 -- 172.21.15.44:0/3463383475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a2013b070 con 0x7f8a200a6680 2020-11-20T12:22:33.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a1e59c700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f89fc056d00 0x7f89fc0591c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f8a10006ef0 tx=0x7f8a10015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.996 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a17fff700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a6680 0x7f8a20136d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f8a2013b2f0 tx=0x7f8a08006dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800ecc0 con 0x7f8a200a6680 2020-11-20T12:22:33.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f8a0800ee20 con 0x7f8a200a6680 2020-11-20T12:22:33.997 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a6680 msgr2=0x7f8a20136d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a167fc700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a6680 0x7f8a20136d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f8a2013b2f0 tx=0x7f8a08006dd0).stop 2020-11-20T12:22:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a6680 msgr2=0x7f8a20136d00 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.979+0000 7f8a167fc700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a6680 0x7f8a20136d00 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0x7f8a2013b2f0 tx=0x7f8a08006dd0).stop 2020-11-20T12:22:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.994+0000 7f8a24829700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a50a0 0x7f8a200a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.994+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200a4a30 con 0x7f8a200a50a0 2020-11-20T12:22:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.994+0000 7f8a17fff700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a50a0 0x7f8a200a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a17fff700 1 -- 172.21.15.44:0/3463383475 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8a200a7980 con 0x7f8a200a50a0 2020-11-20T12:22:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a17fff700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a50a0 0x7f8a200a54b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8a08012000 tx=0x7f8a08016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08012870 con 0x7f8a200a50a0 2020-11-20T12:22:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4595 (secure 0 0 0) 0x7f8a0800eab0 con 0x7f8a200a50a0 2020-11-20T12:22:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a50a0 msgr2=0x7f8a200a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a167fc700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a50a0 0x7f8a200a54b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8a08012000 tx=0x7f8a08016000).stop 2020-11-20T12:22:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a167fc700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a50a0 msgr2=0x7f8a200a54b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:22:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.995+0000 7f8a167fc700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f8a200a50a0 0x7f8a200a54b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f89fc056d00 msgr2=0x7f89fc0591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f89fc056d00 0x7f89fc0591c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f8a10006ef0 tx=0x7f8a10015000).stop 2020-11-20T12:22:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200ab0e0 msgr2=0x7f8a20136810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 --2- 172.21.15.44:0/3463383475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a200ab0e0 0x7f8a20136810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8a1800bab0 tx=0x7f8a180091a0).stop 2020-11-20T12:22:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a1ed9d700 1 -- 172.21.15.44:0/3463383475 reap_dead start 2020-11-20T12:22:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a1ed9d700 1 -- 172.21.15.44:0/3463383475 reap_dead start 2020-11-20T12:22:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 shutdown_connections 2020-11-20T12:22:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 >> 172.21.15.44:0/3463383475 conn(0x7f8a20060170 msgr2=0x7f8a2013c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 shutdown_connections 2020-11-20T12:22:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:33.996+0000 7f8a24829700 1 -- 172.21.15.44:0/3463383475 wait complete. 2020-11-20T12:22:34.005 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:22:34.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:34.005 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:34.005 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:22:34.005 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 186, 2020-11-20T12:22:34.006 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:22:34.006 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:22:34.006 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:22:34.006 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:34.006 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 1, 2020-11-20T12:22:34.007 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:34.007 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:22:34.007 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:22:34.007 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:34.008 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:34.008 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:34.008 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:34.008 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:34.008 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:34.009 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:34.009 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:34.009 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:34.009 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:22:34.009 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:22:34.010 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:34.010 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:34.010 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:34.010 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:34.010 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:34.011 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:34.011 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:34.011 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:34.011 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:34.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.012 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:22:34.012 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:22:34.012 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:22:34.012 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:22:34.012 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:22:34.013 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:22:34.014 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:34.014 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:22:34.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:22:34.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:22:34.014 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:22:34.015 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:22:34.015 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:22:34.015 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:22:34.015 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:22:34.016 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:22:34.016 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:22:34.016 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:22:34.016 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:22:34.016 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:22:34.017 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:22:34.017 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:22:34.017 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:22:34.017 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:34.017 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:22:34.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.018 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:22:34.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.018 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:22:34.018 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:22:34.019 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:34.019 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:34.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:34.020 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:22:34.020 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:34.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.020 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:34.021 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:22:34.021 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:34.021 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.021 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:34.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.022 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:22:34.022 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:22:34.022 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:34.022 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:34.022 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:34.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.023 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:22:34.023 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:22:34.023 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:34.024 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:34.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:34.024 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:22:34.025 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:34.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:34.025 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:22:34.026 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:34.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.026 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:34.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.027 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:22:34.027 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:22:34.027 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:34.027 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:34.028 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:34.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.028 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:22:34.028 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:22:34.029 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:22:34.029 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:22:34.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:22:34.029 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:22:34.030 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:34.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:22:34.031 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:22:34.031 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:22:34.031 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.031 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:34.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.032 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:22:34.032 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:22:34.032 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:22:34.032 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:22:34.032 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:22:34.033 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.033 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:34.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.033 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:22:34.033 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:22:34.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.034 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:34.034 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:34.034 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:34.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.035 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:34.035 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:22:34.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.035 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:34.036 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:34.036 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:34.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.036 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:34.036 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:22:34.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.037 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:34.037 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:34.037 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:22:34.038 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.038 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:22:34.038 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:22:34.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:22:34.038 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:22:34.039 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:22:34.039 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:22:34.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:22:34.039 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:22:34.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:22:34.040 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:22:34.040 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:22:34.041 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2020-11-20T12:22:35.630 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 16% (40/247) Checking out files: 17% (42/247) Checking out files: 18% (45/247) Checking out files: 19% (47/247) Checking out files: 19% (48/247) Checking out files: 20% (50/247) Checking out files: 21% (52/247) Checking out files: 22% (55/247) Checking out files: 23% (57/247) Checking out files: 24% (60/247) Checking out files: 25% (62/247) Checking out files: 26% (65/247) Checking out files: 27% (67/247) Checking out files: 28% (70/247) Checking out files: 29% (72/247) Checking out files: 30% (75/247) Checking out files: 31% (77/247) Checking out files: 32% (80/247) Checking out files: 33% (82/247) Checking out files: 34% (84/247) Checking out files: 35% (87/247) Checking out files: 36% (89/247) Checking out files: 37% (92/247) Checking out files: 38% (94/247) Checking out files: 39% (97/247) Checking out files: 40% (99/247) Checking out files: 41% (102/247) Checking out files: 42% (104/247) Checking out files: 43% (107/247) Checking out files: 44% (109/247) Checking out files: 45% (112/247) Checking out files: 46% (114/247) Checking out files: 47% (117/247) Checking out files: 48% (119/247) Checking out files: 49% (122/247) Checking out files: 50% (124/247) Checking out files: 51% (126/247) Checking out files: 52% (129/247) Checking out files: 53% (131/247) Checking out files: 54% (134/247) Checking out files: 55% (136/247) Checking out files: 56% (139/247) Checking out files: 57% (141/247) Checking out files: 58% (144/247) Checking out files: 59% (146/247) Checking out files: 60% (149/247) Checking out files: 61% (151/247) Checking out files: 62% (154/247) Checking out files: 63% (156/247) Checking out files: 64% (159/247) Checking out files: 65% (161/247) Checking out files: 65% (163/247) Checking out files: 66% (164/247) Checking out files: 67% (166/247) Checking out files: 68% (168/247) Checking out files: 69% (171/247) Checking out files: 70% (173/247) Checking out files: 71% (176/247) Checking out files: 72% (178/247) Checking out files: 73% (181/247) Checking out files: 74% (183/247) Checking out files: 75% (186/247) Checking out files: 76% (188/247) Checking out files: 77% (191/247) Checking out files: 78% (193/247) Checking out files: 79% (196/247) Checking out files: 80% (198/247) Checking out files: 81% (201/247) Checking out files: 82% (203/247) Checking out files: 83% (206/247) Checking out files: 84% (208/247) Checking out files: 85% (210/247) Checking out files: 86% (213/247) Checking out files: 87% (215/247) Checking out files: 88% (218/247) Checking out files: 89% (220/247) Checking out files: 90% (223/247) Checking out files: 91% (225/247) Checking out files: 92% (228/247) Checking out files: 93% (230/247) Checking out files: 94% (233/247) Checking out files: 95% (235/247) Checking out files: 96% (238/247) Checking out files: 97% (240/247) Checking out files: 98% (243/247) Checking out files: 99% (245/247) Checking out files: 100% (247/247) Checking out files: 100% (247/247), done. 2020-11-20T12:22:35.776 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 13b1bf7c49 kclient: initializing kaddr (merge fix) 2020-11-20T12:22:35.780 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.4 2020-11-20T12:22:35.895 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:35.896 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.5 2020-11-20T12:22:35.897 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.5 2020-11-20T12:22:35.897 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.5 2020-11-20T12:22:35.898 INFO:tasks.workunit.client.0.smithi044.stdout:v0.5 2020-11-20T12:22:38.488 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 61% (151/245) Checking out files: 62% (152/245) Checking out files: 63% (155/245) Checking out files: 64% (157/245) Checking out files: 65% (160/245) Checking out files: 66% (162/245) Checking out files: 67% (165/245) Checking out files: 68% (167/245) Checking out files: 69% (170/245) Checking out files: 70% (172/245) Checking out files: 71% (174/245) Checking out files: 72% (177/245) Checking out files: 73% (179/245) Checking out files: 74% (182/245) Checking out files: 75% (184/245) Checking out files: 76% (187/245) Checking out files: 77% (189/245) Checking out files: 78% (192/245) Checking out files: 79% (194/245) Checking out files: 80% (196/245) Checking out files: 81% (199/245) Checking out files: 82% (201/245) Checking out files: 83% (204/245) Checking out files: 84% (206/245) Checking out files: 85% (209/245) Checking out files: 86% (211/245) Checking out files: 87% (214/245) Checking out files: 88% (216/245) Checking out files: 89% (219/245) Checking out files: 90% (221/245) Checking out files: 91% (223/245) Checking out files: 92% (226/245) Checking out files: 93% (228/245) Checking out files: 94% (231/245) Checking out files: 95% (233/245) Checking out files: 95% (235/245) Checking out files: 96% (236/245) Checking out files: 97% (238/245) Checking out files: 98% (241/245) Checking out files: 99% (243/245) Checking out files: 100% (245/245) Checking out files: 100% (245/245), done. 2020-11-20T12:22:38.882 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 1dd420957f journal: detect size of raw block devices properly 2020-11-20T12:22:38.882 INFO:tasks.workunit.client.0.smithi044.stdout:v0.6 2020-11-20T12:22:38.883 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.5 2020-11-20T12:22:38.883 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:38.883 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.6 2020-11-20T12:22:38.883 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.6 2020-11-20T12:22:38.883 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.6 2020-11-20T12:22:40.482 INFO:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:40.485 INFO:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:41.204 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 59% (134/227) Checking out files: 60% (137/227) Checking out files: 61% (139/227) Checking out files: 62% (141/227) Checking out files: 63% (144/227) Checking out files: 64% (146/227) Checking out files: 65% (148/227) Checking out files: 66% (150/227) Checking out files: 67% (153/227) Checking out files: 68% (155/227) Checking out files: 69% (157/227) Checking out files: 70% (159/227) Checking out files: 71% (162/227) Checking out files: 72% (164/227) Checking out files: 73% (166/227) Checking out files: 74% (168/227) Checking out files: 75% (171/227) Checking out files: 76% (173/227) Checking out files: 77% (175/227) Checking out files: 78% (178/227) Checking out files: 79% (180/227) Checking out files: 80% (182/227) Checking out files: 81% (184/227) Checking out files: 82% (187/227) Checking out files: 83% (189/227) Checking out files: 84% (191/227) Checking out files: 85% (193/227) Checking out files: 86% (196/227) Checking out files: 87% (198/227) Checking out files: 88% (200/227) Checking out files: 89% (203/227) Checking out files: 90% (205/227) Checking out files: 91% (207/227) Checking out files: 92% (209/227) Checking out files: 93% (212/227) Checking out files: 94% (214/227) Checking out files: 95% (216/227) Checking out files: 96% (218/227) Checking out files: 97% (221/227) Checking out files: 98% (223/227) Checking out files: 99% (225/227) Checking out files: 100% (227/227) Checking out files: 100% (227/227), done. 2020-11-20T12:22:41.275 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at a4c752df71 v0.6 2020-11-20T12:22:41.278 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.6 2020-11-20T12:22:41.483 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:41.484 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.7 2020-11-20T12:22:41.484 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.7 2020-11-20T12:22:41.484 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.7 2020-11-20T12:22:41.484 INFO:tasks.workunit.client.0.smithi044.stdout:v0.7 2020-11-20T12:22:43.597 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 67% (135/201) Checking out files: 68% (137/201) Checking out files: 69% (139/201) Checking out files: 70% (141/201) Checking out files: 71% (143/201) Checking out files: 72% (145/201) Checking out files: 73% (147/201) Checking out files: 74% (149/201) Checking out files: 75% (151/201) Checking out files: 76% (153/201) Checking out files: 77% (155/201) Checking out files: 78% (157/201) Checking out files: 79% (159/201) Checking out files: 80% (161/201) Checking out files: 81% (163/201) Checking out files: 82% (165/201) Checking out files: 83% (167/201) Checking out files: 84% (169/201) Checking out files: 85% (171/201) Checking out files: 86% (173/201) Checking out files: 87% (175/201) Checking out files: 88% (177/201) Checking out files: 89% (179/201) Checking out files: 90% (181/201) Checking out files: 91% (183/201) Checking out files: 92% (185/201) Checking out files: 93% (187/201) Checking out files: 94% (189/201) Checking out files: 95% (191/201) Checking out files: 96% (193/201) Checking out files: 97% (195/201) Checking out files: 98% (197/201) Checking out files: 99% (199/201) Checking out files: 100% (201/201) Checking out files: 100% (201/201), done. 2020-11-20T12:22:43.749 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at c46e1b320c mkcephfs: btrfsctl -a between mkfs and mount 2020-11-20T12:22:44.309 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.7 2020-11-20T12:22:44.310 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:44.311 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.8 2020-11-20T12:22:44.312 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.8 2020-11-20T12:22:44.313 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.8 2020-11-20T12:22:44.314 INFO:tasks.workunit.client.0.smithi044.stdout:v0.8 2020-11-20T12:22:46.361 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 63% (142/225) Checking out files: 64% (144/225) Checking out files: 65% (147/225) Checking out files: 66% (149/225) Checking out files: 67% (151/225) Checking out files: 68% (153/225) Checking out files: 69% (156/225) Checking out files: 70% (158/225) Checking out files: 71% (160/225) Checking out files: 72% (162/225) Checking out files: 73% (165/225) Checking out files: 74% (167/225) Checking out files: 75% (169/225) Checking out files: 76% (171/225) Checking out files: 77% (174/225) Checking out files: 78% (176/225) Checking out files: 79% (178/225) Checking out files: 80% (180/225) Checking out files: 81% (183/225) Checking out files: 82% (185/225) Checking out files: 83% (187/225) Checking out files: 84% (189/225) Checking out files: 85% (192/225) Checking out files: 86% (194/225) Checking out files: 87% (196/225) Checking out files: 88% (198/225) Checking out files: 89% (201/225) Checking out files: 90% (203/225) Checking out files: 91% (205/225) Checking out files: 92% (207/225) Checking out files: 93% (210/225) Checking out files: 94% (212/225) Checking out files: 95% (214/225) Checking out files: 96% (216/225) Checking out files: 97% (219/225) Checking out files: 98% (221/225) Checking out files: 99% (223/225) Checking out files: 100% (225/225) Checking out files: 100% (225/225), done. 2020-11-20T12:22:46.549 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 284161c00a kclient: kill dput/d_drop debug cruft 2020-11-20T12:22:46.553 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.8 2020-11-20T12:22:46.679 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:46.679 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.9 2020-11-20T12:22:46.679 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.9 2020-11-20T12:22:46.680 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.9 2020-11-20T12:22:46.680 INFO:tasks.workunit.client.0.smithi044.stdout:v0.9 2020-11-20T12:22:48.997 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 50% (97/193) Checking out files: 51% (99/193) Checking out files: 52% (101/193) Checking out files: 53% (103/193) Checking out files: 54% (105/193) Checking out files: 55% (107/193) Checking out files: 56% (109/193) Checking out files: 57% (111/193) Checking out files: 58% (112/193) Checking out files: 59% (114/193) Checking out files: 60% (116/193) Checking out files: 61% (118/193) Checking out files: 62% (120/193) Checking out files: 63% (122/193) Checking out files: 64% (124/193) Checking out files: 65% (126/193) Checking out files: 66% (128/193) Checking out files: 67% (130/193) Checking out files: 68% (132/193) Checking out files: 69% (134/193) Checking out files: 70% (136/193) Checking out files: 71% (138/193) Checking out files: 72% (139/193) Checking out files: 73% (141/193) Checking out files: 74% (143/193) Checking out files: 75% (145/193) Checking out files: 76% (147/193) Checking out files: 77% (149/193) Checking out files: 78% (151/193) Checking out files: 79% (153/193) Checking out files: 80% (155/193) Checking out files: 81% (157/193) Checking out files: 82% (159/193) Checking out files: 83% (161/193) Checking out files: 84% (163/193) Checking out files: 85% (165/193) Checking out files: 86% (166/193) Checking out files: 87% (168/193) Checking out files: 88% (170/193) Checking out files: 89% (172/193) Checking out files: 90% (174/193) Checking out files: 91% (176/193) Checking out files: 92% (178/193) Checking out files: 93% (180/193) Checking out files: 94% (182/193) Checking out files: 95% (184/193) Checking out files: 96% (186/193) Checking out files: 97% (188/193) Checking out files: 98% (190/193) Checking out files: 99% (192/193) Checking out files: 100% (193/193) Checking out files: 100% (193/193), done. 2020-11-20T12:22:49.071 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 4148185392 kclient: strip out kernel version compatibility cruft 2020-11-20T12:22:49.075 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.9 2020-11-20T12:22:49.126 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:49.126 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.10 2020-11-20T12:22:49.126 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.10 2020-11-20T12:22:49.126 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.10 2020-11-20T12:22:49.127 INFO:tasks.workunit.client.0.smithi044.stdout:v0.10 2020-11-20T12:22:51.114 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 89% (146/164) Checking out files: 90% (148/164) Checking out files: 91% (150/164) Checking out files: 92% (151/164) Checking out files: 93% (153/164) Checking out files: 94% (155/164) Checking out files: 95% (156/164) Checking out files: 96% (158/164) Checking out files: 97% (160/164) Checking out files: 98% (161/164) Checking out files: 99% (163/164) Checking out files: 100% (164/164) Checking out files: 100% (164/164), done. 2020-11-20T12:22:51.174 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 007313fe84 v0.10 2020-11-20T12:22:51.178 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.10 2020-11-20T12:22:51.451 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:51.451 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.11 2020-11-20T12:22:51.451 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.11 2020-11-20T12:22:51.451 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.11 2020-11-20T12:22:51.452 INFO:tasks.workunit.client.0.smithi044.stdout:v0.11 2020-11-20T12:22:52.691 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 9fa6f5e7d0 v0.11 2020-11-20T12:22:52.693 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.11 2020-11-20T12:22:52.829 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:52.829 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.12 2020-11-20T12:22:52.829 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.12 2020-11-20T12:22:52.830 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.12 2020-11-20T12:22:52.830 INFO:tasks.workunit.client.0.smithi044.stdout:v0.12 2020-11-20T12:22:54.042 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2020-11-20T12:22:54.042 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2020-11-20T12:22:54.043 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:22:54.043 INFO:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:22:54.053 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:22:54.053 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2020-11-20T12:22:54.053 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:54.333 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.330+0000 7fa0a7c36700 1 Processor -- start 2020-11-20T12:22:54.334 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.330+0000 7fa0a7c36700 1 -- start start 2020-11-20T12:22:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a7c36700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa0a00aa1e0 0x7fa0a00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a7c36700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00a4330 0x7fa0a01427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a7c36700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a00a7880 con 0x7fa0a00a9480 2020-11-20T12:22:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a7c36700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a00a5b00 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.335 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a7c36700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00a5da0 con 0x7fa0a00a4330 2020-11-20T12:22:54.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a51d1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00a4330 0x7fa0a01427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a51d1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00a4330 0x7fa0a01427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55256/0 (socket says 172.21.15.44:55256) 2020-11-20T12:22:54.336 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.331+0000 7fa0a51d1700 1 -- 172.21.15.44:0/471073825 learned_addr learned my addr 172.21.15.44:0/471073825 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a51d1700 1 -- 172.21.15.44:0/471073825 >> v2:172.21.15.44:3300/0 conn(0x7fa0a00aa1e0 msgr2=0x7fa0a00a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:54.338 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a51d1700 1 --2- 172.21.15.44:0/471073825 >> v2:172.21.15.44:3300/0 conn(0x7fa0a00aa1e0 0x7fa0a00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:54.339 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a51d1700 1 -- 172.21.15.44:0/471073825 >> v1:172.21.15.44:6789/0 conn(0x7fa0a00a9480 legacy=0x7fa0a0142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a51d1700 1 -- 172.21.15.44:0/471073825 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00a6040 con 0x7fa0a00a4330 2020-11-20T12:22:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a51d1700 1 --2- 172.21.15.44:0/471073825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00a4330 0x7fa0a01427e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa09c006fd0 tx=0x7fa09c006eb0).ready entity=mon.2 client_cookie=466ae61a0ea72058 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a49d0700 1 -- 172.21.15.44:0/471073825 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c009920 con 0x7fa0a00a4330 2020-11-20T12:22:54.340 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a49d0700 1 -- 172.21.15.44:0/471073825 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c009a80 con 0x7fa0a00a4330 2020-11-20T12:22:54.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.332+0000 7fa0a49d0700 1 -- 172.21.15.44:0/471073825 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c009e00 con 0x7fa0a00a4330 2020-11-20T12:22:54.341 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 -- 172.21.15.44:0/471073825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00a4330 msgr2=0x7fa0a01427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/471073825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00a4330 0x7fa0a01427e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa09c006fd0 tx=0x7fa09c006eb0).stop 2020-11-20T12:22:54.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 -- 172.21.15.44:0/471073825 shutdown_connections 2020-11-20T12:22:54.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/471073825 >> v2:172.21.15.44:3300/0 conn(0x7fa0a00aa1e0 0x7fa0a00a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:54.342 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/471073825 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00a4330 0x7fa0a01427e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 -- 172.21.15.44:0/471073825 >> 172.21.15.44:0/471073825 conn(0x7fa0a0062690 msgr2=0x7fa0a00a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:54.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 -- 172.21.15.44:0/471073825 shutdown_connections 2020-11-20T12:22:54.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.333+0000 7fa0a7c36700 1 -- 172.21.15.44:0/471073825 wait complete. 2020-11-20T12:22:54.343 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.334+0000 7fa0a7c36700 1 Processor -- start 2020-11-20T12:22:54.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.334+0000 7fa0a7c36700 1 -- start start 2020-11-20T12:22:54.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.335+0000 7fa0a7c36700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a00a4330 0x7fa0a0067a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.335+0000 7fa0a7c36700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a00a9480 0x7fa0a00abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.335+0000 7fa0a7c36700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00aa1e0 0x7fa0a01399a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.344 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.335+0000 7fa0a7c36700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a009ef60 con 0x7fa0a00a9480 2020-11-20T12:22:54.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.335+0000 7fa0a7c36700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a009f200 con 0x7fa0a00a4330 2020-11-20T12:22:54.345 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.335+0000 7fa0a7c36700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00ac120 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.346 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.335+0000 7fa0a61d3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00aa1e0 0x7fa0a01399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00aa1e0 0x7fa0a01399a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55264/0 (socket says 172.21.15.44:55264) 2020-11-20T12:22:54.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 -- 172.21.15.44:0/1060647573 learned_addr learned my addr 172.21.15.44:0/1060647573 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:54.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a59d2700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a00a4330 0x7fa0a0067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a51d1700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a00a9480 0x7fa0a00abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.347 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 -- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a00a4330 msgr2=0x7fa0a0067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a00a4330 0x7fa0a0067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 -- 172.21.15.44:0/1060647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a00a9480 msgr2=0x7fa0a00abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a00a9480 0x7fa0a00abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.348 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 -- 172.21.15.44:0/1060647573 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a009eb70 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a51d1700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a00a9480 0x7fa0a00abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:54.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a61d3700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00aa1e0 0x7fa0a01399a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa09000c730 tx=0x7fa090007c30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:54.349 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa08dffb700 1 -- 172.21.15.44:0/1060647573 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000e560 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa08dffb700 1 -- 172.21.15.44:0/1060647573 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09000e6e0 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a0139e90 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.336+0000 7fa08dffb700 1 -- 172.21.15.44:0/1060647573 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000eaa0 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.337+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a0066ae0 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.350 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.338+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0066840 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.338+0000 7fa08dffb700 1 -- 172.21.15.44:0/1060647573 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa090010030 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.351 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.338+0000 7fa08dffb700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa078056d20 0x7fa0780591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.339+0000 7fa08dffb700 1 -- 172.21.15.44:0/1060647573 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa09006f5e0 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.339+0000 7fa0a59d2700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa078056d20 0x7fa0780591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.339+0000 7fa0a59d2700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa078056d20 0x7fa0780591e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fa098009ac0 tx=0x7fa098008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:54.352 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.343+0000 7fa08dffb700 1 -- 172.21.15.44:0/1060647573 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa0a0066840 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.694 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.691+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa0a0066840 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.695 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.692+0000 7fa08dffb700 1 -- 172.21.15.44:0/1060647573 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7fa0a0066840 con 0x7fa0a00aa1e0 2020-11-20T12:22:54.695 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":186,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:54.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa078056d20 msgr2=0x7fa0780591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.697 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa078056d20 0x7fa0780591e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fa098009ac0 tx=0x7fa098008040).stop 2020-11-20T12:22:54.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00aa1e0 msgr2=0x7fa0a01399a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00aa1e0 0x7fa0a01399a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa09000c730 tx=0x7fa090007c30).stop 2020-11-20T12:22:54.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 shutdown_connections 2020-11-20T12:22:54.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa078056d20 0x7fa0780591e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.698 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa0a00aa1e0 0x7fa0a01399a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a00a4330 0x7fa0a0067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 --2- 172.21.15.44:0/1060647573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0a00a9480 0x7fa0a00abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 >> 172.21.15.44:0/1060647573 conn(0x7fa0a0062690 msgr2=0x7fa0a00a8010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:54.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 shutdown_connections 2020-11-20T12:22:54.699 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:54.694+0000 7fa0a7c36700 1 -- 172.21.15.44:0/1060647573 wait complete. 2020-11-20T12:22:54.766 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at f524150822 v0.12 2020-11-20T12:22:54.771 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.12 2020-11-20T12:22:54.834 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:54.834 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.13 2020-11-20T12:22:54.835 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.13 2020-11-20T12:22:54.835 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.13 2020-11-20T12:22:54.835 INFO:tasks.workunit.client.0.smithi044.stdout:v0.13 2020-11-20T12:22:56.444 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 95% (127/133) Checking out files: 96% (128/133) Checking out files: 97% (130/133) Checking out files: 98% (131/133) Checking out files: 99% (132/133) Checking out files: 100% (133/133) Checking out files: 100% (133/133), done. 2020-11-20T12:22:56.670 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 2b32297267 v0.13 2020-11-20T12:22:56.671 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.13 2020-11-20T12:22:56.771 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:56.771 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.14 2020-11-20T12:22:56.771 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.14 2020-11-20T12:22:56.772 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.14 2020-11-20T12:22:56.772 INFO:tasks.workunit.client.0.smithi044.stdout:v0.14 2020-11-20T12:22:57.711 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:22:57.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.894+0000 7ff028025700 1 Processor -- start 2020-11-20T12:22:57.898 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.894+0000 7ff028025700 1 -- start start 2020-11-20T12:22:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.895+0000 7ff028025700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0200a9230 0x7ff02013fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:57.901 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.895+0000 7ff028025700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff0200ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:57.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.895+0000 7ff028025700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 0x7ff020142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:57.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.895+0000 7ff028025700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0200ab790 con 0x7ff0200a9230 2020-11-20T12:22:57.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.895+0000 7ff028025700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02013f1f0 con 0x7ff020142640 2020-11-20T12:22:57.902 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.895+0000 7ff028025700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02013f490 con 0x7ff0200a87c0 2020-11-20T12:22:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0200a9230 0x7ff02013fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff0200a9230 0x7ff02013fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43832/0 (socket says 172.21.15.44:43832) 2020-11-20T12:22:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 -- 172.21.15.44:0/2829550040 learned_addr learned my addr 172.21.15.44:0/2829550040 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff0255c0700 1 --2- 172.21.15.44:0/2829550040 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff0200ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff0265c2700 1 --2- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 0x7ff020142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff0265c2700 1 -- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 msgr2=0x7ff020142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:22:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff0265c2700 1 -- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 msgr2=0x7ff020142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:22:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff0265c2700 1 --2- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 0x7ff020142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:22:57.904 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff0265c2700 1 --2- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 0x7ff020142ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 -- 172.21.15.44:0/2829550040 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 msgr2=0x7ff0200ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 --2- 172.21.15.44:0/2829550040 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff0200ab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 -- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 msgr2=0x7ff020142ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:22:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 --2- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 0x7ff020142ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:57.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 -- 172.21.15.44:0/2829550040 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02013f730 con 0x7ff0200a9230 2020-11-20T12:22:57.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff025dc1700 1 --2- 172.21.15.44:0/2829550040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0200a9230 0x7ff02013fbd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7ff0200a9dd0 tx=0x7ff01c006eb0).ready entity=mon.0 client_cookie=879bb56560d774d6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:57.907 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff024dbf700 1 -- 172.21.15.44:0/2829550040 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c009810 con 0x7ff0200a9230 2020-11-20T12:22:57.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff024dbf700 1 -- 172.21.15.44:0/2829550040 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c009970 con 0x7ff0200a9230 2020-11-20T12:22:57.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.896+0000 7ff024dbf700 1 -- 172.21.15.44:0/2829550040 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c009cf0 con 0x7ff0200a9230 2020-11-20T12:22:57.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.898+0000 7ff028025700 1 -- 172.21.15.44:0/2829550040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0200a9230 msgr2=0x7ff02013fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:57.908 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.898+0000 7ff028025700 1 --2- 172.21.15.44:0/2829550040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0200a9230 0x7ff02013fbd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7ff0200a9dd0 tx=0x7ff01c006eb0).stop 2020-11-20T12:22:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.898+0000 7ff028025700 1 -- 172.21.15.44:0/2829550040 shutdown_connections 2020-11-20T12:22:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.898+0000 7ff028025700 1 --2- 172.21.15.44:0/2829550040 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0200a9230 0x7ff02013fbd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.898+0000 7ff028025700 1 --2- 172.21.15.44:0/2829550040 >> v2:172.21.15.79:3300/0 conn(0x7ff020142640 0x7ff020142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.898+0000 7ff028025700 1 --2- 172.21.15.44:0/2829550040 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff0200ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.898+0000 7ff028025700 1 -- 172.21.15.44:0/2829550040 >> 172.21.15.44:0/2829550040 conn(0x7ff0200621c0 msgr2=0x7ff020060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:57.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.900+0000 7ff028025700 1 -- 172.21.15.44:0/2829550040 shutdown_connections 2020-11-20T12:22:57.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.902+0000 7ff028025700 1 -- 172.21.15.44:0/2829550040 wait complete. 2020-11-20T12:22:57.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.903+0000 7ff028025700 1 Processor -- start 2020-11-20T12:22:57.913 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.904+0000 7ff028025700 1 -- start start 2020-11-20T12:22:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.904+0000 7ff028025700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff020137310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.904+0000 7ff028025700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 0x7ff02014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.904+0000 7ff028025700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff020142640 0x7ff020134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.904+0000 7ff028025700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014efe0 con 0x7ff020142640 2020-11-20T12:22:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.904+0000 7ff028025700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014f280 con 0x7ff0200a9230 2020-11-20T12:22:57.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.904+0000 7ff028025700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014f520 con 0x7ff0200a87c0 2020-11-20T12:22:57.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff020137310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:57.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff0255c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 0x7ff02014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:57.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff020137310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55276/0 (socket says 172.21.15.44:55276) 2020-11-20T12:22:57.915 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 -- 172.21.15.44:0/803917354 learned_addr learned my addr 172.21.15.44:0/803917354 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:22:57.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff0265c2700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff020142640 0x7ff020134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:57.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff0255c0700 1 -- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 msgr2=0x7ff02014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2020-11-20T12:22:57.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff0255c0700 1 -- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 msgr2=0x7ff02014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2020-11-20T12:22:57.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff0255c0700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 0x7ff02014e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:22:57.916 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff0255c0700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 0x7ff02014e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:22:57.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 -- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 msgr2=0x7ff02014e5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:22:57.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 0x7ff02014e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:57.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 -- 172.21.15.44:0/803917354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff020142640 msgr2=0x7ff020134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:57.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff020142640 0x7ff020134dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:57.917 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 -- 172.21.15.44:0/803917354 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0201419f0 con 0x7ff0200a87c0 2020-11-20T12:22:57.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff025dc1700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff020137310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ff020063ea0 tx=0x7ff01c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:57.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff015ffb700 1 -- 172.21.15.44:0/803917354 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c011d50 con 0x7ff0200a87c0 2020-11-20T12:22:57.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff015ffb700 1 -- 172.21.15.44:0/803917354 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c01e460 con 0x7ff0200a87c0 2020-11-20T12:22:57.918 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02014eae0 con 0x7ff0200a87c0 2020-11-20T12:22:57.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.905+0000 7ff015ffb700 1 -- 172.21.15.44:0/803917354 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c01e7e0 con 0x7ff0200a87c0 2020-11-20T12:22:57.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.906+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff020135500 con 0x7ff0200a87c0 2020-11-20T12:22:57.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.907+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02014ee10 con 0x7ff0200a87c0 2020-11-20T12:22:57.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.907+0000 7ff015ffb700 1 -- 172.21.15.44:0/803917354 <== mon.2 v2:172.21.15.79:3301/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7ff01c025030 con 0x7ff0200a87c0 2020-11-20T12:22:57.919 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.907+0000 7ff015ffb700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056d20 0x7ff0000591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:57.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.908+0000 7ff015ffb700 1 -- 172.21.15.44:0/803917354 <== mon.2 v2:172.21.15.79:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7ff01c071730 con 0x7ff0200a87c0 2020-11-20T12:22:57.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.908+0000 7ff0255c0700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056d20 0x7ff0000591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:57.920 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.908+0000 7ff0255c0700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056d20 0x7ff0000591e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7ff0200a5b40 tx=0x7ff010008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:57.921 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:57.911+0000 7ff015ffb700 1 -- 172.21.15.44:0/803917354 <== mon.2 v2:172.21.15.79:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff01c046ab0 con 0x7ff0200a87c0 2020-11-20T12:22:58.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.289+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff0200aa230 con 0x7ff0200a87c0 2020-11-20T12:22:58.294 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.290+0000 7ff015ffb700 1 -- 172.21.15.44:0/803917354 <== mon.2 v2:172.21.15.79:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1465 (secure 0 0 0) 0x7ff01c023020 con 0x7ff0200a87c0 2020-11-20T12:22:58.295 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":186,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:22:58.298 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056d20 msgr2=0x7ff0000591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:58.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056d20 0x7ff0000591e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7ff0200a5b40 tx=0x7ff010008040).stop 2020-11-20T12:22:58.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 msgr2=0x7ff020137310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:58.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff020137310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7ff020063ea0 tx=0x7ff01c015000).stop 2020-11-20T12:22:58.299 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 shutdown_connections 2020-11-20T12:22:58.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7ff000056d20 0x7ff0000591e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:58.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff020142640 0x7ff020134dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:58.300 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7ff0200a87c0 0x7ff020137310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:58.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 --2- 172.21.15.44:0/803917354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0200a9230 0x7ff02014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:58.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 >> 172.21.15.44:0/803917354 conn(0x7ff0200621c0 msgr2=0x7ff020060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:58.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 shutdown_connections 2020-11-20T12:22:58.301 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:22:58.292+0000 7ff028025700 1 -- 172.21.15.44:0/803917354 wait complete. 2020-11-20T12:22:58.702 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 80% (121/151) Checking out files: 81% (123/151) Checking out files: 82% (124/151) Checking out files: 83% (126/151) Checking out files: 84% (127/151) Checking out files: 85% (129/151) Checking out files: 86% (130/151) Checking out files: 87% (132/151) Checking out files: 88% (133/151) Checking out files: 89% (135/151) Checking out files: 90% (136/151) Checking out files: 91% (138/151) Checking out files: 92% (139/151) Checking out files: 93% (141/151) Checking out files: 94% (142/151) Checking out files: 95% (144/151) Checking out files: 96% (145/151) Checking out files: 97% (147/151) Checking out files: 98% (148/151) Checking out files: 99% (150/151) Checking out files: 100% (151/151) Checking out files: 100% (151/151), done. 2020-11-20T12:22:58.915 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 1ec1e8d6fb v0.14 2020-11-20T12:22:58.925 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.14 2020-11-20T12:22:58.959 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:22:58.959 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.15 2020-11-20T12:22:58.959 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.15 2020-11-20T12:22:58.959 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.15 2020-11-20T12:22:58.960 INFO:tasks.workunit.client.0.smithi044.stdout:v0.15 2020-11-20T12:23:00.912 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 75% (85/112) Checking out files: 76% (86/112) Checking out files: 77% (87/112) Checking out files: 78% (88/112) Checking out files: 79% (89/112) Checking out files: 80% (90/112) Checking out files: 81% (91/112) Checking out files: 82% (92/112) Checking out files: 83% (93/112) Checking out files: 84% (95/112) Checking out files: 85% (96/112) Checking out files: 86% (97/112) Checking out files: 87% (98/112) Checking out files: 88% (99/112) Checking out files: 89% (100/112) Checking out files: 90% (101/112) Checking out files: 91% (102/112) Checking out files: 92% (104/112) Checking out files: 93% (105/112) Checking out files: 94% (106/112) Checking out files: 95% (107/112) Checking out files: 96% (108/112) Checking out files: 97% (109/112) Checking out files: 98% (110/112) Checking out files: 99% (111/112) Checking out files: 100% (112/112) Checking out files: 100% (112/112), done. 2020-11-20T12:23:01.355 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2020-11-20T12:23:01.356 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at ab3c421eb9 v0.15 2020-11-20T12:23:01.357 INFO:tasks.workunit.client.0.smithi044.stdout:v0.16 2020-11-20T12:23:01.357 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.15 2020-11-20T12:23:01.357 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:23:01.357 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.16 2020-11-20T12:23:01.358 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.16 2020-11-20T12:23:01.358 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.16 2020-11-20T12:23:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.573+0000 7f524105c700 1 Processor -- start 2020-11-20T12:23:01.642 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.573+0000 7f524105c700 1 -- start start 2020-11-20T12:23:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.574+0000 7f524105c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f523c0a3cb0 0x7f523c0aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.574+0000 7f524105c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.574+0000 7f524105c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c0a57b0 con 0x7f523c0a5fd0 2020-11-20T12:23:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.574+0000 7f524105c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523c0a4090 con 0x7f523c0a3cb0 2020-11-20T12:23:01.643 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.574+0000 7f524105c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0a82f0 con 0x7f523c0a5b70 2020-11-20T12:23:01.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.575+0000 7f523a59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:01.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.575+0000 7f523ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f523c0a3cb0 0x7f523c0aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:01.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.575+0000 7f523a59c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3301/0 says I am v2:172.21.15.44:55286/0 (socket says 172.21.15.44:55286) 2020-11-20T12:23:01.644 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.575+0000 7f523ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f523c0a3cb0 0x7f523c0aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35898/0 (socket says 172.21.15.44:35898) 2020-11-20T12:23:01.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.575+0000 7f523a59c700 1 -- 172.21.15.44:0/1504764815 learned_addr learned my addr 172.21.15.44:0/1504764815 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:01.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.583+0000 7f5239d9b700 1 -- 172.21.15.44:0/1504764815 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665174133 0 0) 0x7f523c0a57b0 con 0x7f523c0a5fd0 2020-11-20T12:23:01.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.638+0000 7f5239d9b700 1 -- 172.21.15.44:0/1504764815 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5224003220 con 0x7f523c0a5fd0 2020-11-20T12:23:01.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.638+0000 7f523a59c700 1 -- 172.21.15.44:0/1504764815 >> v2:172.21.15.79:3300/0 conn(0x7f523c0a3cb0 msgr2=0x7f523c0aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:23:01.645 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.638+0000 7f523a59c700 1 --2- 172.21.15.44:0/1504764815 >> v2:172.21.15.79:3300/0 conn(0x7f523c0a3cb0 0x7f523c0aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.638+0000 7f523a59c700 1 -- 172.21.15.44:0/1504764815 >> v1:172.21.15.79:6789/0 conn(0x7f523c0a5fd0 legacy=0x7f523c060660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.639+0000 7f523a59c700 1 -- 172.21.15.44:0/1504764815 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0a80c0 con 0x7f523c0a5b70 2020-11-20T12:23:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.639+0000 7f523a59c700 1 --2- 172.21.15.44:0/1504764815 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c060170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f522c00dc30 tx=0x7f522c006eb0).ready entity=mon.2 client_cookie=fc5b71d1bdb131b2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.639+0000 7f5239d9b700 1 -- 172.21.15.44:0/1504764815 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c013070 con 0x7f523c0a5b70 2020-11-20T12:23:01.646 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.639+0000 7f5239d9b700 1 -- 172.21.15.44:0/1504764815 <== mon.2 v2:172.21.15.79:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c00c070 con 0x7f523c0a5b70 2020-11-20T12:23:01.649 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.639+0000 7f5239d9b700 1 -- 172.21.15.44:0/1504764815 <== mon.2 v2:172.21.15.79:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c009aa0 con 0x7f523c0a5b70 2020-11-20T12:23:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.639+0000 7f524105c700 1 -- 172.21.15.44:0/1504764815 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 msgr2=0x7f523c060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.640+0000 7f524105c700 1 --2- 172.21.15.44:0/1504764815 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c060170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f522c00dc30 tx=0x7f522c006eb0).stop 2020-11-20T12:23:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.640+0000 7f524105c700 1 -- 172.21.15.44:0/1504764815 shutdown_connections 2020-11-20T12:23:01.650 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.640+0000 7f524105c700 1 --2- 172.21.15.44:0/1504764815 >> v2:172.21.15.79:3300/0 conn(0x7f523c0a3cb0 0x7f523c0aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.640+0000 7f524105c700 1 --2- 172.21.15.44:0/1504764815 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c060170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.640+0000 7f524105c700 1 -- 172.21.15.44:0/1504764815 >> 172.21.15.44:0/1504764815 conn(0x7f523c062600 msgr2=0x7f523c0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.640+0000 7f524105c700 1 -- 172.21.15.44:0/1504764815 shutdown_connections 2020-11-20T12:23:01.651 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.640+0000 7f524105c700 1 -- 172.21.15.44:0/1504764815 wait complete. 2020-11-20T12:23:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.641+0000 7f524105c700 1 Processor -- start 2020-11-20T12:23:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.641+0000 7f524105c700 1 -- start start 2020-11-20T12:23:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f524105c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f523c0a3cb0 0x7f523c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f524105c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c0a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:01.652 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f524105c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c0a5fd0 0x7f523c09d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:01.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f524105c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c157540 con 0x7f523c0a3cb0 2020-11-20T12:23:01.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f524105c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c1577e0 con 0x7f523c0a5fd0 2020-11-20T12:23:01.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f524105c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c157a80 con 0x7f523c0a5b70 2020-11-20T12:23:01.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f523c0a3cb0 0x7f523c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:01.653 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f523c0a3cb0 0x7f523c156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43854/0 (socket says 172.21.15.44:43854) 2020-11-20T12:23:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 -- 172.21.15.44:0/1190062476 learned_addr learned my addr 172.21.15.44:0/1190062476 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523a59c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c0a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523b59e700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c0a5fd0 0x7f523c09d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 -- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 msgr2=0x7f523c0a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:01.654 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c0a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 -- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c0a5fd0 msgr2=0x7f523c09d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c0a5fd0 0x7f523c09d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523ad9d700 1 -- 172.21.15.44:0/1190062476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c066730 con 0x7f523c0a3cb0 2020-11-20T12:23:01.655 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.642+0000 7f523b59e700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c0a5fd0 0x7f523c09d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:23:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.643+0000 7f523ad9d700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f523c0a3cb0 0x7f523c156ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f523400fc90 tx=0x7f523400a2b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.643+0000 7f5222ffd700 1 -- 172.21.15.44:0/1190062476 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400dac0 con 0x7f523c0a3cb0 2020-11-20T12:23:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.643+0000 7f5222ffd700 1 -- 172.21.15.44:0/1190062476 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523400dc20 con 0x7f523c0a3cb0 2020-11-20T12:23:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.643+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c0669e0 con 0x7f523c0a3cb0 2020-11-20T12:23:01.656 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.643+0000 7f5222ffd700 1 -- 172.21.15.44:0/1190062476 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234012460 con 0x7f523c0a3cb0 2020-11-20T12:23:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.643+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c09db20 con 0x7f523c0a3cb0 2020-11-20T12:23:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.644+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c157390 con 0x7f523c0a3cb0 2020-11-20T12:23:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.644+0000 7f5222ffd700 1 -- 172.21.15.44:0/1190062476 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f523400e040 con 0x7f523c0a3cb0 2020-11-20T12:23:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.644+0000 7f5222ffd700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5218056d20 0x7f52180591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:01.657 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.645+0000 7f5222ffd700 1 -- 172.21.15.44:0/1190062476 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f5234070800 con 0x7f523c0a3cb0 2020-11-20T12:23:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.645+0000 7f523a59c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5218056d20 0x7f52180591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:01.658 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.645+0000 7f523a59c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5218056d20 0x7f52180591e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f523c0a36f0 tx=0x7f522c015000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:01.659 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.648+0000 7f5222ffd700 1 -- 172.21.15.44:0/1190062476 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5234045ad0 con 0x7f523c0a3cb0 2020-11-20T12:23:01.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.948+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f523c061650 con 0x7f523c0a3cb0 2020-11-20T12:23:01.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.948+0000 7f5222ffd700 1 -- 172.21.15.44:0/1190062476 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1470 (secure 0 0 0) 0x7f5234016070 con 0x7f523c0a3cb0 2020-11-20T12:23:01.952 INFO:teuthology.orchestra.run.smithi044.stdout:{"election_epoch":188,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138297136906239","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"]},"monmap":{"epoch":1,"fsid":"0437e350-ccfe-424c-9e91-59ff58800c77","modified":"2020-11-20T12:02:06.403922Z","created":"2020-11-20T12:02:06.403922Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2020-11-20T12:23:01.953 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5218056d20 msgr2=0x7f52180591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5218056d20 0x7f52180591e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f523c0a36f0 tx=0x7f522c015000).stop 2020-11-20T12:23:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f523c0a3cb0 msgr2=0x7f523c156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f523c0a3cb0 0x7f523c156ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f523400fc90 tx=0x7f523400a2b0).stop 2020-11-20T12:23:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 shutdown_connections 2020-11-20T12:23:01.954 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f5218056d20 0x7f52180591e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f523c0a5fd0 0x7f523c09d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f523c0a3cb0 0x7f523c156ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 --2- 172.21.15.44:0/1190062476 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f523c0a5b70 0x7f523c0a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 >> 172.21.15.44:0/1190062476 conn(0x7f523c062600 msgr2=0x7f523c0ac5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:01.955 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 shutdown_connections 2020-11-20T12:23:01.956 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:01.950+0000 7f524105c700 1 -- 172.21.15.44:0/1190062476 wait complete. 2020-11-20T12:23:01.965 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2020-11-20T12:23:01.966 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2020-11-20T12:23:02.144 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.138+0000 7fa617c58700 1 Processor -- start 2020-11-20T12:23:02.145 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.138+0000 7fa617c58700 1 -- start start 2020-11-20T12:23:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.139+0000 7fa617c58700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa6100aad00 0x7fa6100a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.139+0000 7fa617c58700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa61013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.146 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.139+0000 7fa617c58700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa6100a86d0 0x7fa6100abc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.139+0000 7fa617c58700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6101385f0 con 0x7fa6100aad00 2020-11-20T12:23:02.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.139+0000 7fa617c58700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61013e250 con 0x7fa6100a86d0 2020-11-20T12:23:02.147 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.139+0000 7fa617c58700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61013e4f0 con 0x7fa6100ab0e0 2020-11-20T12:23:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.140+0000 7fa6159f4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa6100aad00 0x7fa6100a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.140+0000 7fa6159f4700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa6100aad00 0x7fa6100a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43862/0 (socket says 172.21.15.44:43862) 2020-11-20T12:23:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.140+0000 7fa6161f5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa6100a86d0 0x7fa6100abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.148 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.140+0000 7fa6159f4700 1 -- 172.21.15.44:0/402892102 learned_addr learned my addr 172.21.15.44:0/402892102 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:02.149 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.140+0000 7fa6159f4700 1 -- 172.21.15.44:0/402892102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 msgr2=0x7fa61013ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:23:02.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.141+0000 7fa6151f3700 1 --2- 172.21.15.44:0/402892102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa61013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.151 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.142+0000 7fa6159f4700 1 --2- 172.21.15.44:0/402892102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa61013ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.142+0000 7fa6159f4700 1 -- 172.21.15.44:0/402892102 >> v2:172.21.15.79:3300/0 conn(0x7fa6100a86d0 msgr2=0x7fa6100abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:23:02.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.142+0000 7fa6159f4700 1 --2- 172.21.15.44:0/402892102 >> v2:172.21.15.79:3300/0 conn(0x7fa6100a86d0 0x7fa6100abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.142+0000 7fa6159f4700 1 -- 172.21.15.44:0/402892102 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100ac120 con 0x7fa6100aad00 2020-11-20T12:23:02.152 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.142+0000 7fa6159f4700 1 --2- 172.21.15.44:0/402892102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100aad00 0x7fa6100a94d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fa6100a99c0 tx=0x7fa60c006eb0).ready entity=mon.0 client_cookie=264a3dc40f11d13b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.144+0000 7fa6161f5700 1 --2- 172.21.15.44:0/402892102 >> v2:172.21.15.79:3300/0 conn(0x7fa6100a86d0 0x7fa6100abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:23:02.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.144+0000 7fa6151f3700 1 --2- 172.21.15.44:0/402892102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa61013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:23:02.153 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.145+0000 7fa6149f2700 1 -- 172.21.15.44:0/402892102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0098a0 con 0x7fa6100aad00 2020-11-20T12:23:02.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.145+0000 7fa6149f2700 1 -- 172.21.15.44:0/402892102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c009a00 con 0x7fa6100aad00 2020-11-20T12:23:02.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.145+0000 7fa6149f2700 1 -- 172.21.15.44:0/402892102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c009d80 con 0x7fa6100aad00 2020-11-20T12:23:02.154 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.145+0000 7fa617c58700 1 -- 172.21.15.44:0/402892102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100aad00 msgr2=0x7fa6100a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.146+0000 7fa617c58700 1 --2- 172.21.15.44:0/402892102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100aad00 0x7fa6100a94d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fa6100a99c0 tx=0x7fa60c006eb0).stop 2020-11-20T12:23:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.146+0000 7fa617c58700 1 -- 172.21.15.44:0/402892102 shutdown_connections 2020-11-20T12:23:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.146+0000 7fa617c58700 1 --2- 172.21.15.44:0/402892102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100aad00 0x7fa6100a94d0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0x7fa6100a99c0 tx=0x7fa60c006eb0).stop 2020-11-20T12:23:02.155 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.146+0000 7fa617c58700 1 --2- 172.21.15.44:0/402892102 >> v2:172.21.15.79:3300/0 conn(0x7fa6100a86d0 0x7fa6100abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.146+0000 7fa617c58700 1 --2- 172.21.15.44:0/402892102 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa61013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.146+0000 7fa617c58700 1 -- 172.21.15.44:0/402892102 >> 172.21.15.44:0/402892102 conn(0x7fa610060170 msgr2=0x7fa610060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:02.156 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.146+0000 7fa617c58700 1 -- 172.21.15.44:0/402892102 shutdown_connections 2020-11-20T12:23:02.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.149+0000 7fa617c58700 1 -- 172.21.15.44:0/402892102 wait complete. 2020-11-20T12:23:02.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.150+0000 7fa617c58700 1 Processor -- start 2020-11-20T12:23:02.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.150+0000 7fa617c58700 1 -- start start 2020-11-20T12:23:02.161 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa617c58700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a86d0 0x7fa61009e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa617c58700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa6100a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa617c58700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa61014d600 0x7fa6100a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa617c58700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100a53b0 con 0x7fa6100a86d0 2020-11-20T12:23:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa617c58700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100a6ac0 con 0x7fa61014d600 2020-11-20T12:23:02.162 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa617c58700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100a60e0 con 0x7fa6100ab0e0 2020-11-20T12:23:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a86d0 0x7fa61009e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a86d0 0x7fa61009e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43868/0 (socket says 172.21.15.44:43868) 2020-11-20T12:23:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 -- 172.21.15.44:0/497674648 learned_addr learned my addr 172.21.15.44:0/497674648 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6151f3700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa6100a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.163 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6161f5700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa61014d600 0x7fa6100a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 msgr2=0x7fa6100a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fa6100ab0e0 0x7fa6100a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa61014d600 msgr2=0x7fa6100a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa61014d600 0x7fa6100a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.164 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 -- 172.21.15.44:0/497674648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100a6380 con 0x7fa6100a86d0 2020-11-20T12:23:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa6159f4700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a86d0 0x7fa61009e200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa60c006ef0 tx=0x7fa60c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.151+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c011e70 con 0x7fa6100a86d0 2020-11-20T12:23:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.152+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c01e460 con 0x7fa6100a86d0 2020-11-20T12:23:02.165 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.152+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c01e800 con 0x7fa6100a86d0 2020-11-20T12:23:02.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.152+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6100a5dd0 con 0x7fa6100a86d0 2020-11-20T12:23:02.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.152+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6100a7a70 con 0x7fa6100a86d0 2020-11-20T12:23:02.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.153+0000 7fa617c58700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a6670 0x7fa61014d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.153+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61005c5f0 con 0x7fa6100a6670 2020-11-20T12:23:02.166 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.153+0000 7fa6151f3700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a6670 0x7fa61014d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.153+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fa60c025030 con 0x7fa6100a86d0 2020-11-20T12:23:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.153+0000 7fa6151f3700 1 -- 172.21.15.44:0/497674648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa610069680 con 0x7fa6100a6670 2020-11-20T12:23:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa6151f3700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a6670 0x7fa61014d9e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fa61009d830 tx=0x7fa600008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa5e8056d70 0x7fa5e8059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.167 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fa60c071600 con 0x7fa6100a86d0 2020-11-20T12:23:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000a6c0 con 0x7fa6100a6670 2020-11-20T12:23:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa6161f5700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa5e8056d70 0x7fa5e8059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fa60000a820 con 0x7fa6100a6670 2020-11-20T12:23:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a6670 msgr2=0x7fa61014d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.168 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a6670 0x7fa61014d9e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fa61009d830 tx=0x7fa600008040).stop 2020-11-20T12:23:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a6670 msgr2=0x7fa61014d9e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:23:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa5fdffb700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a6670 0x7fa61014d9e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.169 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.154+0000 7fa6161f5700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa5e8056d70 0x7fa5e8059230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fa608009440 tx=0x7fa608008040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.167+0000 7fa617c58700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a42d0 0x7fa61014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.167+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100a6220 con 0x7fa6100a42d0 2020-11-20T12:23:02.170 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.167+0000 7fa6151f3700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a42d0 0x7fa61014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.167+0000 7fa6151f3700 1 -- 172.21.15.44:0/497674648 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa6100446d0 con 0x7fa6100a42d0 2020-11-20T12:23:02.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.168+0000 7fa6151f3700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a42d0 0x7fa61014a0b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fa60000f010 tx=0x7fa600008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.172 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.168+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60001b5e0 con 0x7fa6100a42d0 2020-11-20T12:23:02.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.168+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 <== mon.0 v2:172.21.15.44:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4778 (secure 0 0 0) 0x7fa60000a4b0 con 0x7fa6100a42d0 2020-11-20T12:23:02.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.168+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a42d0 msgr2=0x7fa61014a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.168+0000 7fa5fdffb700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a42d0 0x7fa61014a0b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fa60000f010 tx=0x7fa600008040).stop 2020-11-20T12:23:02.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.168+0000 7fa5fdffb700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a42d0 msgr2=0x7fa61014a0b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:23:02.173 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.168+0000 7fa5fdffb700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a42d0 0x7fa61014a0b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa5e8056d70 msgr2=0x7fa5e8059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fa5e8056d70 0x7fa5e8059230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fa608009440 tx=0x7fa608008040).stop 2020-11-20T12:23:02.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a86d0 msgr2=0x7fa61009e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.174 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 --2- 172.21.15.44:0/497674648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6100a86d0 0x7fa61009e200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa60c006ef0 tx=0x7fa60c015000).stop 2020-11-20T12:23:02.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa6161f5700 1 -- 172.21.15.44:0/497674648 reap_dead start 2020-11-20T12:23:02.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa6161f5700 1 -- 172.21.15.44:0/497674648 reap_dead start 2020-11-20T12:23:02.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 shutdown_connections 2020-11-20T12:23:02.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 >> 172.21.15.44:0/497674648 conn(0x7fa610060170 msgr2=0x7fa6100afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:02.175 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 shutdown_connections 2020-11-20T12:23:02.176 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.169+0000 7fa617c58700 1 -- 172.21.15.44:0/497674648 wait complete. 2020-11-20T12:23:02.176 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:23:02.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:23:02.176 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:23:02.177 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "leader", 2020-11-20T12:23:02.177 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 188, 2020-11-20T12:23:02.177 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:23:02.177 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:23:02.177 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:23:02.178 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:23:02.178 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.178 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 2, 2020-11-20T12:23:02.178 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:23:02.178 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:23:02.179 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:23:02.179 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.179 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.179 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.179 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.180 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.180 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.180 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.180 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.181 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:23:02.181 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:23:02.181 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.181 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.181 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.182 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.182 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.182 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.182 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.182 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.183 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:23:02.183 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:23:02.183 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:23:02.184 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:23:02.184 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:23:02.184 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:23:02.184 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:23:02.184 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:23:02.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:23:02.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:23:02.185 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:23:02.185 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:23:02.185 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:23:02.186 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:23:02.186 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:23:02.186 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.186 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.186 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.187 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.187 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.187 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.187 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.187 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.188 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.188 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:23:02.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.188 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:23:02.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.189 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:23:02.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:23:02.189 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:23:02.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:23:02.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.192 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:23:02.192 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:23:02.192 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.193 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.193 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.193 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:23:02.194 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:23:02.194 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:23:02.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:23:02.196 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:23:02.197 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:23:02.197 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.197 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.197 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.198 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:23:02.198 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:23:02.198 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:23:02.199 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:23:02.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.201 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:23:02.201 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:23:02.202 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.202 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.202 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.202 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.203 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:23:02.203 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:23:02.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.203 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.204 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.204 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:23:02.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.204 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.204 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:23:02.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.205 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.205 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.205 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:23:02.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.206 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.206 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:23:02.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.206 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.206 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.207 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 7 2020-11-20T12:23:02.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.207 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.207 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:23:02.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.208 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.208 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.208 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 5 2020-11-20T12:23:02.208 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.208 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.209 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": [ 2020-11-20T12:23:02.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.209 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.209 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.209 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:23:02.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.210 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.210 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:23:02.210 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:23:02.212 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2020-11-20T12:23:02.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.396+0000 7fc3f2b1c700 1 Processor -- start 2020-11-20T12:23:02.400 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.397+0000 7fc3f2b1c700 1 -- start start 2020-11-20T12:23:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.397+0000 7fc3f2b1c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0aa1e0 0x7fc3ec0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.397+0000 7fc3f2b1c700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3ec0a9480 0x7fc3ec142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.403 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.397+0000 7fc3f2b1c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec0a7880 con 0x7fc3ec0a4330 2020-11-20T12:23:02.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.397+0000 7fc3f2b1c700 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec0a5b00 con 0x7fc3ec0a9480 2020-11-20T12:23:02.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.397+0000 7fc3f2b1c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0a5da0 con 0x7fc3ec0aa1e0 2020-11-20T12:23:02.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f10b9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3ec0a9480 0x7fc3ec142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.404 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f10b9700 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3ec0a9480 0x7fc3ec142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43880/0 (socket says 172.21.15.44:43880) 2020-11-20T12:23:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f10b9700 1 -- 172.21.15.44:0/3757629353 learned_addr learned my addr 172.21.15.44:0/3757629353 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f08b8700 1 --2- 172.21.15.44:0/3757629353 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0aa1e0 0x7fc3ec0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3eb7fe700 1 -- 172.21.15.44:0/3757629353 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63988068 0 0) 0x7fc3ec0a7880 con 0x7fc3ec0a4330 2020-11-20T12:23:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3eb7fe700 1 -- 172.21.15.44:0/3757629353 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d0003240 con 0x7fc3ec0a4330 2020-11-20T12:23:02.405 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f10b9700 1 -- 172.21.15.44:0/3757629353 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0aa1e0 msgr2=0x7fc3ec0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:23:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f10b9700 1 --2- 172.21.15.44:0/3757629353 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0aa1e0 0x7fc3ec0a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f10b9700 1 -- 172.21.15.44:0/3757629353 >> v1:172.21.15.79:6789/0 conn(0x7fc3ec0a4330 legacy=0x7fc3ec1427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.398+0000 7fc3f10b9700 1 -- 172.21.15.44:0/3757629353 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0a6040 con 0x7fc3ec0a9480 2020-11-20T12:23:02.406 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.399+0000 7fc3f10b9700 1 --2- 172.21.15.44:0/3757629353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ec0a9480 0x7fc3ec142ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fc3e4009d50 tx=0x7fc3e4009290).ready entity=mon.0 client_cookie=3dc3680258b61ae0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.399+0000 7fc3eb7fe700 1 -- 172.21.15.44:0/3757629353 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400bba0 con 0x7fc3ec0a9480 2020-11-20T12:23:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.399+0000 7fc3eb7fe700 1 -- 172.21.15.44:0/3757629353 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e400bd00 con 0x7fc3ec0a9480 2020-11-20T12:23:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.399+0000 7fc3eb7fe700 1 -- 172.21.15.44:0/3757629353 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4011610 con 0x7fc3ec0a9480 2020-11-20T12:23:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.400+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/3757629353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ec0a9480 msgr2=0x7fc3ec142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.407 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.400+0000 7fc3f2b1c700 1 --2- 172.21.15.44:0/3757629353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ec0a9480 0x7fc3ec142ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fc3e4009d50 tx=0x7fc3e4009290).stop 2020-11-20T12:23:02.409 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.401+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/3757629353 shutdown_connections 2020-11-20T12:23:02.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.401+0000 7fc3f2b1c700 1 --2- 172.21.15.44:0/3757629353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ec0a9480 0x7fc3ec142ef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.401+0000 7fc3f2b1c700 1 --2- 172.21.15.44:0/3757629353 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0aa1e0 0x7fc3ec0a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.401+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/3757629353 >> 172.21.15.44:0/3757629353 conn(0x7fc3ec062690 msgr2=0x7fc3ec0a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:02.410 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.401+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/3757629353 shutdown_connections 2020-11-20T12:23:02.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.401+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/3757629353 wait complete. 2020-11-20T12:23:02.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.403+0000 7fc3f2b1c700 1 Processor -- start 2020-11-20T12:23:02.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.403+0000 7fc3f2b1c700 1 -- start start 2020-11-20T12:23:02.411 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.403+0000 7fc3f2b1c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec0a4330 0x7fc3ec14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f2b1c700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0a9480 0x7fc3ec13c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f2b1c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ec0aa1e0 0x7fc3ec136a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f2b1c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec13d1a0 con 0x7fc3ec0aa1e0 2020-11-20T12:23:02.412 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f2b1c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec14e160 con 0x7fc3ec0a4330 2020-11-20T12:23:02.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f2b1c700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec14e380 con 0x7fc3ec0a9480 2020-11-20T12:23:02.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec0a4330 0x7fc3ec14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3ebfff700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0a9480 0x7fc3ec13c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec0a4330 0x7fc3ec14ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.44:35936/0 (socket says 172.21.15.44:35936) 2020-11-20T12:23:02.413 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 -- 172.21.15.44:0/1350765955 learned_addr learned my addr 172.21.15.44:0/1350765955 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:02.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0a9480 msgr2=0x7fc3ec13c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fc3ec0a9480 0x7fc3ec13c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ec0aa1e0 msgr2=0x7fc3ec136a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2020-11-20T12:23:02.414 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ec0aa1e0 0x7fc3ec136a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:23:02.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.404+0000 7fc3f08b8700 1 -- 172.21.15.44:0/1350765955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec14e5e0 con 0x7fc3ec0a4330 2020-11-20T12:23:02.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.405+0000 7fc3f08b8700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec0a4330 0x7fc3ec14ed20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc3e000ebd0 tx=0x7fc3e0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.406+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000ca60 con 0x7fc3ec0a4330 2020-11-20T12:23:02.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.406+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e000cbc0 con 0x7fc3ec0a4330 2020-11-20T12:23:02.415 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.406+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00123f0 con 0x7fc3ec0a4330 2020-11-20T12:23:02.417 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.407+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec14e880 con 0x7fc3ec0a4330 2020-11-20T12:23:02.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.407+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec1375e0 con 0x7fc3ec0a4330 2020-11-20T12:23:02.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.409+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7fc3e00127a0 con 0x7fc3ec0a4330 2020-11-20T12:23:02.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.409+0000 7fc3f2b1c700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1378c0 0x7fc3ec135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.418 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.409+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec13cdf0 con 0x7fc3ec1378c0 2020-11-20T12:23:02.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.409+0000 7fc3ebfff700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1378c0 0x7fc3ec135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.409+0000 7fc3e8ff9700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc3c4056d70 0x7fc3c4059230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.409+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7fc3e0070660 con 0x7fc3ec0a4330 2020-11-20T12:23:02.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.409+0000 7fc3f10b9700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc3c4056d70 0x7fc3c4059230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.419 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.410+0000 7fc3ebfff700 1 -- 172.21.15.44:0/1350765955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3ec14ea10 con 0x7fc3ec1378c0 2020-11-20T12:23:02.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.410+0000 7fc3f10b9700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc3c4056d70 0x7fc3c4059230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc3e4009fd0 tx=0x7fc3e4016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.410+0000 7fc3ebfff700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1378c0 0x7fc3ec135fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc3ec143320 tx=0x7fc3d8009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.410+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d800f500 con 0x7fc3ec1378c0 2020-11-20T12:23:02.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.411+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7fc3d800f660 con 0x7fc3ec1378c0 2020-11-20T12:23:02.420 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.411+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1378c0 msgr2=0x7fc3ec135fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.411+0000 7fc3e8ff9700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1378c0 0x7fc3ec135fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fc3ec143320 tx=0x7fc3d8009040).stop 2020-11-20T12:23:02.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.411+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1378c0 msgr2=0x7fc3ec135fb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:23:02.421 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.411+0000 7fc3e8ff9700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1378c0 0x7fc3ec135fb0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0x7fc3ec143320 tx=0x7fc3d8009040).stop 2020-11-20T12:23:02.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.423+0000 7fc3f2b1c700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1364a0 0x7fc3ec1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.426 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.423+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec1368b0 con 0x7fc3ec1364a0 2020-11-20T12:23:02.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.423+0000 7fc3ebfff700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1364a0 0x7fc3ec1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.427 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.424+0000 7fc3ebfff700 1 -- 172.21.15.44:0/1350765955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc3ec061650 con 0x7fc3ec1364a0 2020-11-20T12:23:02.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.424+0000 7fc3ebfff700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1364a0 0x7fc3ec1516e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc3ec14bfc0 tx=0x7fc3d8009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.428 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.424+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8014e90 con 0x7fc3ec1364a0 2020-11-20T12:23:02.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.427+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 <== mon.1 v2:172.21.15.79:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4266 (secure 0 0 0) 0x7fc3d8014e90 con 0x7fc3ec1364a0 2020-11-20T12:23:02.430 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.427+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1364a0 msgr2=0x7fc3ec1516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.427+0000 7fc3e8ff9700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1364a0 0x7fc3ec1516e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc3ec14bfc0 tx=0x7fc3d8009040).stop 2020-11-20T12:23:02.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.427+0000 7fc3e8ff9700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1364a0 msgr2=0x7fc3ec1516e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:23:02.431 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.427+0000 7fc3e8ff9700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec1364a0 0x7fc3ec1516e0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0x7fc3ec14bfc0 tx=0x7fc3d8009040).stop 2020-11-20T12:23:02.431 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:23:02.432 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:23:02.432 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:23:02.432 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:23:02.432 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 188, 2020-11-20T12:23:02.432 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:23:02.433 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:23:02.433 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:23:02.433 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:23:02.434 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.434 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:23:02.434 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:23:02.434 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:23:02.435 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:23:02.435 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.435 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.435 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.435 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.436 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.436 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.436 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.436 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.436 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.437 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:23:02.437 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:23:02.437 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.437 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.437 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.438 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.438 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.438 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.438 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.439 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.439 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.439 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:23:02.439 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:23:02.440 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:23:02.440 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:23:02.440 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:23:02.440 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:23:02.440 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:23:02.441 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:23:02.441 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:23:02.441 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:23:02.441 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:23:02.441 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:23:02.442 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:23:02.442 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:23:02.442 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:23:02.442 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.442 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.443 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.443 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.443 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.443 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.443 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.444 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.444 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.444 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:23:02.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.444 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:23:02.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.446 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:23:02.446 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:23:02.446 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.447 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.447 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.447 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:23:02.448 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.449 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:23:02.450 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.450 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.450 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:23:02.451 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:23:02.451 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.451 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.452 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:23:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:23:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.453 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:23:02.454 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.455 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.456 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:23:02.456 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.456 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.457 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.458 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:23:02.458 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:23:02.459 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.459 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.459 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.460 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:23:02.461 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:23:02.461 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.461 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.462 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.462 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:23:02.462 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.464 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.464 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:23:02.464 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.464 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.465 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.466 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:23:02.466 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:23:02.466 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.467 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.467 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.468 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.468 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:23:02.468 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:23:02.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.469 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.469 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.470 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:23:02.470 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.470 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.470 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:23:02.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.471 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.471 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.472 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:23:02.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.472 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.473 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:23:02.473 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:23:02.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.428+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc3c4056d70 msgr2=0x7fc3c4059230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.476 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.428+0000 7fc3f2b1c700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7fc3c4056d70 0x7fc3c4059230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc3e4009fd0 tx=0x7fc3e4016000).stop 2020-11-20T12:23:02.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.428+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec0a4330 msgr2=0x7fc3ec14ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.428+0000 7fc3f2b1c700 1 --2- 172.21.15.44:0/1350765955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc3ec0a4330 0x7fc3ec14ed20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc3e000ebd0 tx=0x7fc3e0006df0).stop 2020-11-20T12:23:02.477 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.428+0000 7fc3f10b9700 1 -- 172.21.15.44:0/1350765955 reap_dead start 2020-11-20T12:23:02.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.428+0000 7fc3f10b9700 1 -- 172.21.15.44:0/1350765955 reap_dead start 2020-11-20T12:23:02.478 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.429+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 shutdown_connections 2020-11-20T12:23:02.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.429+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 >> 172.21.15.44:0/1350765955 conn(0x7fc3ec062690 msgr2=0x7fc3ec1425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:02.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.429+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 shutdown_connections 2020-11-20T12:23:02.479 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.429+0000 7fc3f2b1c700 1 -- 172.21.15.44:0/1350765955 wait complete. 2020-11-20T12:23:02.480 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2020-11-20T12:23:02.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.656+0000 7f761c9fe700 1 Processor -- start 2020-11-20T12:23:02.660 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.657+0000 7f761c9fe700 1 -- start start 2020-11-20T12:23:02.661 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.657+0000 7f761c9fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7618140810 0x7f7618142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.657+0000 7f761c9fe700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76180a6260 con 0x7f7618060330 2020-11-20T12:23:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.657+0000 7f761c9fe700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76180a6490 con 0x7f761813a990 2020-11-20T12:23:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.657+0000 7f761c9fe700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76180a52e0 con 0x7f7618140810 2020-11-20T12:23:02.662 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.657+0000 7f761659c700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f7618060330 0x7f76180a4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47644/0 (socket says 172.21.15.44:47644) 2020-11-20T12:23:02.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f761659c700 1 -- 172.21.15.44:0/2439218353 learned_addr learned my addr 172.21.15.44:0/2439218353 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:23:02.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f7616d9d700 1 --2- 172.21.15.44:0/2439218353 >> v2:172.21.15.79:3300/0 conn(0x7f7618140810 0x7f7618142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f761559a700 1 -- 172.21.15.44:0/2439218353 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833796888 0 0) 0x7f76180a6260 con 0x7f7618060330 2020-11-20T12:23:02.663 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f761559a700 1 -- 172.21.15.44:0/2439218353 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75fc003240 con 0x7f7618060330 2020-11-20T12:23:02.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f7616d9d700 1 -- 172.21.15.44:0/2439218353 >> v1:172.21.15.79:6789/0 conn(0x7f761813a990 legacy=0x7f761813ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f7616d9d700 1 -- 172.21.15.44:0/2439218353 >> v1:172.21.15.44:6789/0 conn(0x7f7618060330 legacy=0x7f76180a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f7616d9d700 1 -- 172.21.15.44:0/2439218353 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180a6730 con 0x7f7618140810 2020-11-20T12:23:02.664 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.658+0000 7f7616d9d700 1 --2- 172.21.15.44:0/2439218353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7618140810 0x7f7618142c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7610009c90 tx=0x7f76100091d0).ready entity=mon.1 client_cookie=4a8321a6a420ff1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.659+0000 7f761559a700 1 -- 172.21.15.44:0/2439218353 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761000ba70 con 0x7f7618140810 2020-11-20T12:23:02.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.659+0000 7f761559a700 1 -- 172.21.15.44:0/2439218353 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761000bbd0 con 0x7f7618140810 2020-11-20T12:23:02.665 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.659+0000 7f761559a700 1 -- 172.21.15.44:0/2439218353 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011400 con 0x7f7618140810 2020-11-20T12:23:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.659+0000 7f761c9fe700 1 -- 172.21.15.44:0/2439218353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7618140810 msgr2=0x7f7618142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.659+0000 7f761c9fe700 1 --2- 172.21.15.44:0/2439218353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7618140810 0x7f7618142c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f7610009c90 tx=0x7f76100091d0).stop 2020-11-20T12:23:02.666 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.660+0000 7f761c9fe700 1 -- 172.21.15.44:0/2439218353 shutdown_connections 2020-11-20T12:23:02.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.660+0000 7f761c9fe700 1 --2- 172.21.15.44:0/2439218353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7618140810 0x7f7618142c00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.660+0000 7f761c9fe700 1 -- 172.21.15.44:0/2439218353 >> 172.21.15.44:0/2439218353 conn(0x7f7618062690 msgr2=0x7f76180a9910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:02.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.661+0000 7f761c9fe700 1 -- 172.21.15.44:0/2439218353 shutdown_connections 2020-11-20T12:23:02.667 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.661+0000 7f761c9fe700 1 -- 172.21.15.44:0/2439218353 wait complete. 2020-11-20T12:23:02.668 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.662+0000 7f761c9fe700 1 Processor -- start 2020-11-20T12:23:02.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.662+0000 7f761c9fe700 1 -- start start 2020-11-20T12:23:02.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761c9fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7618060330 0x7f76180aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761c9fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f761813a990 0x7f7618144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.669 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761c9fe700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618140810 0x7f7618140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761c9fe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76180ac4d0 con 0x7f7618060330 2020-11-20T12:23:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761c9fe700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76180ac770 con 0x7f761813a990 2020-11-20T12:23:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761c9fe700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76180aca10 con 0x7f7618140810 2020-11-20T12:23:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7618060330 0x7f76180aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.670 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7618060330 0x7f76180aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43902/0 (socket says 172.21.15.44:43902) 2020-11-20T12:23:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f761659c700 1 -- 172.21.15.44:0/2171949177 learned_addr learned my addr 172.21.15.44:0/2171949177 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f7616d9d700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618140810 0x7f7618140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.663+0000 7f7615d9b700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f761813a990 0x7f7618144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f7615d9b700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618140810 msgr2=0x7f7618140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f7615d9b700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618140810 0x7f7618140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f7615d9b700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7618060330 msgr2=0x7f76180aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f7615d9b700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7618060330 0x7f76180aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f7615d9b700 1 -- 172.21.15.44:0/2171949177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180ad490 con 0x7f761813a990 2020-11-20T12:23:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f7615d9b700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f761813a990 0x7f7618144020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f760c00c6e0 tx=0x7f760c007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.672 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c00e490 con 0x7f761813a990 2020-11-20T12:23:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c00e5f0 con 0x7f761813a990 2020-11-20T12:23:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76180ad620 con 0x7f761813a990 2020-11-20T12:23:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c00e970 con 0x7f761813a990 2020-11-20T12:23:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.664+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76180abe70 con 0x7f761813a990 2020-11-20T12:23:02.674 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.665+0000 7f761c9fe700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618128100 0x7f76181284e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.665+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7618061650 con 0x7f7618128100 2020-11-20T12:23:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.665+0000 7f761659c700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618128100 0x7f76181284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.675 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.666+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f760c010030 con 0x7f761813a990 2020-11-20T12:23:02.676 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.666+0000 7f76067fc700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75ec056d50 0x7f75ec059210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.666+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f760c06fd50 con 0x7f761813a990 2020-11-20T12:23:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.666+0000 7f7616d9d700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75ec056d50 0x7f75ec059210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.666+0000 7f761659c700 1 -- 172.21.15.44:0/2171949177 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76180446d0 con 0x7f7618128100 2020-11-20T12:23:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.666+0000 7f761659c700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618128100 0x7f76181284e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f7608031710 tx=0x7f7608007b60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.667+0000 7f7616d9d700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75ec056d50 0x7f75ec059210 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7610009210 tx=0x7f7610016000).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.667+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76080070d0 con 0x7f7618128100 2020-11-20T12:23:02.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.667+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8042 (secure 0 0 0) 0x7f7608007230 con 0x7f7618128100 2020-11-20T12:23:02.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.667+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618128100 msgr2=0x7f76181284e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.678 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.667+0000 7f76067fc700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618128100 0x7f76181284e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f7608031710 tx=0x7f7608007b60).stop 2020-11-20T12:23:02.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.667+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618128100 msgr2=0x7f76181284e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:23:02.679 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.667+0000 7f76067fc700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f7618128100 0x7f76181284e0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0x7f7608031710 tx=0x7f7608007b60).stop 2020-11-20T12:23:02.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.680+0000 7f761c9fe700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f761812c4c0 0x7f761812c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.680+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76180680f0 con 0x7f761812c4c0 2020-11-20T12:23:02.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.681+0000 7f761659c700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f761812c4c0 0x7f761812c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.685 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.681+0000 7f761659c700 1 -- 172.21.15.44:0/2171949177 --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76180613b0 con 0x7f761812c4c0 2020-11-20T12:23:02.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.681+0000 7f761659c700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f761812c4c0 0x7f761812c8d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7608031710 tx=0x7f7608007b60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.686 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.683+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.2 v2:172.21.15.79:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608033bf0 con 0x7f761812c4c0 2020-11-20T12:23:02.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.683+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 <== mon.2 v2:172.21.15.79:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f7608033d50 con 0x7f761812c4c0 2020-11-20T12:23:02.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.683+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f761812c4c0 msgr2=0x7f761812c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.683+0000 7f76067fc700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f761812c4c0 0x7f761812c8d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7608031710 tx=0x7f7608007b60).stop 2020-11-20T12:23:02.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.683+0000 7f76067fc700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f761812c4c0 msgr2=0x7f761812c8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2020-11-20T12:23:02.687 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.683+0000 7f76067fc700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f761812c4c0 0x7f761812c8d0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0x7f7608031710 tx=0x7f7608007b60).stop 2020-11-20T12:23:02.688 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2020-11-20T12:23:02.688 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:23:02.688 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:23:02.688 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "peon", 2020-11-20T12:23:02.689 INFO:teuthology.orchestra.run.smithi044.stdout: "election_epoch": 188, 2020-11-20T12:23:02.689 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum": [ 2020-11-20T12:23:02.689 INFO:teuthology.orchestra.run.smithi044.stdout: 0, 2020-11-20T12:23:02.689 INFO:teuthology.orchestra.run.smithi044.stdout: 1, 2020-11-20T12:23:02.690 INFO:teuthology.orchestra.run.smithi044.stdout: 2 2020-11-20T12:23:02.690 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.690 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_age": 3, 2020-11-20T12:23:02.690 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:23:02.690 INFO:teuthology.orchestra.run.smithi044.stdout: "required_con": "2449958747317026820", 2020-11-20T12:23:02.691 INFO:teuthology.orchestra.run.smithi044.stdout: "required_mon": [ 2020-11-20T12:23:02.691 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.691 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.691 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.691 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.692 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.692 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.692 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.692 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.692 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.693 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_con": "4540138297136906239", 2020-11-20T12:23:02.693 INFO:teuthology.orchestra.run.smithi044.stdout: "quorum_mon": [ 2020-11-20T12:23:02.693 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.693 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.694 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.694 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.694 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.694 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.694 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.695 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.695 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.695 INFO:teuthology.orchestra.run.smithi044.stdout: "outside_quorum": [], 2020-11-20T12:23:02.695 INFO:teuthology.orchestra.run.smithi044.stdout: "extra_probe_peers": [], 2020-11-20T12:23:02.696 INFO:teuthology.orchestra.run.smithi044.stdout: "sync_provider": [], 2020-11-20T12:23:02.696 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": { 2020-11-20T12:23:02.696 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 1, 2020-11-20T12:23:02.696 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "0437e350-ccfe-424c-9e91-59ff58800c77", 2020-11-20T12:23:02.696 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:23:02.697 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2020-11-20T12:02:06.403922Z", 2020-11-20T12:23:02.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release": 16, 2020-11-20T12:23:02.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min_mon_release_name": "pacific", 2020-11-20T12:23:02.697 INFO:teuthology.orchestra.run.smithi044.stdout: "election_strategy": 1, 2020-11-20T12:23:02.698 INFO:teuthology.orchestra.run.smithi044.stdout: "disallowed_leaders: ": "", 2020-11-20T12:23:02.698 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false, 2020-11-20T12:23:02.698 INFO:teuthology.orchestra.run.smithi044.stdout: "features": { 2020-11-20T12:23:02.698 INFO:teuthology.orchestra.run.smithi044.stdout: "persistent": [ 2020-11-20T12:23:02.698 INFO:teuthology.orchestra.run.smithi044.stdout: "kraken", 2020-11-20T12:23:02.699 INFO:teuthology.orchestra.run.smithi044.stdout: "luminous", 2020-11-20T12:23:02.699 INFO:teuthology.orchestra.run.smithi044.stdout: "mimic", 2020-11-20T12:23:02.699 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap-prune", 2020-11-20T12:23:02.699 INFO:teuthology.orchestra.run.smithi044.stdout: "nautilus", 2020-11-20T12:23:02.699 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus", 2020-11-20T12:23:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific", 2020-11-20T12:23:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: "elector-pinging" 2020-11-20T12:23:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: "optional": [] 2020-11-20T12:23:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: "mons": [ 2020-11-20T12:23:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 0, 2020-11-20T12:23:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "a", 2020-11-20T12:23:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.702 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.702 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:3300", 2020-11-20T12:23:02.702 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.703 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789", 2020-11-20T12:23:02.703 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.704 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.704 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6789/0", 2020-11-20T12:23:02.704 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6789/0", 2020-11-20T12:23:02.705 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.705 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.705 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.706 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 1, 2020-11-20T12:23:02.706 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "b", 2020-11-20T12:23:02.706 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.706 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.707 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3300", 2020-11-20T12:23:02.707 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.708 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789", 2020-11-20T12:23:02.708 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.708 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6789/0", 2020-11-20T12:23:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6789/0", 2020-11-20T12:23:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.709 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "rank": 2, 2020-11-20T12:23:02.710 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "c", 2020-11-20T12:23:02.711 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2020-11-20T12:23:02.711 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2020-11-20T12:23:02.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2020-11-20T12:23:02.712 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:3301", 2020-11-20T12:23:02.712 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2020-11-20T12:23:02.713 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790", 2020-11-20T12:23:02.713 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 0 2020-11-20T12:23:02.713 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.713 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.714 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.79:6790/0", 2020-11-20T12:23:02.714 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.79:6790/0", 2020-11-20T12:23:02.714 INFO:teuthology.orchestra.run.smithi044.stdout: "priority": 0, 2020-11-20T12:23:02.714 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 0, 2020-11-20T12:23:02.715 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "{}" 2020-11-20T12:23:02.715 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.715 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.715 INFO:teuthology.orchestra.run.smithi044.stdout: "feature_map": { 2020-11-20T12:23:02.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mon": [ 2020-11-20T12:23:02.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.716 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.716 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.717 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:23:02.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.717 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": [ 2020-11-20T12:23:02.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.718 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.718 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.718 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:23:02.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.718 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.719 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": [ 2020-11-20T12:23:02.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.719 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.719 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.719 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 1 2020-11-20T12:23:02.720 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.720 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2020-11-20T12:23:02.720 INFO:teuthology.orchestra.run.smithi044.stdout: "client": [ 2020-11-20T12:23:02.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2020-11-20T12:23:02.720 INFO:teuthology.orchestra.run.smithi044.stdout: "features": "0x3f01cfb9fffdffff", 2020-11-20T12:23:02.721 INFO:teuthology.orchestra.run.smithi044.stdout: "release": "luminous", 2020-11-20T12:23:02.721 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 2 2020-11-20T12:23:02.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2020-11-20T12:23:02.721 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2020-11-20T12:23:02.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2020-11-20T12:23:02.722 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": false 2020-11-20T12:23:02.722 INFO:teuthology.orchestra.run.smithi044.stdout:} 2020-11-20T12:23:02.723 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.685+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75ec056d50 msgr2=0x7f75ec059210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.685+0000 7f761c9fe700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f75ec056d50 0x7f75ec059210 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7610009210 tx=0x7f7610016000).stop 2020-11-20T12:23:02.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.685+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f761813a990 msgr2=0x7f7618144020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.685+0000 7f761c9fe700 1 --2- 172.21.15.44:0/2171949177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f761813a990 0x7f7618144020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f760c00c6e0 tx=0x7f760c007b70).stop 2020-11-20T12:23:02.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.685+0000 7f7616d9d700 1 -- 172.21.15.44:0/2171949177 reap_dead start 2020-11-20T12:23:02.724 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.685+0000 7f7616d9d700 1 -- 172.21.15.44:0/2171949177 reap_dead start 2020-11-20T12:23:02.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.686+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 shutdown_connections 2020-11-20T12:23:02.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.686+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 >> 172.21.15.44:0/2171949177 conn(0x7f7618062690 msgr2=0x7f76180a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:02.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.686+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 shutdown_connections 2020-11-20T12:23:02.725 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.686+0000 7f761c9fe700 1 -- 172.21.15.44:0/2171949177 wait complete. 2020-11-20T12:23:02.726 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2020-11-20T12:23:02.726 INFO:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2020-11-20T12:23:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.933+0000 7f2a37539700 1 Processor -- start 2020-11-20T12:23:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.933+0000 7f2a37539700 1 -- start start 2020-11-20T12:23:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.934+0000 7f2a37539700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2a300a86d0 0x7f2a300a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.934+0000 7f2a37539700 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a301385f0 con 0x7f2a300aad00 2020-11-20T12:23:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.934+0000 7f2a37539700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a300aca50 con 0x7f2a300ab0e0 2020-11-20T12:23:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.934+0000 7f2a37539700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3013e250 con 0x7f2a300a86d0 2020-11-20T12:23:02.938 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.934+0000 7f2a352d5700 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2a300aad00 0x7f2a300abc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:47662/0 (socket says 172.21.15.44:47662) 2020-11-20T12:23:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.934+0000 7f2a352d5700 1 -- 172.21.15.44:0/818385450 learned_addr learned my addr 172.21.15.44:0/818385450 (peer_addr_for_me v1:172.21.15.44:0/0) 2020-11-20T12:23:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.934+0000 7f2a35ad6700 1 --2- 172.21.15.44:0/818385450 >> v2:172.21.15.79:3300/0 conn(0x7f2a300a86d0 0x7f2a300a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a1ffff700 1 -- 172.21.15.44:0/818385450 <== mon.1 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211889881 0 0) 0x7f2a300aca50 con 0x7f2a300ab0e0 2020-11-20T12:23:02.939 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a35ad6700 1 -- 172.21.15.44:0/818385450 >> v1:172.21.15.79:6789/0 conn(0x7f2a300ab0e0 legacy=0x7f2a300ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a35ad6700 1 -- 172.21.15.44:0/818385450 >> v1:172.21.15.44:6789/0 conn(0x7f2a300aad00 legacy=0x7f2a300abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.940 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a35ad6700 1 -- 172.21.15.44:0/818385450 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3013e4f0 con 0x7f2a300a86d0 2020-11-20T12:23:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a35ad6700 1 --2- 172.21.15.44:0/818385450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300a86d0 0x7f2a300a94d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f2a2c00b9a0 tx=0x7f2a2c006df0).ready entity=mon.1 client_cookie=2a95d997fdaa5ceb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.941 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a1ffff700 1 -- 172.21.15.44:0/818385450 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0096b0 con 0x7f2a300a86d0 2020-11-20T12:23:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a1ffff700 1 -- 172.21.15.44:0/818385450 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c009810 con 0x7f2a300a86d0 2020-11-20T12:23:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.935+0000 7f2a1ffff700 1 -- 172.21.15.44:0/818385450 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c009b90 con 0x7f2a300a86d0 2020-11-20T12:23:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.936+0000 7f2a37539700 1 -- 172.21.15.44:0/818385450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300a86d0 msgr2=0x7f2a300a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.936+0000 7f2a37539700 1 --2- 172.21.15.44:0/818385450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300a86d0 0x7f2a300a94d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f2a2c00b9a0 tx=0x7f2a2c006df0).stop 2020-11-20T12:23:02.942 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.936+0000 7f2a37539700 1 -- 172.21.15.44:0/818385450 shutdown_connections 2020-11-20T12:23:02.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.936+0000 7f2a37539700 1 --2- 172.21.15.44:0/818385450 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300a86d0 0x7f2a300a94d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.936+0000 7f2a37539700 1 -- 172.21.15.44:0/818385450 >> 172.21.15.44:0/818385450 conn(0x7f2a30060170 msgr2=0x7f2a30060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:02.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.937+0000 7f2a37539700 1 -- 172.21.15.44:0/818385450 shutdown_connections 2020-11-20T12:23:02.943 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.937+0000 7f2a37539700 1 -- 172.21.15.44:0/818385450 wait complete. 2020-11-20T12:23:02.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.938+0000 7f2a37539700 1 Processor -- start 2020-11-20T12:23:02.944 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.938+0000 7f2a37539700 1 -- start start 2020-11-20T12:23:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a37539700 1 --2- >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a300a86d0 0x7f2a3013a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.945 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a37539700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a300aad00 0x7f2a300a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a37539700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300ab0e0 0x7f2a30137850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a37539700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a30065780 con 0x7f2a300aad00 2020-11-20T12:23:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a37539700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a30065a20 con 0x7f2a300ab0e0 2020-11-20T12:23:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a37539700 1 -- --> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300a7610 con 0x7f2a300a86d0 2020-11-20T12:23:02.946 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a34ad4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a300aad00 0x7f2a300a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a34ad4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a300aad00 0x7f2a300a7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43920/0 (socket says 172.21.15.44:43920) 2020-11-20T12:23:02.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a34ad4700 1 -- 172.21.15.44:0/877979055 learned_addr learned my addr 172.21.15.44:0/877979055 (peer_addr_for_me v2:172.21.15.44:0/0) 2020-11-20T12:23:02.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a352d5700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a300a86d0 0x7f2a3013a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.947 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.939+0000 7f2a35ad6700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300ab0e0 0x7f2a30137850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a34ad4700 1 -- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a300a86d0 msgr2=0x7f2a3013a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a34ad4700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a300a86d0 0x7f2a3013a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a34ad4700 1 -- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300ab0e0 msgr2=0x7f2a30137850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a34ad4700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300ab0e0 0x7f2a30137850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:02.948 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a34ad4700 1 -- 172.21.15.44:0/877979055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300a5fd0 con 0x7f2a300aad00 2020-11-20T12:23:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a35ad6700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300ab0e0 0x7f2a30137850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:23:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a34ad4700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a300aad00 0x7f2a300a7ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f2a200096e0 tx=0x7f2a20006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a1d7fa700 1 -- 172.21.15.44:0/877979055 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000d490 con 0x7f2a300aad00 2020-11-20T12:23:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a1d7fa700 1 -- 172.21.15.44:0/877979055 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a20000e90 con 0x7f2a300aad00 2020-11-20T12:23:02.949 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3013ac60 con 0x7f2a300aad00 2020-11-20T12:23:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a1d7fa700 1 -- 172.21.15.44:0/877979055 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000d820 con 0x7f2a300aad00 2020-11-20T12:23:02.950 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.940+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3013b3b0 con 0x7f2a300aad00 2020-11-20T12:23:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.943+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a300a77d0 con 0x7f2a300aad00 2020-11-20T12:23:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.943+0000 7f2a1d7fa700 1 -- 172.21.15.44:0/877979055 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) v1 ==== 67517+0+0 (secure 0 0 0) 0x7f2a2000f070 con 0x7f2a300aad00 2020-11-20T12:23:02.951 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.944+0000 7f2a1d7fa700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a08056d20 0x7f2a080591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:23:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.944+0000 7f2a1d7fa700 1 -- 172.21.15.44:0/877979055 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f2a2006fab0 con 0x7f2a300aad00 2020-11-20T12:23:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.944+0000 7f2a352d5700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a08056d20 0x7f2a080591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:23:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.945+0000 7f2a352d5700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a08056d20 0x7f2a080591e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f2a28007fd0 tx=0x7f2a28009040).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:23:02.952 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:02.948+0000 7f2a1d7fa700 1 -- 172.21.15.44:0/877979055 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2a20044c70 con 0x7f2a300aad00 2020-11-20T12:23:02.956 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at 38dbaa5175 v0.16 2020-11-20T12:23:02.960 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.16 2020-11-20T12:23:03.166 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:23:03.166 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.17 2020-11-20T12:23:03.167 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.17 2020-11-20T12:23:03.167 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.17 2020-11-20T12:23:03.167 INFO:tasks.workunit.client.0.smithi044.stdout:v0.17 2020-11-20T12:23:03.222 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.219+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f2a30069680 con 0x7f2a300aad00 2020-11-20T12:23:03.223 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.220+0000 7f2a1d7fa700 1 -- 172.21.15.44:0/877979055 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f2a20048290 con 0x7f2a300aad00 2020-11-20T12:23:03.225 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a08056d20 msgr2=0x7f2a080591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a08056d20 0x7f2a080591e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f2a28007fd0 tx=0x7f2a28009040).stop 2020-11-20T12:23:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a300aad00 msgr2=0x7f2a300a7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:23:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a300aad00 0x7f2a300a7ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f2a200096e0 tx=0x7f2a20006d90).stop 2020-11-20T12:23:03.226 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 shutdown_connections 2020-11-20T12:23:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:6832/24383,v1:172.21.15.79:6833/24383] conn(0x7f2a08056d20 0x7f2a080591e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a300ab0e0 0x7f2a30137850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7f2a300a86d0 0x7f2a3013a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 --2- 172.21.15.44:0/877979055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a300aad00 0x7f2a300a7ab0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:23:03.227 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 >> 172.21.15.44:0/877979055 conn(0x7f2a30060170 msgr2=0x7f2a300b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:23:03.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 shutdown_connections 2020-11-20T12:23:03.228 INFO:teuthology.orchestra.run.smithi044.stderr:2020-11-20T12:23:03.222+0000 7f2a37539700 1 -- 172.21.15.44:0/877979055 wait complete. 2020-11-20T12:23:03.242 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2020-11-20T12:23:05.349 INFO:tasks.workunit.client.0.smithi044.stderr:Checking out files: 80% (175/217) Checking out files: 81% (176/217) Checking out files: 82% (178/217) Checking out files: 83% (181/217) Checking out files: 84% (183/217) Checking out files: 85% (185/217) Checking out files: 86% (187/217) Checking out files: 87% (189/217) Checking out files: 88% (191/217) Checking out files: 89% (194/217) Checking out files: 90% (196/217) Checking out files: 91% (198/217) Checking out files: 92% (200/217) Checking out files: 93% (202/217) Checking out files: 94% (204/217) Checking out files: 95% (207/217) Checking out files: 96% (209/217) Checking out files: 97% (211/217) Checking out files: 98% (213/217) Checking out files: 99% (215/217) Checking out files: 100% (217/217) Checking out files: 100% (217/217), done. 2020-11-20T12:23:05.538 INFO:tasks.workunit.client.0.smithi044.stdout:HEAD is now at dac3a8c35f v0.17 2020-11-20T12:23:05.544 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir .snap/v0.17 2020-11-20T12:23:05.821 INFO:tasks.workunit.client.0.smithi044.stderr:+ for v in $versions 2020-11-20T12:23:05.822 INFO:tasks.workunit.client.0.smithi044.stderr:+ ver=v0.18 2020-11-20T12:23:05.822 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo v0.18 2020-11-20T12:23:05.822 INFO:tasks.workunit.client.0.smithi044.stderr:+ git reset --hard v0.18 2020-11-20T12:23:05.823 INFO:tasks.workunit.client.0.smithi044.stdout:v0.18